Turbotodd

Ruminations on tech, the digital media, and some golf thrown in for good measure.

Posts Tagged ‘deep blue

Live @ Information On Demand 2012: A Q&A With Nate Silver On The Promise Of Prediction

with 2 comments

Day 3 at Information On Demand 2012.

The suggestion to “Think Big” continued, so Scott Laningham and I sat down very early this morning with Nate Silver, blogger and author of the now New York Times bestseller, “The Signal and the Noise” (You can read the review of the book in the Times here).

Nate, who is a youngish 34, has become our leading statistician through his innovative analyses of political polling, but made his original name by building a widely acclaimed baseball statistical analysis system called “PECOTA.”

Today, Nate runs the award-winning political website FiveThirtyEight.com, which is now published in The New York Times and which has made Nate the public face of statistical analysis and political forecasting.

In his book, the full title of which is “The Signal and The Noise: Why Most Predictions Fail — But Some Don’t,” Silver explores how data-based predictions underpin a growing sector of critical fields, from political polling to weather forecasting to the stock market to chess to the war on terror.

In the book, Nate poses some key questions, including what kind of predictions can we trust, and are the “predicters” using reliable methods? Also, what sorts of things can, and cannot, be predicted?

In our conversation in the greenroom just prior to his keynote at Information On Demand 2012 earlier today, Scott and I probed along a number of these vectors, asking Nate about the importance of prediction in Big Data, statistical influence on sports and player predictions (a la “Moneyball”), how large organizations can improve their predictive capabilities, and much more.

It was a refreshing and eye-opening interview, and I hope you enjoy watching it as much as Scott and I enjoyed conducting it!

Happy Anniversary, Deep Blue

leave a comment »

It was fifteen years ago today that the IBM chess-playing supercomputer, Deep Blue, beat he-who-shall-remain-nameless, a world grandmaster, after a six-game match, which brought two wins for IBM, one for the world champion, and three draws.

On May 11, 1997, an IBM computer called IBM Deep Blue beat the world chess champion after a six-game match: two wins for IBM, one for the champion and three draws. The match lasted several days and received massive media coverage around the world. It was the classic plot line of man vs. machine. Behind the contest, however, was important computer science, pushing forward the ability of computers to handle the kinds of complex calculations needed to take computing to the its next stage of evolution.

It was classic man-versus-machine, but underlying the mythology that enveloped the John Henry storyline was something far more important: The opportunity to push the frontiers of computer science, to push computers to handle the kind of complex calculations necessary for helping discover new pharmaceuticals; to conduct the kind of financial modeling needed to identify trends and do risk analysis; to perform the kinds of massive calculations needed in many fields of science.

Solving The Problem That Is Chess

Since artificial intelligence emerged as a concept along with the first real computers in the 1940s, computer scientists compared the performance of these “giant brains” with human minds, and many gravitated to chess as a way of testing the calculating abilities of computers. Chess is a game that represents a collection of challenging problems for minds and machines, but had simple rules, and was thus an excellent testbed for laying the groundwork for the “big data” era that was soon to come.

There’s but no question that Deep Blue was such a powerful computer programmed to solve the complex, strategic game of chess.  But IBM’s goal was far deeper: To enable researchers to discover and understand the limits and opportunities presented by massively parallel processing and high performance computing.

IBM Deep Blue: Analyzing 200 Million Chess Positions Per Second

If, in fact, Deep Blue could explore up to 200 million possible chess positions per second, then could this deep computing capability be used to help society handle the kinds of complex calculations required in some of these other aforementioned areas.

Deep Blue did, in fact, prove that industry could tackle these issues with smart algorithms and sufficient computational power.

I recalled earlier this year in a blog post my own experience witnessing the Deep Blue chess match.  It evoked a lot of nostalgia for me and so many others.

IBM’s Deep Blue supercomputer could explore up to 200 million possible chess positions per second on 510 processors. Juxtapose that with IBM Blue Gene’s ability a few short years later to to routinely handle 478 trillion operations every second!

But it also laid a foundation, paving the way for new kinds of advanced computers and breakthroughs, including IBM’s Blue Gene and, later, IBM Watson.

Forever In Blue Genes

Blue Gene, introduced in 2004, demonstrated the next grand challenge in computing and was both the most powerful supercomputer and the most efficient, but was built to help biologists observe the invisible processes of protein folding and gene development. Deep Blue was also one of the earliest experiments in supercomputing that propelled IBM to become a market leader in this space to this day.

Fifteen years on, we’ve seen epic growth in the volume and variety of data being generated around the planet, via business, the social media, new sensor data helping with instrumentation of the physical world vis-a-vis IBM’s smarter planet initiative.  We’ve created so much new data that, in fact, 90% of the data in the world today was created in the last two years alone!

Calling Doctor Watson

Most recently, IBM embarked upon the next wave of this computing progress through the development of IBM’s Watson, which can hold the equivalent of about one million books worth of information. But make no mistake, Watson’s significance wasn’t just the amount of information it could process, but rather, a new generation of technology that uses algorithms to find answers in unstructured data more effectively than standard search technology, while also “understanding” natural language.

The promise of IBM Watson is now being put to productive use in industry — as an online tool to assist medical professionals in formulating diagnoses; by simplifying the banking experience by analyzing customer needs in the context of vast amounts of ever-changing financial , economic, product, and client data; and, I’m sure, other industries near you soon.

Those early chess matches were exciting, nail-biting even (and who’d have thought we’d ever say that about chess?)! But they pale by comparison to the productive work and problem-solving IBM’s Watson, and other IBM technologies, are now and will continue to be involved with as the world of big data matures and becomes adopted by an ever-increasing audience.

You can now visit Deep Blue, which ultimately was retired to the Smithsonian Museum in Washington, D.C.

But its groundbreaking contributions to artificial intelligence and computing in general continues, and now extends well beyond the confines of the chess board.

Deep Blue Anniversary

with 2 comments

The Atlantic Monthly online reminds us that it was sixteen years ago today that world chess grandmaster Garry Kasparov sat down to play the sixth game of his match against IBM’s Deep Blue supercomputer.  Kasparov won that match, three games, drawing in two, and losing one.

Garry Kasparov, right, faces off against IBM's Deep Blue in his final match of a six-game tournament on this day in 1996.

I recall in this December 2010 post what happened the following year.

 

 

Written by turbotodd

February 17, 2012 at 9:47 pm

Watson, Come Here, I Need You

leave a comment »

Did you hear the on about the IBM computer that played the game show “Jeopardy” and beat two former world champion human “Jeopardy” players?

If not, you missed the outcome from yesterday’s practice round, where Watson won a practice round $4,400 to Ken Jennings $3,400 and Brad Rutter’s $1,200.

This was just a precursor to the final rounds airing on February 14-16, but after watching some video footage of the action yesterday, it was eerily reminiscent of those chess matches in 1997 that shall go unnamed. Only this time, I think the computer is actually starting to think.  And sdpeak.

Me, I’m not going anywhere near it. I’ve already got enough machines beating up on me in my life: my iPad, my iPhone 4, my new AT&T U-Verse umbilical cord to the world.

But I would lie if I were to say I weren’t anxious to see the final results of the Watson/Jeopardy mano a mano competition. Latest odds from our Watson/Jeopardy poll give Watson the edge at 52%, with Ken Jennings at 40% and Brad Rutter at 7% (NOT a scientific poll, by any means…just a polling pulse of the crowd via our Web site.)

Which, by the way, I would strongly encourage you to visit and spend some time with before the official matches air in February.  My colleagues have been busting their humps to produce some great video content explaining this story in a way we mere mortals can understand, and beyond all the hyperbole, explaining the potential ramifications of this breakthrough in artificial intelligence.

Me, I’m finishing my week off here in a frigid NYC, meeting with colleagues and thinking about gridiron action this weekend.  I’m gonna go out on a limb and pick the following: Pittsburgh over Baltimore, New England over the Jets, Chicago over Seattle, and Green Bay over Atlanta.

That’s not necessarily all of which I want to happen.  I’m just picking what I think might happen.  Then again, I’m a Dallas Cowboys fan – do keep that in mind.

Before I flee the wintry cold of the northeast, let me also remind you of the forthcoming Lotusphere 2011 event in sunny Orlando, Florida.  I’ll be jetting that way in a couple of weeks to blog and podvodcast from there with mi developerWorks amigo, Scott Laningham.

Note that this year we’re also seeing the introduction of the Lotusphere Social Business Summit, where C-level execs and other senior leaders from sales, marketing, operations, and other non-IT related functions can come together and learn how social business practices can help boost their bottom lines.  And if last fall’s IBM Industry Summit in Barcelona was any indication, those attending are going to be in for a real treat.

You can learn more about the Social Business Summit here.

P.S. Tweet @turbotodd and let me know the origination of today’s headline!

Written by turbotodd

January 14, 2011 at 4:37 pm

Natural Language Processing For $500, Alex

with one comment

I recently provided a personal remembrance of the Kasparov v. Deep Blue chess matches of the late 1990s, the IBM “John Henry” contest between a Russian chess grandmaster and an IBM supercomputer.

At the end of the post, per the custom of American TV game show Jeopardy!,”I posed the answer with a question: “What is Watson?”

Here’s your expanded answer: Watson is a new supercomputer, named after founder Thomas J. Watson, and programmed once again by a set of IBM Researchers, this time to compete on the longstanding game show Jeopardy! against the show’s two most successful and celebrated contestants — Ken Jennings and Brad Rutter.

The first-ever man vs. machine Jeopardy! competition will air on February 14, 15 and 16, 2011, with two matches being played over three consecutive days.

Watson was built by a team of IBM scientists who set out to accomplish a grand challenge –- build a computing system that rivals a human’s ability to answer questions posed in natural language with speed, accuracy and confidence.

The Jeopardy! format provides the ultimate challenge because the game’s clues involve analyzing subtle meaning, irony, riddles, and other complexities in which humans excel and computers traditionally do not.

Competing against Watson will be two of the most celebrated players ever to appear on Jeopardy! Ken Jennings broke the Jeopardy! record for the most consecutive games played by winning 74 games in a row during the 2004-2005 season, resulting in winnings of more than $2.5 million.

World-class Jeopardy! players Ken Jennings and Brad Rutter will test the mettle of IBM's Watson supercomputer in February 2011.

Brad Rutter won the highest cumulative amount ever by a single Jeopardy! player, earning $3,255,102. The total amount is a combination of Rutter’s original appearance in 2002, plus three Tournament wins:  the “Tournament of Champions” and the “Million Dollar Masters Tournament” in 2002 and the “Ultimate Tournament of Champions” in 2005.

Artificial Intelligence That Could Save Humanity For $300

The grand prize for this competition will be $1 million with second place earning $300,000 and third place $200,000. Rutter and Jennings will donate 50 percent of their winnings to charity and IBM will donate 100 percent of its winnings to charity.

“After four years, our scientific team believes that Watson is ready for this challenge based on its ability to rapidly comprehend what the Jeopardy! clue is asking, analyze the information it has access to, come up with precise answers, and develop an accurate confidence in its response,” said Dr. David Ferrucci, the scientist leading the IBM Research team that has created Watson.

Dr. David Dr. David Ferucci is the principal investigator and team lead for the DeepQA/Watson system that will be challenging the world-class Jeopardy! champions.

Beyond our excitement for the match itself, our team is very motivated by the possibilities that Watson’s breakthrough computing capabilities hold for building a smarter planet and helping people in their business tasks and personal lives.”

“We’re thrilled that Jeopardy! is considered a benchmark of ultimate knowledge,” said Harry Friedman, Executive Producer of Jeopardy!. “Performing well on Jeopardy! requires a combination of skills, and it will be fascinating to see whether a computer can compete against arguably the two best Jeopardy! players ever.”

Watson: Spars In More Than 50 Games To Prep For The Best

This past fall, Watson played more than 50 “sparring games” against former Jeopardy! Tournament of Champions contestants in final preparation for its television debut.

In addition, Watson has taken and passed the same Jeopardy! contestant test that humans take to qualify to play on the show, giving Jeopardy! producers confidence that the match will be both entertaining and competitive.

You can see highlights of those sparring matches here.

Real World Applications for Watson’s Technology

One of the criticisms that emanated during and after the Deep Blue chess matches was that IBM didn’t demonstrate the real-world applicability of the technology behind the chess moves.

This time around, IBM is working to answer more of those kinds of questions up front.

To be sure, beyond Jeopardy!, the technology behind Watson can be adapted to solve problems and drive progress in various fields (I’m already trying to understand how we could leverage it on our Web site!)

The Watson computer has the ability to sift through vast amounts of data and return precise answers, ranking its confidence in its answers. Such technology could be applied to a whole range of industries: healthcare, to more accurately diagnose patients based on empirical data; tech, to improve online health desks; tourism, to help provide tourists with information about cities; customer service, to improve prompting and directing customer support inquiries via phone and web…the list goes on.

Yes, But What Is Watson?

Watson is a breakthrough human achievement in the scientific field of Question and Answering, also known as “QA.” The Watson software is powered by an IBM POWER7 server optimized to handle the massive number of tasks that Watson must perform at rapid speeds to analyze complex language and deliver correct responses to Jeopardy! clues.

The system incorporates a number of proprietary technologies for the specialized demands of processing an enormous number of concurrent tasks and data while analyzing information in real time.

About Jeopardy!

Jeopardy!, the winner of 28 Emmy awards since its syndicated debut in 1984, is in the Guinness Book of World Records for the most awards won by a TV Game Show. The series is the #1-rated quiz show in syndication with nearly 9 million daily viewers. Jeopardy! is produced by Sony Pictures Television, a Sony Pictures Entertainment Company. It is distributed domestically by CBS Television Distribution and internationally by CBS Television International, both units of CBS Corp.

For more information about Jeopardy!, visit www.Jeopardy.com

To learn more about Watson and to view a video series about the DeepQA technology powering Watson, please visit www.ibmwatson.com.

You can also join the social discussion about Watson (be sure to include the hashtag #ibmwatson in your Tweets!)

If you want, you can also become Watson’s friend on Facebook at http://www.facebook.com/ibmwatson.

Even supercomputers need friends.

Written by turbotodd

December 14, 2010 at 2:23 pm

%d bloggers like this: