The Vindication Of Nate Silver

I was all set to write a closer examination of statistician and blogger Nate Silver’s most recent election predictions, a ramp up to during which he was lambasted by a garden variety of mostly conservative voices for either being politically biased, or establishing his predictions on a loose set of statistical shingles.

Only to be informed that one of my esteemed colleagues, David Pittman, had already written such a compendium post.  So hey, why reinvent the Big Data prediction wheel?

Here’s a link to David’s fine post, which I encourage you to check out if you want to get a sense of how electoral predictions provide an excellent object lesson for the state of Big Data analysis. (David’s post also includes the on-camera interview that Scott Laningham and I conducted with Nate Silver just prior to his excellent keynote before the gathered IBM Information On Demand 2012 crowd.)

I’m also incorporating a handful of other stories I have run across that I think do a good job of helping people better understand the inflection point for data-driven forecasting that Silver’s recent endeavor represents, along with its broader impact in media and punditry.

They are as follows:

 “Nate Silver’s Big Data Lessons for the Enterprise”

 “What Nate Silver’s success says about the 4th and 5th estates”

“Election 2012: Has Nate Silver destroyed punditry?” 

Nate Silver After the Election: The Verdict

As Forbes reporter wrote in his own post about Silver’s predictions, “the modelers are here to stay.”

Moving forward, I expect we’ll inevitably see an increased capability for organizations everywhere to adopt Silver’s methodical, Bayesian analytical strategies…and well beyond the political realm.

Live @ Information On Demand 2012: Craig Rhinehart On Predictive Healthcare

I made it back to Austin late last night, mostly no worse for the wear.

There were a number of key announcements made at Information On Demand 2012 over the course of the past few days in Las Vegas.

One of those that I mentioned in one of my keynote post summaries was IBM Patient Care and Insights, new analytics software based on innovations from IBM Labs that helps healthcare organizations improve patient care and lower operational costs by considering the specific health history of each individual patient.

This is a fascinating new capability with profound implications for healthcare providers.

The new IBM solution provides the core capabilities for devising predictive models of various health conditions, which can be used to identify early intervention opportunities to improve the patient’s outlook by minimizing or avoiding potential health problems.

It features advanced analytics and care management capabilities to help identify early intervention opportunities and coordinate patient care.

Providing Individualized Care

At the core of IBM Patient Care and Insights, developed by IBM’s software, research and services teams, are similarity analytics that help drive smart, individualized care delivery.

Born in IBM Research, IBM similarity analytics is a set of core capabilities and algorithms that allow healthcare professionals to examine thousands of patient characteristics at once — including demographic, social, clinical and financial factors along with unstructured data such as physicians’ notes — to generate personalized evidence and insights, and then provide care according to personalized treatment plans.

By way of example, physicians can make personalized recommendations to improve a patient’s outcome by finding other patients with similar clinical characteristics to see what treatments were most effective or what complications they may have encountered.

They can also perform patient-physician matching so an individual is paired with a doctor that is optimal for a specific condition. With this solution, caregivers can better tap into the collective memory of the care delivery system to uncover new levels of tailored insight or “early identifiers” from historical/long term patient data that enable doctors and others to help manage a patient’s healthcare needs well into the future.

Craig Rhinehart, director for IBM’s ECM Strategy and Market Development organization, sat down with Scott Laningham and I earlier this week to describe the challenges facing health care, and how the IBM Patient Care and Insights can help improve health care by delivering dynamic case-based, patient-centric electronic care plans and population analysis.

Go here for more information on IBM Patient Care and Insights and IBM Intelligent Investigation Manager.

Live @ Information On Demand 2012: A Q&A With Nate Silver On The Promise Of Prediction

Day 3 at Information On Demand 2012.

The suggestion to “Think Big” continued, so Scott Laningham and I sat down very early this morning with Nate Silver, blogger and author of the now New York Times bestseller, “The Signal and the Noise” (You can read the review of the book in the Times here).

Nate, who is a youngish 34, has become our leading statistician through his innovative analyses of political polling, but made his original name by building a widely acclaimed baseball statistical analysis system called “PECOTA.”

Today, Nate runs the award-winning political website FiveThirtyEight.com, which is now published in The New York Times and which has made Nate the public face of statistical analysis and political forecasting.

In his book, the full title of which is “The Signal and The Noise: Why Most Predictions Fail — But Some Don’t,” Silver explores how data-based predictions underpin a growing sector of critical fields, from political polling to weather forecasting to the stock market to chess to the war on terror.

In the book, Nate poses some key questions, including what kind of predictions can we trust, and are the “predicters” using reliable methods? Also, what sorts of things can, and cannot, be predicted?

In our conversation in the greenroom just prior to his keynote at Information On Demand 2012 earlier today, Scott and I probed along a number of these vectors, asking Nate about the importance of prediction in Big Data, statistical influence on sports and player predictions (a la “Moneyball”), how large organizations can improve their predictive capabilities, and much more.

It was a refreshing and eye-opening interview, and I hope you enjoy watching it as much as Scott and I enjoyed conducting it!

Thinking Big @ Information On Demand 2012

Nate Silver, author of the blog “FiveThirtyEight,” will be one of the featured keynote speakers at this year’s IBM Information On Demand 2012 event in Las Vegas, Nevada, October 21-25. Silver correctly predicted the results of the primaries and the U.S. presidential winner in 2008 in 49 states through his statistical analysis of polling data, and at IOD will explain how to distinguish real signals from noisy data as well as how predictive analytics is used in politics.

That annual festouche and gathering of all things data is just around the corner.

Yes, that’s right, it’s almost time for IBM Information on Demand 2012.

So in order to start the drumbeat, I wanted to take a few moments and point you to some useful resources as you prepare to make your way to the Bay of Mandalay, and to optimize your time on the ground in Vegas.

First, the new (and official) IBM Information on Demand blog, which you can find here.

The blog includes easy access to some of the social media channels that will be covering the event (including Facebook, Twitter, LinkedIn and YouTube).

Of course, never forget the official IOD hashtag, #ibmiod, where you’ll be able to follow the endless stream of tidings leading up to, during, and after the event.

The blog also has links off to the IOD 2012 registration engine, as well as to the IOD SmartSite so you can start thinking about your IOD calendar now (I do NOT advise waiting until the last minute…talk about information overload!)

We’ve got some exciting guest speakers this year, including Nate Silver, statistics blogging extraordinaire who first found fame with his “FiveThirtyEight” blog, which is now part of The New York Times family of media properties.

Silver analyzes politics the way most of us should be analyzing our business: Through data…and lots of it.

His analysis of political polling data is unparalleled, and in the 2008 U.S. presidential election, Silver correctly predicted the results of the primaries and the presidential winner in 49 states.

His recent book, “The Signal and The Noise: Why Most Predictions Fail — But Some Don’t,” explores the world of prediction, “investigating how we can distinguish a true signal from a universe of noisy data.” Silver tackles some of the big questions about big data, so we’re very excited to have him join us in Vegas for IBM’s own big data marathon event.

At this year’s event, we’ll continue our trend of including tracks for specialized areas of interest, including forums for Information Management, Business Analytics, Business Leadership, and Enterprise Content Management.

And, of course, you’ll be able to find Scott Laningham and myself down in the EXPO center, where we’ll be talking to and interviewing many of the IBM and industry luminaries on the important data-related topics being discussed at the event.

Speaking of data, this will be my seventh IOD in a row, so I’m looking forward to seeing many of you once again.

Meanwhile, keep an eye here on the Turbo blog for future IOD-relevant posts.

Mac v. PC Shopping Guy

Welcome to Austin, Texas, where it’s expected to reach a balmy 106 degrees today.

I would retire to the pool and use my newfangled Verizon Jetpack Internet everywhere device to let me do a few emails while sipping virgin Pina Coladas, but I’m afraid my skin might start burning and smoking like some bad horror movie. Yes, it’s going to be THAT hot outside (and it’s only June 26).

In the summer, I tend to get up really early to do all my grocery shopping and things, so that I can then come home and never leave the house until the sun goes back down.

And on the subject of shopping, while flipping through the news on my iPad this morning, I discover this whopper of a story in The Wall Street Journal online.

Travel company Orbitz recently discovered that people who use Apple’s Mac computers spend as much as 30% or more a night on hotels. So, in turn, Orbitz is starting to give them different, and sometimes costlier, travel options than what Windows visitors see.

You mean, I have to go back to using Windows in order to get the best deals on Orbitz? Not necessarily, but it’s quite evident that you’ll be given different promotions, many of which will cost more because you’re part of the cool, Apple fanboy set.

Now if we could just see what Orbitz would offer up to Ubuntu Linux users…a cardboard shack out back?!

I’ve been writing about IBM’s smarter commerce initiative for several months now, and this is a perfectly good example of how companies are using all that great information they’re garnering in their web browsing and sales activities, then using that information to market differently to different folks.

Before you Mac users pull your long dormant Windows7 machine out of hibernation, first, remember you can always opt to rank all your results on Orbitz (and other travel sites) by price, and you’re obviously not limited to the promotions you are offered.

But Orbitz did find that Mac users spend an average of $20-$30 a night more on hotels than their PC counterparts, according to the WSJ story, which is a substantial difference considering that the site’s average nightly hotel booking is around $100.

I sense a whole new wave of Mac v. PC commercials coming on:

PC Guy: Dude, I stayed at the Four Seasons for $30 less than you did last night because I run Windows!

Mac Guy: Yes, but you didn’t look nearly as cool as me hanging out at the hotel bar with all the hipsters. And when I turn my computer on it just works!

The data also revealed that Mac users tend to stay in more expensive rooms than the Windows crowd.

So, this is the part where I go back outside, grab my Pina Colada and multiple cans of 60 SPS Walgreen tanning spray, and hand you off to the IBM Smarter Commerce Website so you can read more about how you can utilize such predictive analytics for business advantage.

And don’t forget about my last post, where I mentioned some recent announcements IBM made in the predictive analytics space.

IBM Business Analytics: Preventing Fraud, Predicting Profits

Scott Laningham and I are starting to think about repacking our suitcases and preparing to head back out on the road, this time across the pond to Madrid for the IBM Smarter Commerce Global Summit May 22-24.

In Madrid, we expect to hear quite a bit about IBM’s investment in the analytics space, but that doesn’t mean we have to wait to visit the Prado to relate some interesting details about business analytics.

Specifically, predictive analytics that can help companies across the span of industries to prevent fraud.

Here’s a sound byte you may not have yet heard: Did you know that insurance fraud has reached an estimated $80 billion per year in the U.S. alone??

And in South Africa, the rate of short-term insurance fraud is about 15 percent of all premium costs.

And yet, we’ve also found that organizations that effectively apply predictive analytics are 2.2 times more likely to outperform their peers.

One such client of IBM is Santam, South Africa’s leading short term insurance company, which has saved $2.4 million on fraudulent claims in the first four months of using IBM business analytics software.

This new analytics solution has not only enhanced Santam’s fraud detection capabilities, however — it has also enabled faster payouts for legitimate claims.

In partnering with IBM, Santam’s claims division developed a new operating model for processing claims, depending on varying risk levels. IBM’s predictive analytics software has enabled Santam to automatically assess if there is any fraud risk associated with incoming claims and allows the insurer to distribute claims to the appropriate processing channel for immediate settlement or further investigation, which in turn optimizes Santam’s operational efficiency.

In turn, Santam is able to reduce the number of claims that need to be assessed by mobile operatives visiting the customer or claim site, resulting in further considerable cost savings for the company.

IBM: Investing In Analytics, Predicting Results

In the last five years, IBM has invested more than $14 billion in acquisitions. With investments in SPSS, Clarity, OpenPages, i2 and Algorithmics, and others, IBM is building business analytics solutions providing clients with capabilities for managing fraud, risk and threat. In addition, IBM has assembled almost 9,000 dedicated analytics consultants with industry expertise, and created a network of eight global analytics solution centers.

The Santam project also illustrates IBM’s leadership in analytics in Africa. IBM is also actively laying the foundations for a major presence throughout the African continent, with offices in more than 20 African countries, where the company is assisting businesses and governments in building strategies, expertise, solutions, frameworks and operating procedures to help improve performance.

You can learn more about Santam here, and their new predictive analytics solution in the video below.  You can learn more about IBM business analytics solutions here.

Information On Demand 2011: New Predictive Analytics For Healthcare

This morning at Information On Demand 2011, IBM introduced new software for the healthcare industry to help health care providers and payers improve patient care and reduce costs.

According to the New England Journal of Medicine, one in five patients suffer from preventable readmissions, which represents $17.4 billion of the current $102.6 billion Medicare budget. Beginning in 2012, hospitals will be penalized for high readmission rates with reductions in Medicare discharge payments.

The new software offering uses content analytics similar to what is found in IBM’s Watson technology. IBM today introduced new software for the healthcare industry to help health care providers and payers improve patient care and reduce costs. The new software offering uses content analytics similar to what is found in IBM’s Watson technology.

Seton Healthcare Family is the first client to adopt and use the technology, called “IBM Content and Predictive Analytics for Healthcare.”

The solution will allow healthcare organizations to extract relevant clinical information from vast amounts of patient data to better analyze the past, understand the present, and predict future outcomes.

Calling Dr. Watson

By combining IBM’s Watson technology with industry solutions offerings, Seton intends to focus the new content and predictive analytics solution on the root causes of hospital readmissions, and ways it can decrease preventable multiple hospital visits.

Most healthcare organizations are drowning in data but are challenged to gain reliable, actionable insights from this information. In fact, more than 80 percent of an institution’s data today is unstructured. In healthcare, this is in the form of physician notes, registration forms, discharge summaries, documents and more is doubling every five years.  Different from machine- ready data, this content lacks structure and is arduous for healthcare enterprises to include in business analysis and therefore is routinely left out. As a result, millions of patient notes and records often sit unavailable in separate clinical data silos. This content contains valuable information, but there’s historically been no easy way to analyze it.

IBM Content and Predictive Analytics for Healthcare enables doctors and healthcare professionals to go far beyond traditional search and analysis of unstructured data. They can advance diagnosis and treatment by accurately extracting medical facts and understanding relationships buried in large volumes of clinical and operational data.

The IBM solution transforms raw information into healthcare insight quickly by revealing trends, patterns, deviations and predicting the probability of outcomes, allowing organizations to derive insight in minutes versus weeks or months, or not at all. As a result, healthcare professionals can find more effective ways to care for high-risk patients, provide safer patient care, and develop new models for reimbursement for quality care.

Powered By POWER

The new IBM solution gives clinical and other knowledge workers and executives several ways to interact with analyzed information including searching, exploring, mining, monitoring and reporting. It delivers a set of proven technologies that meet the rigorous standards and requirements of the healthcare community.

The software is also compatible with IBM’s Health Integration Framework, which means healthcare organizations can realize more value from existing information system investments such as data warehouse, business intelligence, master data management and advanced case management.

IBM is offering new content and predictive solution services through its Business Analytics and Optimization initiatives, which includes a new center of competence for UIMA-based text analysis solutions. This center of competence draws on resources from IBM Global Services, IBM Software Lab Services, and the IBM jStart emerging technology team.

IBM Content and Predictive Analytics for Healthcare is optimized to run on IBM Power Systems, which are designed for high throughput and complex analysis of structured and unstructured data. Built on the foundation of IBM POWER7 processor technology, Power Systems are available at many different price points and can be tailor fit for purpose and rapidly deployed for a broad range of customer environments with leadership performance, ease of management and efficiency.

For more information go here. IBM Content and Predictive Analytics for Healthcare.

Planets Aligned

I guess I’ve been living under a rock.

I keep CNBC on in the background at times, and I just saw reporter Erin Burnett being given a sendoff from the floor of the New York Stock Exchange.  Apparently she’s taken a new position at CNN, says the Times’ Media Decoder blog.

Adios and vayo condios, Erin, “International Superstar,” as the “Morning Joe” crew likes to refer to Erin.  International markets coverage on CNBC won’t be the same without her.

In the category of mean-spirited piling on, CNET’s Circuit Breaker blog is reporting a group of hackers are planning another wave of cyberattacks against Sony, this time apparently “in retaliation for its handling of the PlayStation network breach.”

They can’t say they weren’t warned.

Then again, there’s an upside to everything.  If you social media acolytes were wondering what it would take to get more C-level execs out there blogging, have your global gaming network be hacked two times in two weeks on the tail end of even more bad news in the form of a horrible tsunami and earthquake, and you might just get your answer.

Sony’s PlayStation Blog welcomed a post (a letter, actually) from Sir Howard Stringer, in which he apologized as well as announced measures to reassure Sony network users, including one for U.S. PlayStation Network and Qriocity customers that includes a $1 million identity theft insurance policy.

Sony will also be offering a “Welcome Back” package to its customers “once [their] PlayStation Network and Qriocity services are up and running,” one which will include a month of free PlayStation Plus membership for all PSN customers, among other treats.

When will that be?  VentureBeat’s GamesBeat says Sony has “entered the final stages of internal testing to bring its beleaguered online network” back online.

This, too, shall pass, and it’s time for a small moment of celebration.

I’ve always been a bit of a space junkie — it’s probably one of the reasons I ended up working in technology.

And if you’ve followed IBM’s Centennial communications this year, you’ve seen a number of videos online and TV spots that highlighted the role IBM played in helping land a man on the moon.

We didn’t get there in one fell swoop.  It took the better part of the decade, and Project Mercury was a key step in the direction of manned spaceflight for the U.S.

NASA celebrated its 50th anniversary of manned space flight yesterday at an event at the Kennedy Space Center.  Concurrently, IBM celebrated the team of mathematicians and technologists which supported the Project Mercury missions in the 1960s.

Professor Arthur Cohen led the IBM team that supported Project Mercury and recounts the project in the video below.  You can also read the fascinating details behind IBM’s support of Project Mercury here.

As you prepare to be launched back to the liftoff of the Space Race, does anyone but me find it strange that this celebration occurs as all the planets come into alignment for much of the month of May??

Predictive Perspectives

If you’ve been reading this blog for any length of time (and I don’t just mean for the past five minutes), then you know I’ve been focusing coverage more and more around the topic of business analytics.

Not only because it’s something IBM has been focused more and more on (considering our investments in acquiring Cognos, SPSS, and other analytics-software-oriented vendors, as well as our major ramp-up in the Business Analytics and Optimization services space).

No, but also because it just makes good business sense. We find ourselves in a world where financial resources are more limited than ever, and prioritization by government, industry, and even individuals of those precious resources has probably never been more important.

Some of those tools include the ability to more effectively predict change and outcomes using technology, which is why I wanted to share some information passed along to me earlier today by one of my colleagues in our SPSS group.

If you’ve been wanting to get the lowdown on SPSS, and how predictive analytics can help your organization, look no further.

There are a series of events occurring in the U.S. and Canada starting on August 19 in the City of Angels (Los Angeles) called “Predictive Perspectives.”

At each event, there will be three tracks: 1) Predictive Analytics Overview 2) Technology Deep Dive and 3) Business Intelligence and Management Overview

Check out the overview of this series here.

It will be a great (and free) venue at which you can learn more about predictive analytics.