Big Study On Big Data

Perfect timing.

In advance of IBM’s massive event next week in Las Vegas featuring all things information management, Information On Demand 2012, IBM and the Saïd Business School at the University of Oxford today released a study on Big Data.

According to a new global report from IBM and the Said Business School at the University of Oxford, less than half of the organizations engaged in active Big Data initiatives are currently analyzing external sources of data, like social media.

The headline: Most Big Data initiatives currently being deployed by organizations are aimed at improving the customer experience, yet less than half of the organizations involved in active Big Data initiatives are currently collecting and analyzing external sources of data, like social media.

One reason: Many organizations are struggling to address and manage the uncertainty inherent within certain types of data, such as the weather, the economy, or the sentiment and truthfulness of people expressed on social networks.

Another? Social media and other external data sources are being underutilized due to the skills gap. Having the advanced capabilities required to analyze unstructured data — data that does not fit in traditional databases such as text, sensor data, geospatial data, audio, images and video — as well as streaming data remains a major challenge for most organizations.

The new report, entitled “Analytics: The real-world use of Big Data,” is based on a global survey of 1,144 business and IT professionals from 95 countries and 26 industries. The report provides a global snapshot of how organizations today view Big Data, how they are building essential capabilities to tackle Big Data and to what extent they are currently engaged in using Big Data to benefit their business.

Only 25 percent of the survey respondents say they have the required capabilities to analyze highly unstructured data — a major inhibitor to getting the most value from Big Data.

The increasing business opportunities and benefits of Big Data are clear. Nearly two-thirds (63 percent) of the survey respondents report that using information, including Big Data, and analytics is creating a competitive advantage for their organizations. This is a 70 percent increase from the 37 percent who cited a competitive advantage in a 2010 IBM study.

Big Data Drivers and Adoption

In addition to customer-centric outcomes, which half (49 percent) of the respondents identified as a top priority, early applications of Big Data are addressing other functional objectives.

Nearly one-fifth (18 percent) cited optimizing operations as a primary objective. Other Big Data applications are focused on risk and financial management (15 percent), enabling new business models (14 percent) and employee collaboration (4 percent).

Three-quarters (76 percent) of the respondents are currently engaged in Big Data development efforts, but the report confirms that the majority (47 percent) are still in the early planning stages.

However, 28 percent are developing pilot projects or have already implemented two or more Big Data solutions at scale. Nearly one quarter (24 percent) of the respondents have not initiated Big Data activities, and are still studying how Big Data will benefit their organizations.

Sources of Big Data

More than half of the survey respondents reported internal data as the primary source of Big Data within their organizations. This suggests that companies are taking a pragmatic approach to Big Data, and also that there is tremendous untapped value still locked away in these internal systems.

Internal data is the most mature, well-understood data available to organizations. The data has been collected, integrated, structured and standardized through years of enterprise resource planning, master data management, business intelligence and other related work.

By applying analytics, internal data extracted from customer transactions, interactions, events and emails can provide valuable insights.

Big Data Capabilities

Today, the majority of organizations engaged in Big Data activities start with analyzing structured data using core analytics capabilities, such as query and reporting (91 percent) and data mining (77 percent).

Two-thirds (67 percent) report using predictive modeling skills.

But Big Data also requires the capability to analyze semi-structured and unstructured data, including a variety of data types that may be entirely new for many organizations.

In more than half of the active Big Data efforts, respondents reported using advanced capabilities designed to analyze text in its natural state, such as the transcripts of call center conversations.

These analytics include the ability to interpret and understand the nuances of language, such as sentiment, slang and intentions. Such data can help companies, like a bank or telco provider, understand the current mood of a customer and gain valuable insights that can be immediately used to drive customer management strategies.

You can download and read the full study here.

Update: Also check out the new IBM Big Data Hub, a compendium of videos, blog posts, podcasts, white papers, and other useful assets centering on this big topic!