Big Study On Big Data

Perfect timing.

In advance of IBM’s massive event next week in Las Vegas featuring all things information management, Information On Demand 2012, IBM and the Saïd Business School at the University of Oxford today released a study on Big Data.

According to a new global report from IBM and the Said Business School at the University of Oxford, less than half of the organizations engaged in active Big Data initiatives are currently analyzing external sources of data, like social media.

The headline: Most Big Data initiatives currently being deployed by organizations are aimed at improving the customer experience, yet less than half of the organizations involved in active Big Data initiatives are currently collecting and analyzing external sources of data, like social media.

One reason: Many organizations are struggling to address and manage the uncertainty inherent within certain types of data, such as the weather, the economy, or the sentiment and truthfulness of people expressed on social networks.

Another? Social media and other external data sources are being underutilized due to the skills gap. Having the advanced capabilities required to analyze unstructured data — data that does not fit in traditional databases such as text, sensor data, geospatial data, audio, images and video — as well as streaming data remains a major challenge for most organizations.

The new report, entitled “Analytics: The real-world use of Big Data,” is based on a global survey of 1,144 business and IT professionals from 95 countries and 26 industries. The report provides a global snapshot of how organizations today view Big Data, how they are building essential capabilities to tackle Big Data and to what extent they are currently engaged in using Big Data to benefit their business.

Only 25 percent of the survey respondents say they have the required capabilities to analyze highly unstructured data — a major inhibitor to getting the most value from Big Data.

The increasing business opportunities and benefits of Big Data are clear. Nearly two-thirds (63 percent) of the survey respondents report that using information, including Big Data, and analytics is creating a competitive advantage for their organizations. This is a 70 percent increase from the 37 percent who cited a competitive advantage in a 2010 IBM study.

Big Data Drivers and Adoption

In addition to customer-centric outcomes, which half (49 percent) of the respondents identified as a top priority, early applications of Big Data are addressing other functional objectives.

Nearly one-fifth (18 percent) cited optimizing operations as a primary objective. Other Big Data applications are focused on risk and financial management (15 percent), enabling new business models (14 percent) and employee collaboration (4 percent).

Three-quarters (76 percent) of the respondents are currently engaged in Big Data development efforts, but the report confirms that the majority (47 percent) are still in the early planning stages.

However, 28 percent are developing pilot projects or have already implemented two or more Big Data solutions at scale. Nearly one quarter (24 percent) of the respondents have not initiated Big Data activities, and are still studying how Big Data will benefit their organizations.

Sources of Big Data

More than half of the survey respondents reported internal data as the primary source of Big Data within their organizations. This suggests that companies are taking a pragmatic approach to Big Data, and also that there is tremendous untapped value still locked away in these internal systems.

Internal data is the most mature, well-understood data available to organizations. The data has been collected, integrated, structured and standardized through years of enterprise resource planning, master data management, business intelligence and other related work.

By applying analytics, internal data extracted from customer transactions, interactions, events and emails can provide valuable insights.

Big Data Capabilities

Today, the majority of organizations engaged in Big Data activities start with analyzing structured data using core analytics capabilities, such as query and reporting (91 percent) and data mining (77 percent).

Two-thirds (67 percent) report using predictive modeling skills.

But Big Data also requires the capability to analyze semi-structured and unstructured data, including a variety of data types that may be entirely new for many organizations.

In more than half of the active Big Data efforts, respondents reported using advanced capabilities designed to analyze text in its natural state, such as the transcripts of call center conversations.

These analytics include the ability to interpret and understand the nuances of language, such as sentiment, slang and intentions. Such data can help companies, like a bank or telco provider, understand the current mood of a customer and gain valuable insights that can be immediately used to drive customer management strategies.

You can download and read the full study here.

Update: Also check out the new IBM Big Data Hub, a compendium of videos, blog posts, podcasts, white papers, and other useful assets centering on this big topic!

Just How Big IS Big Data?

So just how big IS big data?

This is your opportunity to find out, and, to contribute.

IBM’s Institute for Business Value is conducting a study on big data, and we’d like to hear from you.

The idea behind the study is simple: To develop a fact-based analysis of big data activities in the global marketplace.

Through this research, IBM hopes to help the marketplace better understand some key tenets behind the big data movement: To gain an organizational view of big data and organizations’ primary objectives for investments in this burgeoning area.  To understand better the drivers and leaders of big data activities. To understand the current and planned state of big data activities, and patterns that suggest best practices of big data implementations.

The survey is slated to run through June 29, 2012, and takes approximately 10-15 minutes per respondent.

All responses will be viewed in the aggregate, and individual responses will not be disclosed beyond the survey analysis team without expression permission of the respondent.

The audience for the survey: Global business executives, management and analysts, as well as IT professionals, across all levels of the organizational hierarchy (from C-suite to data analysts).

Once the fielding is completed, the survey results will be analyzed by a wide team of subject matter experts from within IBM, along with a team of faculty from a globally recognized university.

This data will be combined with interviews and case studies to develop a final reporting of findings and big data benchmarks to be published in October of this year.

So, in short, this is your opportunity to be part of the benchmarks that will define the big data era, one that you can use to compare with your own organization.

All participants will receive a copy of the final study, and will also be eligible to download the IBM e-book entitled “Understanding Big Data.”

Here’s the link if you’d like to be part of this exciting big data discovery!

Live @ Pulse 2010: Chesapeake’s Smarter City

At today’s IBM Pulse 2010 event here in Las Vegas, IBM and the City of Chesapeake, Virginia, announced how they’ve partnered to build a “smarter city,” one in which intelligent city-wide systems enhance public services, including public works, public utilities, public safety, and even its parks and recreation initiatives.

The use of IBM technology in this effort is enhancing services delivered to the public ranging from maintenance and operations of traffic signals and water systems to the management of the City’s Fire and Police Departments.

As a result of these efforts, the City of Chesapeake is consistently improving the quality of life for its citizens.

The Background

The City of Chesapeake is one of the larger cities in Virginia, covering some 353 square miles. It’s a diverse community, with suburban, urban, and rural areas, with with a business community that is equally diverse.

It includes more than 80 foreign-based companies from 19 different countries, and a city that has more miles of deep water canals, including the Intracoastal Waterway, than any other city in the U.S.

IBM software manages the maintenance of equipment and facilities for the Chesapeake Public Works Department, including the people and trucks using brine for snow removal at the City of Chesapeake Expressway Toll Plaza

The size and location of the city makes it a complex infrastructure to manage. These unique challenges can be addressed in part by using technology to collect and analyze data that can be used to improve how transportation, utility management, and public safety systems react to constantly changing conditions.

A Comprehensive Plan for a Smarter City

In accordance with its comprehensive plan, the City of Chesapeake is paving the way to a smarter future by currently investing more than $1.2M U.S. in capital improvement projects affecting community facilities, economic development, and other key departments.

“Technology is the power tool of today,” said Peter R. Wallace, CIO of the City of Chesapeake. “We’re using IBM Software to give staff the data and tools to continually improve processes, which is essential in this economy. The City of Chesapeake is less than 50 years old, but those founders inherited hundreds of years of infrastructure. Until now, we haven’t had a quick or convenient way to look at the City’s assets and make smart decisions. To succeed, we must be efficient in the way we work and transparent to our citizens. IBM is helping us accomplish those goals.”

IBM’s software is connecting city systems, and providing the various city departments with a transparent view of what’s going on at any given time. By analyzing the data and sharing the findings across departments, the city is able to detect and react to potential problems more quickly.

IBM: A Vision for Smarter Cities

A recent IBM Institute for Business Value report entitled “A Vision of Smarter Cities” asserts that the digitization of data within a city’s core systems will enable city managers to collect data on the efficiency of processes that could not be previously measured, like wastewater treatment. This, in turn, will lead to more informed decision-making and planning from city leaders.

Michael Fitchett, the City of Chesapeake Systems Development Coordinator and a former city firefighter, discussed at a press conference today hosted by Tivoli general manager Al Zollar his team’s efforts in building a smarter City of Chesapeake.

“When we went live in December 2008 with IBM Maximo asset management software, I never thought I’d be up here talking about a smarter city initiative. The system has allowed me and my team to see a lot of different facets and look across the entire environment of the city and to, in turn, provide our citizens with better services.

“My CIO looks at me everyday and says “ATA” — Accountability, Transparency, and Agility.  I have a mayor who walks into the room and asks what our motto is: ‘We’re open for business.'”

Fitchett explained that he had strong management support, and that every day he is looking for “efficiencies and effectiveness” and explained what he considers to be the essence of a smarter city:

“Making sure we’re getting needed data out to our field personnel so that they can make real-time decisions: Working on a broken water main, perhaps having a video feed down that pipe to see proximity locations and able to analyze and instantly make critical infrastructure decisions.”

“The City of Chesapeake serves as a great example of how cities can take advantage of technology to provide citizens and businesses with a better, smarter place to live,” said Bill Sawyer, vice president of operations, IBM Maximo software. “By using these IBM technologies to better manage critical systems like water management and public safety, the city is both improving the quality of life for its citizens today and building a more sustainable future.”