Posts Tagged ‘business analytics’
Big Moves In Big Data: IBM New Data Acceleration, Hadoop Capabilities

Click to enlarge. IBM just announced new technologies designed to help companies and governments tackle Big Data by making it simpler, faster and more economical to analyze massive amounts of data. New data acceleration innovation results in as much as 25 times faster reporting and analytics.
IBM made a significant announcement earlier today concerning new technologies designed to help companies and governments tackle Big Data by making it simpler, faster and more economical to analyze massive amounts of data. The new data acceleration innovation results in as much as 25 times faster reporting and analytics.
Today’s announcement, which represents the work of hundreds of IBM developers and researchers in labs around the world, includes an industry-first innovation called “BLU Acceleration,” which combines a number of techniques to dramatically improve analytical performance and simplify administration.
Also announced was the new IBM PureData System for Hadoop, designed to make it easier and faster to deploy Hadoop in the enterprise. Hadoop is the game-changing open-source software used to organize and analyze vast amounts of structured and unstructured data, such as posts to social media sites, digital pictures and videos, online transaction records, and cell phone location data.
The new system can reduce from weeks to minutes the ramp-up time organizations need to adopt enterprise-class Hadoop technology with powerful, easy-to-use analytic tools and visualization for both business analysts and data scientists.
In addition, it provides enhanced Big Data tools for monitoring, development and integration with many more enterprise systems.
IBM Big Data Innovations: More Accessible, Enterprise-ready
As organizations grapple with a flood of structured and unstructured data generated by computers, mobile devices, sensors and social networks, they’re under unprecedented pressure to analyze much more data at faster speeds and at lower costs to help deepen customer relationships, prevent threat and fraud, and identify new revenue opportunities.
BLU Acceleration enables users to have much faster access to key information, leading to better decision-making. The software extends the capabilities of traditional in-memory systems — which allows data to be loaded into Random Access Memory instead of hard disks for faster performance — by providing in-memory performance even when data sets exceed the size of the memory.
During testing, some queries in a typical analytics workload were more than 1000 times faster when using the combined innovations of BLU Acceleration.
Innovations in BLU Acceleration include “data skipping,” which allows the ability to skip over data that doesn’t need to be analyzed, such as duplicate information; the ability to analyze data in parallel across different processors; and greater ability to analyze data transparently to the application, without the need to develop a separate layer of data modeling.
Another industry-first advance in BLU Acceleration is called “actionable compression,” where data no longer has to be decompressed to be analyzed.
Not IBM’s First Big Data Rodeo
The new offerings expand what is already the industry’s deepest portfolio of Big Data technologies and solutions, spanning software, services, research and hardware. The IBM Big Data platform combines traditional data warehouse technologies with new Big Data techniques, such as Hadoop, stream computing, data exploration, analytics and enterprise integration, to create an integrated solution to address these critical needs.
IBM PureData System for Hadoop is the next step forward in IBM’s overall strategy to deliver a family of systems with built-in expertise that leverages its decades of experience reducing the cost and complexity associated with information technology.
This new system integrates IBM InfoSphere BigInsights, which allows companies of all sizes to cost-effectively manage and analyze data and add administrative, workflow, provisioning and security features, along with best-in-class analytical capabilities from IBM Research.
Today’s announcement also includes the following new versions of IBMs Big Data solutions:
- A new version of InfoSphere BigInsights, IBM’s enterprise-ready Hadoop offering, which makes it simpler to develop applications using existing SQL skills, compliance security and high availability features vital for enterprise applications. BigInsights offers three entry points: free download, enterprise software and now an expert integrated system, IBM PureData System for Hadoop.
- A new version of InfoSphere Streams, unique “stream computing” software that enables massive amounts of data in motion to be analyzed in real-time, with performance improvements, and simplified application development and deployment.
- A new version of Informix including TimeSeries Acceleration for operational reporting and analytics on smart meter and sensor data.
Pricing and Availability
All offerings are available in Q2, except the PureData System for Hadoop, which will start shipping to customers in the second half 2013. Credit-qualified clients can take advantage of simple, flexible lease and loan packages with no up-front payments for the software and systems that deliver a new generation of data analytics.
IBM Global Financing offers attractive leasing programs with 90-day payment deferrals for the PureData System for Hadoop, as well as zero percent loans for the broader portfolio of IBM big data solutions.
IBM Taps Next Generation Leaders For Watson Innovation

The Watson Case Competition at USC, the third in a series hosted by IBM, is the latest example of IBM’s work with academia to advance interest among students in Science Technology Engineering and Math (STEM) curriculums that will lead to high-impact, high-value careers. The competition is in keeping with IBM’s Academic Initiative which delivers course work, case studies and curricula to more than 6,000 universities and 30,000 faculty members worldwide to help students prepare for high-value future job opportunities.
While I was out trying to grok all things SXSW Interactive these past several days, IBM continued with its efforts to put IBM Watson to work for the betterment of mankind by turning to the next generation of brilliant young minds to help figure out where Watson should work next.
Imagine a Watson-powered system that could uncover data-driven insights to help medical professionals identify those who may be suffering silently from Post-Traumatic Stress Disorder.
Imagine a Watson that could provide lawyers with faster research capabilities to improve their cases.
Imagine a Watson that could help businesses hire the best talent in the job market.
This is the magnitude of ideas sparked by more than 100 University of Southern California students who gathered recently to compete in the IBM Watson Academic Case Competition.
A debut on the West Coast, the Case Competition put USC students in the spotlight to create business plans for applying Watson to pressing business and societal challenges — and IBM business leaders were present and listening carefully.
IBM: Partnering To Learn
IBM partners with thousands of universities to offer curricula, internships and hands-on experiences to help students learn first hand about new technologies in the fields of Big Data, analytics and cognitive computing.
The company is at the forefront of creating a new workforce of Big Data trained professionals, from IBM’s collaboration with Cleveland Clinic, which provides Watson as a collaborative learning tool for medical students, to its public-private partnership with the New York City Department of Education and the City University of New York to create the Pathways in Technology Early College High School program (P-TECH), which allows students to participate in a six year science and technology program and graduate with an associates degree for free in computer science or engineering.
To kick-off the competition at USC’s campus, IBM provided students with a crash course on Watson’s breakthrough capabilities, including a demonstration of how Watson is helping WellPoint, Inc. and Memorial Sloan Kettering Cancer Center improve the speed and quality of treatment for cancer patients.
As the first cognitive computing system of its kind in the marketplace, Watson is able to understand and process the subtleties of human questions, sift through vast amounts of data, and use sophisticated analytics to generate fast, accurate answers for its human users.
Watson also learns from its interactions, constantly improving with each use. This represents a major shift in organizations’ ability to quickly analyze, understand and respond to Big Data, in industries such as healthcare — and this is where student minds were put to the test.
As part of the competition, students were assigned into 24 teams and given 48 hours to define a new purpose for Watson, develop a business plan, and present it to a panel of judges comprising school officials, IBM executives and local business leaders.
The challenge was unique among USC competitions because students worked toward a common goal with peers from other disciplines — similar to how IBM combines the talent of business leaders and research scientists to develop its patented innovations.
To foster interdisciplinary collaboration, each team was required to have at least one business and one engineering member, from USC’s Marshall Business School and Viterbi School of Engineering.
What’s Your Business Plan For Watson?
The student teams faced two rounds of judging based on four areas of criteria: how well the concept and supporting plan articulated and supported the team’s vision; the feasibility of bringing the product or service to market and the supporting elements; the extent the proposed solution leverages Watson’s key capabilities; and the team’s presentation. Three winning ideas were selected by a panel of eight industry and faculty judges, including representatives from Bank of America, Ernst & Young, and IBM.
- 1st Place – Legal Research: Let Watson Do the Discovery for Your Next Legal Case – For corporate legal departments, building a case — or defending one’s own — relies heavily on fast and accurate research. Past legal trials, court documents, articles and digital evidence: all of these materials can make or break a case, and together they comprise a sea of unstructured data that is both time-consuming and costly to pore through. The first place USC team proposed using Watson to process its users’ research needs, based on its ability to think like a human, quickly sift through online legal documents for facts, and not only identify evidence to support a case — but forecast its probability of success. The first place team’s viewpoint: by placing Watson in charge of research, firms can recover time and costs, while delivering better legal outcomes. In turn, firms that leverage Watson’s speed and efficiency can address the growing legal trend towards “flat fee” billing and research outsourcing.
- 2nd Place – Employee Training: Watson Uncovers the Keys to Success for Your Employees – According to the American Society for Training and Development (ASTD), 41 percent of employees at companies with inadequate training programs plan to leave within a year, versus 12 percent of employees at companies who provide excellent training and professional development programs. Conversely, the ASTD also states that effective employee training can lead to 218 percent higher income per employee and 45 percent higher shareholder return than market average. The second place USC team proposes that corporate human resource departments use Watson to optimize employee training, by crunching data pertaining to the employers’ HR needs, the employees’ career goals, and the range of training options available that can help both parties succeed. The second place team’s viewpoint: by improving employee satisfaction and retention, a Watson-powered employee training system can also drive higher shareholder value.
- 3rd Place – Post-Traumatic Stress Disorder: Watson Helps Doctors Find Patients – It is reported that Post-Traumatic Stress Disorder affects nearly 7.7 million U.S. adults aged 18 and older. This includes people who have served in combat, experienced domestic violence, have been in car accidents, or other traumatic events. Many with PTSD suffer silently, including the 400,000+ U.S. veterans who have yet to be identified and treated, per the U.S. Veterans Administration. Thankfully, the catalysts behind this illness need no longer remain invisible — due largely to Big Data. For example, there are now unprecedented amounts of data that accompany soldiers who return from war, from medical histories to information on combat experiences. The third place USC team proposes that physicians use Watson to identify people who may develop PTSD, by uncovering insights from data that can help piece together their personal story and shed light on pain he or she may be experiencing. The team’s viewpoint: by helping physicians find and diagnose those suffering from PTSD, Watson can help medical professionals offer patients the treatment they deserve.
Fueling Innovation While Investing In The Next Generation Of Tech Leaders
This competition is the latest example of how IBM is fueling innovation and working with students in higher education to hone valuable business skills that will shape the next generation of industry leaders.

“Partnering with universities such as USC gives IBM a unique opportunity to tap into the minds of our next-generation of leaders, whose training, skills and ideas for changing the world are all forward-thinking and based on a desire to make a meaningful impact,” said Manoj Saxena, IBM General Manager, Watson Solutions, about the new initiative. “These students see what Watson is doing right now and think — how else will cognitive computing impact my life and career in the years to come? To us, that’s exactly the mindset that should be fueling IBM innovations, and the very reason we host Watson Academic Case Competitions.”
Due to the overwhelming response from USC students seeking to participate in the Watson Academic Case Competition, students had to join a waiting list, once the 24-team maximum had been reached. One faculty sponsor, noting that the level of interest was unprecedented for a campus case competition, predicted registration could reach 500 next year.
“For USC students, the opportunity to share their own ideas with IBM on how to commercialize Watson is truly a unique experience,” said Ashish Soni, Executive Director of Digital Innovation and Founding Director of the Viterbi Student Innovation Institute at the USC Viterbi School of Engineering. “As educators, we’re quite pleased to see students getting excited about cognitive computing innovation, because we know there’s a business demand for the types of skills they get to showcase in Watson Case Competitions.”
Watson — Building a New Big Data Workforce
It’s no secret that employers across the U.S. are seeking job candidates who can analyze and build strategy around Big Data, or the 2.5 quintillion bytes of information gleaned from sensors, mobile devices, online transactions and social networks, to name just a few sources. A recent Gartner report estimates that 1.9 million Big Data jobs will be created in the U.S. by 2015.
The Watson Case Competition at USC, the third in a series hosted by IBM, is the latest example of IBM’s work with academia to advance interest among students in Science Technology Engineering and Math (STEM) curriculums that will lead to high-impact, high-value careers. The competition is in keeping with IBM’s Academic Initiative which delivers course work, case studies and curricula to more than 6,000 universities and 30,000 faculty members worldwide to help students prepare for high-value future job opportunities.
IBM worked closely with academic institutions during the development and introduction of Watson. Eight leading universities around the world participated in the development phase of the system; and more than 10,000 students watched Watson triumph on the Jeopardy! quiz show in February 2011. Most recently, IBM announced it would provide a modified version of an IBM Watson system to Rensselaer Polytechnic Institute, making it the first university to receive such a system that will enable leading-edge research by faculty and students.
The competition at USC marks the latest collaboration between the university and IBM. Over the last two years, students at the school’s Annenberg Innovation Lab have been using Big Data analytics technologies to conduct social sentiment analyses and determine public engagement on topics such as sports, film, retail and fashion.
Two of the biggest projects looked at Major League Baseball’s World Series and the Academy Awards, projects developed for students to explore and expand their skills as they prepare for new data-intensive careers. IBM also collaborated with the USC Marshall School of Business for “The Great Mind Challenge,” a global academic initiative focused on providing students with an opportunity to turn their social networking savvy into business ready skills to prepare for the jobs of the future.
IBM To Acquire Business Analytics Firm Star Analytics, Inc.
IBM yesterday announced a definitive agreement to acquire the software portfolio of Star Analytics Inc., a privately held business analytics company headquartered in Redwood City, California.
Financial terms were not disclosed.
The combination of IBM and Star Analytics software will further advance IBM’s business analytics initiatives, allowing organizations to gain faster access and real-time insight into specialized data sources.
With growing challenges in gaining a more complete view into varying types of data, companies are increasingly looking for ways to automate and provide business users with self-service access to critical information.
Star Analytics software addresses a rising challenge for organizations — helping to automatically integrate essential information, reporting applications and business intelligence tools across their enterprises, on premise or from cloud computing environments.
The software removes typical custom coding for specialized sources that is hard to maintain. It also eliminates manual processes that are cumbersome and time consuming.
“IBM sees an enormous opportunity for our clients to apply Star Analytics to the information they have stored in their financial applications,” said Leslie J Rechan, General Manager, IBM Business Analytics. “And to then easily access it within their IBM performance management and business intelligence solutions.”
IBM Business Analytics
IBM has established the world’s deepest portfolio of Smarter Analytics and Big Data technologies and industry expertise, including almost 9,000 dedicated business analytics and optimization consultants, and 400 researchers.
Nearly 500 of the patents from IBM’s record breaking 20th year of innovation will serve as the building blocks for future analytics innovations that will help businesses and governments unlock the power of big data.
The acquisition is subject to customary closing conditions and is expected to be completed in the first quarter of 2013.
You can learn more about Star Analytics here.
Blue Water
Population growth, massive urbanization and climate change are placing increasing demands on our limited water supply. Forty one percent of the world’s population – that’s 2.3 billion people – live in water-stressed areas; this number is expected to grow to 3.5 billion by 2025.
And according to the United Nations, water use has been growing at more than twice the rate of population increase over the last century.
With advances in technology — deep computing and Big Data analytics linked to sophisticated sensor networks and smart meters — IBM is helping clients and partners make smarter decisions about water management.
By monitoring, measuring and analyzing water systems, from rivers and reservoirs to pumps and pipes, we can better understand the issues around water. IBM is applying its expertise in smart systems and Big Data to help companies, governments and citizens understand and more effectively deal with these issues.
Waterfund LLC announced today that it has signed an agreement with IBM to develop a Water Cost Index (WCI).
Scientists from IBM Research will apply Big Data expertise, acting as a calculation agent, to analyze large and diverse unstructured data sets. This will be used to develop of a WCI framework that would estimate the cost of water in different regions around the world. With its market and financial product expertise, Waterfund will work to structure and commercialize the WCI.
Discerning The Real Cost Of Water
As governments are increasingly forced to turn to the private sector to fund the construction and maintenance of complex water networks, the Rickards Real Cost Water Index™ will serve as a benchmark for helping measure hundreds of critical projects on a like-for-like basis.
Index values will reflect estimated water production costs measured in US dollars per cubic metre for a variety of major global water infrastructure projects ranging from retail water utilities and wholesale water utilities to major transmission projects.
“The backlog of investment in water systems around the world by some estimates approaches $1 trillion – quite apart from the hundreds of millions of people who have never had access to a water or sanitation system at all,” said IBM Distinguished Engineer and Big Green Innovations CTO Peter Williams.
“By creating a benchmark cost for water we intend to harness the capital market to this supremely important cause. If we can make it easier to price investments in the water sector, we can improve the flow of capital into an area where it is desperately needed. We look forward to working with Waterfund to bring this about.”
Scott Rickards, President & CEO of Waterfund said, “The principal reason behind our decision to work with IBM was their unique combination of expertise in the water sector combined with the best data analytics available. Our initiative with IBM will finally bring real financial transparency to the water sector. By calculating the unsubsidized cost of freshwater production using IBM’s Big Data expertise, Waterfund can offer the first flexibly-tailored financial tools to investors in water infrastructure. The Rickards Real Cost Water Index™ highlights the energy costs, interest rate risk, and capital expenditures required to build and maintain large-scale water treatment and delivery networks.”
Smarter Water Management Examples
Typically, investors have turned to the public equity markets to gain exposure to the water sector, with mixed results. The WCI is intended to provide a market benchmark and to spur the development of third-generation financial products for both water producers and investors and to aid the growth of the water sector globally.
Here are two examples of how it would work:
Scenario 1: A Water Agency cannot obtain bank financing for Phase 2 of a seawater desalination plant project due to previous cost overruns on Phase
1. Yet the Agency lacks the water it needs to supply a contractually specified daily volume of water to its largest customer, with a consequent risk of large penalties for each day of insufficient volume. Using strike and trigger values based on the WCI, the Water Agency could purchase a $25 million, 2 year insurance product.
Payout to the Water Agency would be triggered on the total change in its Water Cost Index (as well as some other conditions, such as a specified increase in asset failure costs). This approach would enable the Water Agency to enhance its overall credit profile with the insurance enabled by the WCI, finance Phase 2 of the desalination plant and meet its supply obligations.
Scenario 2: A large desalination and water transmission system project needs to secure private equity and institutional funding alongside that from development banks and sovereign funds, to the tune of one third of the total project cost. To achieve this, the project needs a way to reduce risk to its investors.
Based on movement in the WCI, the project could purchase $50 million in insurance. This would enable the insurance product to then be underwritten by a large reinsurer and allow the project to secure the private sector contribution it needs in order to proceed.
Go here to learn more about IBM Smarter Water Management initiatives. You can also go here to register for a report IBM prepared on why we need smarter water management for our world’s most precious resource.
Live @ Information On Demand 2012: Analyze This

Jason Silva explains to the gathered IBM Information On Demand 2012 his utopic vision of a technology-rich future, one where everything is connected to everything else.
Techno DJ futurist Jason Silva (formerly of Al Gore’s Current TV) kicked off the Information On Demand 2012 event here at the Mandalay Bay Arena by telling us all to “Think Big.”
Though I’d known this was the conference theme, I didn’t realize how big big was until the small, but limber, Silva gave his big presentation.
As he kickstarted the event with a blend of hyper animations and visualizations reeling behind him on a huge video screen in post-MTV fashion, I wanted to stop him and explain that to talk about big things so rapidly would allow a lot of his big ideas to disappear into the ether and to just slow downnnnn.
Jason’s look at the big picture was an interesting one, wherein he described a world that was “hyperconnected,” where we extended sensors into everything…on planes, bridges…even our conference IDs for IOD!
But Silva’s utopian vision could easily merge into a dystopia, if proffered without regard to some of the more realistic and mundane issues presented in a Big Data universe.
Small, and petty human concerns like agendas, and greed, and lack of privacy, and bias, and the other nasty little buggers which make us human.
So, though I wanted to go along with Silva’s optimistic joy ride snowblind to those considerations, someone has to be the buzz kill at this emerging Big Data party and explain there are some very real and concerning issues that will need to be dealt with, none of which Silva seemed even to allude to.

Steve Mills explains to the Information On Demand 2012 press conference Monday morning how economics has made big data not only possible, but inevitable.
But, as techno joy rides go, his was fun even as it went by in the blink of an eye.
Once he blinked, it was IBM Software vice president Robert LeBlanc who really set the stage for the week’s tidings, explaining to the gathered audience of 12,000+ in the Mandalay Bay arena how smarter analytics would be required in the new era of computing.
As always, Leblanc started with some facts: Like how Big Analytics is what’s driving innovation and market growth in IBM’s recent CTO study.
How “technology factors” has risen to the top of the CEO agenda as the number one issue during the study’s last six years.
And how it’s no matter what part of the world you inhabit or what industry you’re in…all and everywhere will be impacted by the need for smarter analytics. This kind of transformational change is a movie we’ve seen before, first with transaction processing in the 1960s, with Internet-enabled e-business in the mid-1990s, and now, the move towards analytics becoming foundational to computing.
Two IBM customers provided two very different, yet compelling, views into this future, one they’re each already living.
ConocoPhillips principal scientist Dr. Phil Anno explained how his organization is utilizing big data analysis to maximize the economic performance of petroleum extraction in the Arctic (and prevent damage to their drilling rigs by shifting ice flows!)
Keith Figlioli, senior VP with Premier, a U.S.-based healthcare IT provider, explained how they’re using IBM technologies to drive substantial costs out of the U.S. healthcare system (he explained that 30 cents on every dollar is wasted on unneeded care and fraud in the U.S.)
Also in the opening general session, we heard from Inhi Cho Suh, VP of Information Management at IBM, who gave an excellent, if quick, summary of the three PureData systems options.
Deepak Advani, who gave an excellent flyover of how big data analytics is bringing about the rapid integration of structured and unstructured data, also highlighted www.analyticszone.com, where you can download some free tools for conducting your own personal analytics.
As the general session concluded, I scooted on over to the day one press conference, where I heard some opening comments from IBM senior vice president Steve Mills.
Mills explained how IT economics laid the red carpet for big data, that it wouldn’t have been possible had the economics of hardware, in particular, been driven down to such an affordable level so as to enable these higher performing systems required for big data analytics.
Mills also highlighted the fact that smarter analytics is a delivery of the real promise of information technology, that now customers are “buying outcomes, and time to value,” as opposed to systems and processes, and that it made sense for them to invest in such projects.
More on the actual announcements as details emerge…