Boxed In In Bangalore: Analyzing Sentiment On Indian Traffic Congestion

Click to enlarge. With a population of more than 1.2 billion, India is projected to be the world’s most populous country by 2025. By 2050, it is estimated that India’s urban population will constitute nearly half of that country’s total population, straining an already stressed infrastructure. The good news: Urbanization is an indicator of positive economic development. With improved urban planning, India can tackle urbanization challenges and increasing population to create a country that is poised for sustainable growth.

We heard a number of discussions about the potential for social listening intelligence last week at the Smarter Commerce Global Summit in Orlando.

This is an area I’ve been involved in within the IBM team for several years now, starting with some early explorations for how social data could be informative for our marketing efforts stretching all the way back to 2008.

It’s been exciting to watch this space evolve and mature, and with the advent of the IBM Social Sentiment index, we’re starting to see very practical uses of social data for better understanding if not the wisdom, then certainly the perspectives, of the crowd.

Yesterday, IBM held a Smarter Cities Forum in New Delhi, India, where we unveiled a new social sentiment capability to assist our customers in their Smarter Cities engagements.

We also unveiled findings from the latest IBM Social Sentiment Index on traffic, which looked at public sentiment across India’s largest cities — Bangalore, New Delhi and Mumbai.

Boxed In In Bangalore

If you’ve never experienced traffic in India, you can get a taste of the Sunday traffic in this video I shot during my first visit in June 2010.

But the recent analysis of publically available social media showed that the worst congestion in India is primarily caused by accidents and bad weather (three out of four times) when looking at the three cities together.

It also indicated some interesting variations between the three. For example, social conversation in Mumbai about stress around traffic is about half as high as Bangalore and New Delhi; references to the impact of rush hour on congestion in New Delhi are between five and seven times more negative than in Bangalore and Mumbai.

With a wealth of online content and public commentary on social channels such as Twitter and Facebook, city officials need new ways to measure positive, neutral and negative opinions shared by citizens regarding important city issues.

IBM’s advanced analytics and natural language processing technologies used to analyze large volumes of public social media data in order to assess and understand citizen opinions are now available to city governments around the world via new capabilities delivered with the IBM Intelligent Operations Center (IOC) for Smarter Cities.

Making Cities Smarter: The IBM Intelligent Operations Center

The IOC — which combines IBM software and services to integrate city operations through a single dashboard view to help cities improve efficiency — is now augmented with social media analytics capabilities that will help city officials make more informed decisions by looking at unfiltered citizen attitudes and actions, distinguishing between sincerity and sarcasm and even predicting trends as they surface online.

Combining the knowledge that population will rapidly increase in Bangalore, New Delhi and Mumbai in the coming years, with sentiment on commuters’ preferred mode of transportation, could help these cities more accurately plan for needed investments in transportation infrastructure and its potential impact.

City officials could also gauge where public awareness campaigns need to be administered to shift commuters to different modes of transport in order to alleviate growing traffic congestion.

The IBM Social Sentiment Index on transportation in India’s three largest cities surfaced several insights including:

• The top three factors impacting traffic congestion that citizens in each city talked about most online were diverse. Delhites chattered about public transportation, weather and the stress of commuting, while Bangaloreans show more concern for their overall driving experience, construction and parking issues, and Mumbaikars are talking about private transportation, accidents and pollution more often.
• Conversation in Bangalore around parking is viewed three times more negatively than in the other cities. Despite recent infrastructure improvements, less pollution and a solid public transit system, Delhites are experiencing a far higher amount of stress (50 percent) than those in Mumbai (29 percent) or Bangalore (34 percent). Most likely, this can be explained by an uptick in rallies and weather events this year, as well as the recent power outage.
• Surprisingly, sentiment on the topic of construction was relatively positive in Bangalore and New Delhi, and positive and negative sentiment on infrastructure in each was relatively even. Together, these may suggest that the transportation infrastructure improvements being made over the last two years in each city are beginning to positively impact citizens.
• Analysis shows that the relative negative sentiment for rush hour (35 percent) is one of the key drivers impacting traffic in New Delhi, which may explain why citizens talk about stress significantly more than commuters in Mumbai or Bangalore.

By applying analytics capabilities to the area of social media sentiment, organizations are able to better understand public opinions, and city officials can gain additional insights in order to draw logical conclusions about where they should focus their attentions and resources.

For example:

• Take Bangalore, the technology hub of India. Understanding that most commuters prefer private transportation despite negative sentiment around parking and construction may indicate that city officials should consider if it makes sense to advocate for more commuters to use mass transit and invest in infrastructure that will keep up with demand as more companies locate there.
• Since Dehlite’s indicate that public transportation is the preferred mode of transportation, city officials could use this insight to study which areas have high ridership and less road traffic and then implement similar actions in highly congested areas.
• In Mumbai, negative sentiment around traffic and weather at the peak of monsoon season (August) generated 5.5 times more chatter than in November. If the city could measure the fluctuation of public sentiment on these potential causes over time combined with specific weather data like rainfall or temperature, it might be able to better prepare to divert traffic during monsoon season or determine areas where a public safety campaign is needed.

“Like all rapidly growing cities across the world, there are infrastructure growing pains in many Indian cities,” said Guru Banavar, vice president and chief technology officer, Smarter Cities, IBM. “However, when city officials can factor public sentiment — positive, negative or otherwise — around city services like transportation, they can more quickly pinpoint and prioritize areas that are top of mind for their citizens. This could mean more targeted investment, improving a particular city service, more effective communication about a service that is offered, and even surfacing best practices and successful efforts that could be applied to other zones of a city.”

Methodology: IBM Cognos Consumer Insights And 168,000+ Discussions

Public social media content was analyzed by IBM Cognos Consumer Insight, which assessed 168,330 online discussions from September 2011 to September 2012 across social platforms including Twitter, Facebook, Blogs, Forums and News Sources and derived 54,234 High Value Snippets through a series of advanced filtration techniques for insight analysis.

The IBM Social Sentiment Index helps companies tap into consumer desires and make more informed decisions by looking at unfiltered consumer attitudes and actions, distinguishing between sincerity and sarcasm, and even predicting trends.

About the IBM Social Sentiment Index

The IBM Social Sentiment Index uses advanced analytics and natural language processing technologies to analyze large volumes of social media data in order to assess public opinions. The Index can identify and measure positive, negative and neutral sentiments shared in public forums such as Twitter, blogs, message boards and other social media, and provide quick insights into consumer conversations about issues, products and services.

Representing a new form of market research, social sentiment analyses offer organizations new insights that can help them better understand and respond to consumer trends. For more information about IBM Business Analytics go here.

You can also follow the conversation at #IBMIndex on Twitter.

For more information about IBM Smarter Cities go here, and follow the conversation at #smartercities on Twitter.

The IBM Pulse 2012 Circus Begins

Greetings from Viva Las Vegas, Nevada.

I arrived here under the cover of darkness yesterday.

Actually, I arrived in the afternoon, but “cover of darkness” sounds so much more dramatic.

It’s been a crazy week on the road, but we’re only halfway through. Now, Pulse 2012 starts.

Pulse is one of my favorite IBM Software events.  It was Tivoli that brought me back to my native Texas, and to Austin in particular, in the summer of 2001.

I made a lot of great friends during my time working with the Tivoli brand, and I also got a lot of great work done.

And Tivoli has evolved over the past eleven years.  Dramatically.

I need not tell any Tivolian, customer or employee, that.

For my money, it’s evolved all for the better.  The focus of the Tivoli business has far expanded well beyond its core systems management focus, which is what it was centered around when I arrived.

Here’s a factoid: I’ve never seen a Cirques du Soleil performance.  Until last night, when I took in the “Ka” show here at the MGM Grand.

That might seem like a random transition.  But follow me here.  A Cirques du Soleil performance is like one big ecosystem that must be managed across its disparate parts.

A former theatre major myself, I watched in fascination at all the systems that were in play during the Cirques’ performance of “Ka.”  The massive staging and hydraulic systems.  The flying systems that allowed the performers to defy gravity.  The house staff that welcomed the audience into the show.  The audience itself.  The cast. The scores of stagehands in the background.

If you’ve seen a Cirques du Soleil performance, you know of which I speak: It’s a massive and complex linkage of disparate systems coming together to create the wonder that are their shows.

These days, your world is a lot like all those systems.  And to be able to understand and manage it all, and extract new value out of the knowledge you have about all those systems…well, that’s where Tivoli comes in.

I’m going to leave it at that, lest you think I’m completely off my rocker.  But, I’ve done my homework preparing for Pulse 2012, and between the focus on managing mobile, physical assets and infrastructure, the cloud, and the underlying security, there’s plenty of opportunity for systems linkage and improved understanding of those systems.

So, welcome to Las Vegas for Pulse 2012.

Speaking of systems, be sure and check your bathroom for Bengali tigers.  I think it’s just always better to be safe than sorry.

In the meantime, keep an eye here on the Turbo blog and on the Twitter hashtag #ibmpulse.  There’s going to be a firehose of information coming at you these next few days!

IBM SmartCamp Finalist Profile: Palmap — Building Virtual Bridges, Online And Off

Palmap's Dr. Ronald Zhang explains to the IBM SmartCamp Global Finals audience how Palmap's point of sale and indoor mapping technology will change the way people live and shop, not only in China but around the globe.

Dr. Ronald Zhang left his home city of Beijing to attend the University of Central Florida, and didn’t go back home for eight years.

When he returned, how found there were new buildings and roads and shopping malls, and he almost didn’t recognize the place, never mind couldn’t find his way around.

After catching the American entrepreneurial bug during his time in the States, along with his PhD, Dr. Zhang concluded that what was missing in the GPS, location-based services market was the inside out view.

Google Streetview and Keyhole had captured the outside in view, but Dr. Zhang explains that people spend 90% of their time indoors — at shopping malls, restaurants, and the like.  Where was the data feed for them?

And that’s how Palmap came to be founded, a Shanghai-located company now with offices also in Beijing and Xi’an.

Though American entrepreneurialism may seem to be far removed from the Confucian approach to orderly development in the East, that’s precisely what drew Dr. Zhang to the U.S. “With American entrepreneurs, there are no rules, boundaries, you can just go mad and crazy, and only be limited by your imagination. More and more, that’s what’s happening in China, but here (in the U.S.), there’s a spirit that we want to bring back to China.”

Dr. Zhang went on to explain such people “don’t necessarily make revenue yet” but that “they have services that can change the world and make life better.”

His idea for Palmap started around the time the iPhone was released, and he explained that “the Internet changed everything in China, and those technologies are implemented by people like us. So that’s my dream, to do something with my own mind.”

Zhang’s ultimate vision with Palmap is to bridge the divide between click-n-mortar and brick-n-mortar, or as he explained it, “online to offline.”

Between those two endpoints — and not unlike his transcendence of two very different worlds, the U.S. and China — Dr. Zhang and his team plan on making a lot of people happy…and then, and perhaps only then, will the money follow.

Commuters: IBM Feels Your Pain

You really can’t make this stuff up.

Here I am, the next to my last day in Bangalore, which has some of the worst traffic in the world (at least, in terms of what *I* have seen…a guy from Oracle here assured me it was much worse in Bangkok and Sri Lanka), and suddenly the IBM Commuter Pain Study results get released.

The headline: The daily commute in some of the world’s most economically important international cities is longer and more grueling than before imagined, reflecting the failure of transportation infrastructure to keep pace with economic activity.

This is the first time we’ve done such a study on a global basis (earlier versions looked only at U.S. traffic).

IBM surveyed 8,192 motorists in 20 cities on six continents, the majority of whom say that traffic has gotten worse in the past three years.

The congestion in many of today’s developing cities is a relatively recent phenomenon, having paralleled the rapid economic growth of those cities during the past decade or two.

By contrast, the traffic in places like New York, Los Angeles or London developed gradually over many decades, giving officials more time and resources to address the problem. 

For example, the middle class in China is growing rapidly, with the number of new cars registered in Beijing in the first four months of 2010 rising 23.8% to 248,000, according to the Beijing municipal taxation office.

Beijing’s total investments in its subway system are projected to be more than 331.2 billion yuan by 2015 as the city expands the system to more than double its current size, according to Beijing Infrastructure Investment Co., Ltd. 

The city plans to invest 80 billion yuan in 2010 in building its transportation infrastructure. 

The study did offer a number of bright spots.

Forty-eight percent of drivers surveyed in Beijing reported that traffic has improved in the past three years – the high for the survey – reflecting substantial initiatives to improve the transportation network in that city. 

In addition, the commute for drivers in Stockholm, Sweden seems to be, if not pleasant, then largely pain-free. Only 14% of Stockholm drivers surveyed said that roadway traffic negatively affected work or school performance.

Overall, though, the study paints a picture of metropolitan-area commuters in many cities struggling to get to and from work each day.

For example, 57% of all respondents say that roadway traffic has negatively affected their health, but that percentage is 96% in New Delhi and 95% in Beijing.

29% overall say that roadway traffic has negatively affected work or school performance, but that percentage rises to 84% in Beijing, 62% in New Delhi, and 56% in Mexico City.

Moscow was notable for the duration of its traffic jams. Drivers there reported an average delay of two-and-a-half hours when asked to report the length of the worst traffic jam they experienced in the past three years.

A Top 10 List You DON’T Want To Be On

IBM compiled the results of the survey into an Index that ranks the emotional and economic toll of commuting in each city on a scale of one to 100, with 100 being the most onerous.

The Index reveals a tremendous disparity in the pain of the daily commute from city to city. Stockholm had the least painful commute of the cities studied, followed by Melbourne and Houston (which tied) and New York City. Here’s how the cities stack up:

The index is comprised of 10 issues:

1. Commuting time
2. Time stuck in traffic, agreement that:
3. Price of gas is already too high
4. Traffic has gotten worse
5. Start-stop time is a problem
6. Driving causes stress
7. Driving causes anger
8. Traffic affects work
9. Traffic so bad driving stopped
10. Decided not to make trip due to traffic.

Drum roll, please (or, with due deference to Bangalorians, a nice long honk of the horn), here are the Top 20 winners (err, losers):

Calgon, Take Me Away: The IBM Commuter Pain Index lists the world’s most onerous cities for traffic.

1. Beijing: 99
2. Mexico City: 99
3. Johannesburg: 97
4. Moscow: 84
5. New Delhi: 81
6. Sao Palo: 75
7. Milan: 52
8. Buenos Aires: 50
9. Madrid: 48
10. London: 36
11. Paris: 36
12. Toronto: 32
13. Amsterdam: 25
14. Los Angeles: 25
15. Berlin: 24
16. Montreal: 23
17. New York: 19
18. Houston: 17
19. Melbourne: 17
20. Stockholm: 15

"Traditional solutions — building more roads — will not be enough to overcome the growth of traffic in these rapidly developing cities, so multiple solutions need to be deployed simultaneously to avoid a failure of the transportation networks," said Naveen Lamba, IBM’s global industry lead for intelligent transportation.

"New techniques are required that empower transportation officials to better understand and proactively manage the flow of traffic."

IBM Global Commuter Pain Survey: Major Findings

Analysis of the survey results indicated a number of key findings related to how traffic impacts commuters:

• 49% of drivers in the 20 cities think that roadway traffic has gotten worse in the last three years, and 18% think it has gotten a lot worse.  Five percent say traffic has improved substantially, with only Beijing (16%) and New Delhi (17%) reaching double digit scores.  There are seven trouble spots based on the bottom two box scores (ranking traffic as "somewhat" or "a lot worse"):  Johannesburg (80%), Moscow (64%), Toronto (64%), Mexico City (62%), Sao Paulo (61%), Milan (59%) and Buenos Aires (57%).
• 87% of the respondents have been stuck in roadway traffic in the last three years.  The average delay is one hour. The "best" cities are Melbourne, Stockholm and Buenos Aires, where 25% or more say they have never been stuck in traffic. On the other end of the spectrum, the average reported delay in Moscow is 2.5 hours, where more than 40% say they have been stuck in traffic for more than three hours.
• 31% of respondents said that during the past three years traffic has been so bad that they turned around and went home. The percentage in Beijing, however, is 69%, the high for the survey; and only 15% in Berlin, representing the low.
• If commuting time could be reduced, 16% of respondents worldwide would choose to work more. In New Delhi, 40% said they would work more, the high for the survey; while 5% in Madrid would work more, representing the low.

The Commuter Pain Survey was conducted by IBM to better understand consumer thinking toward traffic congestion as the issue reaches crisis proportions nationwide and higher levels of auto emissions stir environmental concerns.

These events are impacting communities around the world, where governments, citizens and private sector organizations are looking beyond traditional remedies like additional roads and greater access to public transportation to reverse the negative impacts of increased road congestion.

IBM is actively working in the area of Smarter Transportation using a worldwide team of scientists, industry experts and IT services professionals to research, test and deploy new traffic information management capabilities in cities around the world.

Findings from the Commuter Pain Survey will be used to assess citizen concerns about traffic and commuter issues; expand solutions like automated tolling, real-time traffic prediction, congestion charging, and intelligent route planning; and serve as a basis for pioneering innovative new approaches to traffic mitigation.

As for me, I’m about to head back into the Bangalore traffic fray once more, but this time with some comfort that traffic is a LOT worse in other parts of India and the rest of the world, and with some hope that IBM smarter transportation folks are going to be working on this horrible plight.