« Back to Home Page

Drowning in Big Data, Yet Starving for Information

| Monday April 29th, 2013 | 1 Comment

bigdataOver the past decade, we have entered the Age of Big Data, where digital technology allows people around the world to transmit information at an unthinkable rate – 2.5 quintillion (2.5×1018) bytes of data per day, if you speak the language. I don’t, so I’d rather translate this to monetary terms: In 2010, the industry that has formed around big data management was worth more than $100 billion and was growing at roughly 10 percent a year. The world’s actual volume of data grows much more rapidly, doubling every 18 months according to the IDC.

The numbers are overwhelming, but it’s imperative that we understand what this means for our future. I was privileged to attend two conferences that discussed the potential effects of big data on business and society, respectively.

From data to information

OSISoft, the world’s leading infrastructure for managing real-time data, held it’s annual user’s conference on April 16-19, where corporate leaders discussed how big data is revolutionizing the ways we think about business operations like manufacturing productivity, market research, water management and waste reduction. In industries ranging from paper production to oil extraction, businesses are leveraging software like OSISoft’s PI system to capture data on the most intricate complexities of their businesses. According Jim Crompton, Chevron’s Senior Advisor of Global Upstream IT, there are over 80,000 sensors on each mid-size oil refinery and each one captures data. “The world is drowning in data, yet we are starving for information in many cases.” The great challenge is determining which data really matters. In data speak, this means identifying key indicators.

Leveraging big data for social good

At the Global Philanthropy Forum on April 15-17, I learned that there are no indicators more critical to humanity than those which offer insight into global challenges like poverty, education health and present day human slavery (yes, this is still an enormous problem, as you’ll find in this video).

While the private sector has clearly paved the way for managing big data, nonprofits tackling the above issues are not so well equipped. Fortunately, as I learned, many cutting edge technology companies have already joined nonprofits on the journey to navigate big data for social good. As Deo Niyizonkiza, Founder of Village Health Works asserted, “Protecting human wellbeing protects your markets. Companies see this convergence.”

Companies like SAS and Crimson Hexagon have partnered with UN Global Pulse to “develop advanced analytics software that organizations use to sift through vast quantities of data in search of emerging trends.” Several of the more interesting projects have analyzed Twitter activity to better understand the public dynamics of issues like food inflation in Indonesia and economic outlook in the U.S. While resources like social media can be useful for understanding the immediate concerns of communities, the limitless potential of big data will be driven by the mobile phone.

Mobile empowerment – the Vodafone way

There are 4.6 billion mobile phone subscriptions worldwide. Each time we use our phone, a digital signal is created in real time. When tracked and aggregated, these signals emit information that tell compelling stories about how we live. Companies like Vodafone understand the unprecedented opportunities – both monetary and social – to leverage these signals for social good.

After launching M-Pesa to enable millions of Kenyans to manage their bank accounts through their mobile phones, Vodafone is now rolling out this revolutionary service to 700 million Indian people who are currently unbanked. This means that Vodafone India will soon have the means to access data indicating individual financial health of the majority of India’s population.

The world’s second largest mobile telecom company (behind China Mobile) operates on a micro level as well, pioneering new ways to use big data for social good. Vodafone Americas Foundation designed the Wireless Innovation Project as a global competition to foster creativity of mobile applications to solve the world’s most pressing issues. This year’s winners were unveiled at the Global Philanthropy Forum. Finishing first in the competition, ColdTrace developed a low-cost wireless sensor designed to remotely monitor the temperature of vaccines so that governments, global partners and clinics can protect thousands of children against diseases such as tuberculosis and polio. Mobosens, coming in second, invented a smartphone based platform to collect and share environmental data, such nitrate concentrations of potentially contaminated water. As Fay Arjomandi, Chair of the Vodafone Americas Foundation and Head of Vodafone xone in Silicon Valley, explains, “Mobile technology has the potential to significantly improve people’s lives and we are delighted to support emerging new talent in developing ideas which can have a transformational impact on society.”

Examples like these offer a mere glimpse of a world where opportunities to create social good are as limitless as technology’s capacity to produce big data.

[Image Credit: Creative Commons]


▼▼▼      1 Comment     ▼▼▼

Newsletter Signup
  • DataH

    Travis, good article. With the explosion of big data, companies are faced with data challenges in three different areas. First, you know the type of results you want from your data but it’s computationally difficult to obtain. Second, you know the questions to ask but struggle with the answers and need to do data mining to help find those answers. And third is in the area of data exploration where you need to reveal the unknowns and look through the data for patterns and hidden relationships. The open source HPCC Systems big data processing platform can help companies with these challenges by deriving insights from massive data sets quick and simple. Designed by data scientists, it is a complete integrated solution from data ingestion and data processing to data delivery. More info at http://hpccsystems.com