Big Data Needs to Make the World a Better Place

Big data has been used for the better part of the past decade, some of the first uses being by epidemiologists and biological scientists to track the spread of disease. Though Google Analytics launched in 2005/2006, my undergrad degree in marketing at a major university in the US was at that same time and involved little to no mention of the power of aggregated, automated data. Today, big data and analytics in the Global North[1] are used as a core part of the marketing, advertising, and entertainment industries, to name a few.

When applied to contexts in economically developing countries, even the perfect set of data is not enough to predict what will happen, as infrastructure is lacking, government systems have limited capacity, and a large part of the population is impoverished. Still, the power of big data is no less dramatic than say, for the first time in world history, getting some kind record of the overwhelming majority of the world’s population. If someone can figure out how to aggregate this data (respecting privacy, no doubt) in a meaningful way, we can figure what the situation is – be it the impact of a natural disaster, rates of hunger, spikes in unemployment, increases in violence – when it is happening. And that means we can respond more effectively and efficiently.

Whereas a large push to use big data in sub-Saharan Africa, South Asia and parts of Latin America is to better human development, big data in the Global North has largely been used for for-profit endeavors. There have been, however, community-betterment efforts of big data in the Global North. Bloomberg used big data to reduce crime in New York City, the US Center for Disease Control uses big data to help track the flu, and crowdfunding sites online use big data to help people get money for projects that they could otherwise not fund.

Despite examples of social good uses of big data in the Global North, as a proponent of using big data as part of international humanitarian and development solutions, one thing does continually bother me – though big data has revolutionized certain industries, I am not convinced it has made countries that use it the most any better. The past 10 years have seen the rise of big data, but they have also been difficult years for the Global North, not least of which, the United States.

Here are some statistics about the US between the years 2000 and 2010:[2]

According to the American Foundation for Suicide Prevention, suicide death rates rose.

The U.S. Census reports poverty rates rose.

Inflation data shows the cost of higher education versus inflation skyrocketed up.

Of course all of the above are due to a huge set of variables, including national policies, consumer behavior, and the world economy. Even if we were able to model every driving factor and outcome of the trends listed above and present that data to everyone who has any influence, there is no guarantee a person will make the best decision for either themselves or for the country. Likewise, when politicians, constituents and bystanders make decisions, there is no guarantee good data will be followed.

But why is it that with all of the data available, dialogues going on in present-day America reflect how bullying, a lack of respect of women, mass shootings, and unfair immigration practices are destroying our country? With all of the data available, why are we still talking about the same topics as 10, 20, 30, even 40 years ago?

I personally think the reason goes back to the predominant goal industries have in using big data - to make a profit in the short-term. There is a huge imbalance to what big data can currently offer to the average person in countries that are at the forefront of its usage curve. We have figured out how to generate an automatic song recommendation to someone with a few clicks of a button, but we have not figured out how to use big data to keep someone out of poverty. You can use big data to figure out what pair of shoes to buy based on your preferences, but not a customized plan based on those same characteristics – occupation, location, age and interests – to overcome suicidal thoughts.

Big data will not be able to tell us whether the world is a better place. That is determined by other measures - happiness, access to services, security, safety, expression, among others. What big data can do is improve the performance of the measures we use to determine if the world is a better place. It’s time to make big data less about consumer products. Simply put, big data in the Global North needs to become more relevant to issues that truly matter.

---

[1] When I use this term, I generally mean economically developed countries, including the US, Canada, Europe, Australia, Singapore, Japan and New Zealand

[2] To be fair, not all is lost. Notably, murder rates are on the general decline.

International Development Needs to Disrupt Itself

A few months ago, I heard Joi Ito, the current director of MIT’s Media Lab speak. One of the points I found the most telling in his talk was the idea of how the Internet has disrupted the traditional order of operations in creating a new technological innovation, especially at the global level. Due to heavy research and development costs, overhead, and patent protection, technological innovation pre-Internet often came in a very top-down approach at the behest of large government organizations or private corporations.

The Internet changed all of that; the Internet allowed individuals or small groups of people with little to no assets to create something and find investors to bring it to scale. One of Ito’s examples was YouTube, which was hatched by three software developers in 2005. Not knowing what the precise use of YouTube would be, the developers concentrated on creating the product, and eventually attached it to an online dating service that allowed customers to upload video clips. With time, YouTube migrated to MySpace before becoming its own platform that was eventually bought by Google.

This sort of disruption is seen on a daily basis in New York. In fact, Mayor Bloomberg even started a large campaign called Made in NY, which gives media backing to more than 3,000 tech start-ups in the city. From social media, to analytics, to online shopping, to crowdfunding, interactive mapping, and more, the transformative potential of the next wave of Internet behemoths is well known and well documented.

Indeed, this wave of innovation and entrepreneurship extends far beyond the reaches of major tech cities; in fact, it goes beyond small cities in the States – including my hometown of Richmond, VA. GSMA’s Mobile for Development Intelligence recently published a report on scaling mobile technology in economically developing contexts. In the report, they write about seven Internet innovation centers across Africa, including iHub in Nairobi.

If Internet technological innovation at this level has made it this far, if Internet technology is so pervasive in our lives, why then has it not been embraced at a strategic level in much of the international development community? ICT brings efficient, low-cost solutions, which are two characteristics continually sought out by an increasingly financially strained donor community. Among many other reasons, I believe one of the driving factors is that the disruption that has occurred in the ICT community has not happened in international development world.

One of the favorite international development expressions is,

            “In the context of.”

We as development practitioners are taught to build a “context” – a case – for everything that we do, and often for good reason. Solving one small problem in an environment with little infrastructure, high gender inequality, the majority of a population impoverished, and with highly imperfect systems means that nothing can be looked at in isolation.

A lack of due diligence before deploying a solution could very easily negatively affect those we are trying to help. Distributing computers to slum-dwelling youth to better education resources might put them at high risk for violence by non-recipients trying to steal the computer. Connecting adult females to job boards targeting women might ostracize these new workforce participants from their community. Automatically distributing internationally sourced food at the end of a bad crop season might crowd out farmers who cannot afford to sell their stocks at a lower price.

It takes time to understand a context, to understand how a solution to one problem can and should work in conjunction with solutions in the same community. It takes time to conduct literature reviews, secure ongoing donor funding, find political support, and build an argument as to how, why and when a solution should be implemented.

What this then means is that the pace of ICT solutions on the technical side often outpaces the international development side. As we all know, a perfectly executed piece of software, hardware, operating system, whatever, is useless if unused. Without constant market feedback loops telling us whether a solution has worked, the iterative process to innovation loses steam quickly.  

I have no doubt that ICT-based international development solutions will continue to grow in prominence and relevance in coming years. However, in order to achieve a better value add, the international development community – practitioners, donors, governments – will have to change the approach. We have to figure out how to disrupt ourselves.