#15 responsibility
Tobias Knobloch / Julia Manske

Big Data, Big Development?

New technologies open up new solutions to long lingering problems. Yet, this promise comes with risks of exclusion. How can big data be used responsibly?

When Nepal was rocked by the earthquake of the century last month, it kicked off more than just traditional humanitarian relief. Facebook’s large server farms also sprang into action to assist victims of the catastrophe. They analysed huge amounts of data to locate users in the crisis region, then sent out an automated request for a status update, asking users to post their health status. This is Facebook’s Safety Check feature and is intended to provide peace of mind to families and friends of potential victims. Organisations like the office for the Coordination of Humanitarian Affairs (OCHA) use data services that evaluate Facebook postings and Tweets, using the content to identify and localise desperate need or new destruction. Big data analyses have made the ad hoc reaction to catastrophes easier and qualitatively better. They can uncover correlations that might not be recognized using common sense alone, thus expanding the scope for action.

… could big data also be used to generate answers to larger development policy issues like hunger, climate change, health, urbanisation and fragility?

Extrapolating from such positive examples, could big data also be used to generate answers to larger development policy issues like hunger, climate change, health, urbanisation and fragility? If we are serious about taking responsibly for solving these problems, then we have to explore this option carefully. This article will provide motivation for taking steps in that direction.

Big data refers to the automated analysis of large amounts of digital data from a number of sources at rapid processing speeds. Big data depends on increases in the processing capacity of computers and the availability and integration of data. Both are growing rapidly, as digitalization and datafication go hand in hand.

Currently most big data applications are used by the private sector who extracts commercially useful information from the analyses. But interest in big data on the part of the government and civil society organisations is also on the rise. Under the “big data for development” catchphrase, taken from a white paper published by the UN’s Global Pulse organisation, options for applying this instrument for socio-economic development in the global South are currently under serious consideration. With some limited success too: In Haiti, for example, the Flowminder NGO analysed mobile phone data to understand patterns of movement following the earthquake, which enabled them to prevent additional cholera outbreaks. Flowminder also helped the Namibian Ministry of Health use satellite and mobile phone data to identify potential new malaria hot spots in order to organize the distribution of preventative measures, such as mosquito nets, more efficiently.

Donor organisations from the USA and Great Britain, such as the Bill and Melina Gates Foundation, are investing in big data research in order to understand events better after the fact and ideally to predict future catastrophes as well.

Hopes are high that additional success stories will soon follow. Donor organisations from the USA and Great Britain, such as the Bill and Melina Gates Foundation, are investing in big data research in order to understand events better after the fact and ideally to predict future catastrophes as well. The key in the explorative analysis of large amounts of data is to identify significant patterns. During the process, the analysis parameters are adjusted for known deviations to allow for increasingly reliable conclusions about questions and problems that exist in reality, such as the geographic spread of an epidemic.

A new leitmotiv for development cooperation?

This trend will not be without consequences for organising development cooperation. The goal is to fight the causes of underdevelopment, conflicts and disease outbreaks. A new analysis paradigm like big data, which is relevant to planning activities where the what is more important than the why, will also shape future international cooperation. Let us take a look at fighting crime with the assistance of data sets, an approach being pursued with great interest both in Anglo-Saxon countries and on the Latin American continent, such as the linking of crowdsourcing and big data in Brazil. Will this mean that development cooperation will only focus on preventative measures from now on? Or are resources more likely to be invested in preventing concrete events? The success of an intervention that prevents a crime is, after all, much easier to measure than the success of an educational programme or anti-corruption campaign. Will this make development goals easier to reach, or might it prevent a sustainable, holistic approach to development?

The type, amount and quality of information and the analysis of same could also create a new gap between the global North and the global South, between the in- and offliners...

The type, amount and quality of information and the analysis of same could also create a new gap between the global North and the global South, between the in- and offliners: If in future IBM, for example, analyses health data from Apple services and devices for a subsidiary of 2,000 employees, this may be very useful to users of these services and devices from the global North, though it might also raise privacy issues. With respect to the global South though, this approach is highly discriminatory since nothing about health problems in developing countries can really be learned from such data sets. This has political consequences as well: Using increasingly elaboration machine programs, complex questions about social behaviour, which to date have been very difficult to analyse and especially to predict, can be answered using big data. The publically visible “likes” on Facebook are one simple example. Analysing them alone allows very accurate (85% to 95%) predictions to be made about a person’s sexual orientation, racial affiliation and political preferences, as scientists from the University of Cambridge have shown. Facebook could influence elections today just by urging specific user groups to vote. This could have more far-reaching consequences for the global South than for the North. There the digital divide between urban and rural populations is much more pronounced than in the global North. The era of big data might make the voices of rural populations or other digitally underrepresented groups even less audible. This works against the ideal of inclusive development.

New opportunities come with inherent risks

In addition to potential opportunities for development, the promising field of big data also brings a multiplicity of new responsibilities for development cooperation. The legitimate uses also raise the issue of how we can protect ourselves from risks we can hardly even begin to imagine.

There is an enormous power imbalance between those who have the data and those who want to use that data for the greater good.

Issues of privacy and proprietary data are particularly virulent: Where do these data come from, who has the data, and to whom do they belong? In developing countries for example CDR (call detail record) data and satellite data are often the only type of data available. They include information about the caller’s and recipient’s locations, for example, based on mobile phone data. This data is in the hands of the telecommunications companies and strict guidelines govern its release. This resulted in considerable delays before data analysis could be done during the Ebola crisis, though it could have contributed to improving crisis response. There is an enormous power imbalance between those who have the data and those who want to use that data for the greater good. At the same time, companies are well aware of the immense value their data has today and will have in future. There is a good reason the social networking service Twitter only provides ten percent of its data for research purposes and requires payment for the rest. This raises the valid question of why a corporation should own data others leave behind.

It seems guidelines good at protecting privacy and individual proprietary data can be bad for policy planning.

An example from Latin American highlights the dilemma that can result from the other end of the spectrum as well, from a decidedly user-friendly data ownership approach: Columbia has a “habeas data” law that prohibits the correlation of data sets in order to protect individual data. This has resulted in an almost complete lack of data on poverty at a local level, for example. And a budget to address real poverty distribution cannot be drawn up on a community level for something that does not exist in the first place. It seems guidelines good at protecting privacy and individual proprietary data can be bad for policy planning.

Big data advocates argue that only anonymous data already present in the net and left behind by users more or less knowingly and freely is accessed in most cases. But studies have shown that it is impossible to ensure that data is completely anonymized. The successful employment of data prognoses to limit the spread of an epidemic like the Ebola crisis in West Africa could therefore easily become a problem for those who fell ill or were in contact with sick people. A woman from the Ivory Coast, for example, might have her access to the government health system limited or blocked after returning from her travels in Sierra Leone due to an analysis of her movement profile.

Conclusions for development cooperation

What conclusions can we draw for development cooperation from the triumphant march of big data? First the positives, the opportunities: It might be possible to create better prognoses with respect to socio-economic development using big data instead of traditional methods. Especially for humanitarian disasters, evaluating these data could help save lives. From intelligent traffic control systems and improved ways of adapting to climate change to the managing of crises and the flow of refugees: There are a huge number of possible scenarios that could lead to positive examples of useful applications for big data.

At the same time, big data could also expand the digital divide in many ways. Developing countries might become dependent on ‘experts’ from the global North in a range of new ways, and marginalized groups might be easier to overlook. And their data could be misused: whether by the private sector to improve product marketing; by repressive governments that want to control their citizens; or by development organisations that prefer to observe the behaviour of populations on screen rather than to interact with them.

How can big data help tap full development potential and achieve development goals more quickly while strengthening state and civil society structures at the same time?

One of the major challenges facing development will be to create effective partnerships between state agencies, the private sector and civil society. The question at hand: How can big data help tap full development potential and achieve development goals more quickly while strengthening state and civil society structures at the same time? Finding the answer will not be easy. On the one hand, the economic expectations of the use of big data by the private sector are lower for the countries of the global South. On the other though, the interests of the different stakeholder groups linked to big data also lead in distinctly different directions.

If big data is understood as the fuel of the information-driven knowledge economy, and if we assume that big data will also leapfrog in many emerging and developing countries as we have observed in other areas of information technology, than it is high time to work with partners in the South to clearly accentuate the open questions and development potential of big data.

Photo: “NO CHAOS” by Daniela Hartmann
2010 - licenced under Creative Commons Attribution (2.0)

Related Articles

Cambodia: How Data Lowers Poverty

Falling rice prices strongly affect the rural poor in Cambodia. Open data and freer information flows help farmers to deal with it.

» more

A New Information Paradigm for International Development: Open Data

Open data is not a niche topic for technology enthusiasts.

» more