It's Just a Fad!
Why most of the arguments against open data are short-sighted.
Would you allow hackers into your organisation? Would you even invite them in and offer them office space, plenty of coffee, sweets and broadband? Would you invite them to help you fight global poverty?
The British aid agency Oxfam decided to do just that and organised a Hackathon on the 3rd and 4th of December in collaboration with an international network of programmers called the Random Hacks of Kindness. Only a few days earlier Oxfam had published large datasets in the data registry of the International Aid Transparency Initiative and now they presented the tech community with concrete challenges to address based on this data. How, though, can data be the driver behind this new approach to tackle poverty?
Just as oil and minerals are the primary resources in the industrial age, data is the primary resource in an information society. The more we can "exploit" data, the greater its value for society becomes. Unlike oil or minerals, the value of information is not diminished by use. Instead the social and economic value of data increases the more it is used. The core concept of open data is therefore to promote the "re-use" of existing data to the maximum extent possible. The Open Knowledge Foundation (OKF) has developed a definition of what is meant by open. For data to be truly open, it must be both technically and legally open.
Data is technically open when it is machine readable. That means data can be analysed, combined and re-used by computer programs. A financial report stored as a PDF document, for example, is not machine-readable. PDF documents cannot be easily copied, restructured or linked to other data. They cannot be accessed in bulk by computer programs. Data is legally open if the licences that govern the re-use of the data encourage this re-use as much as possible. For open licences, the only restriction for re-use is that data cannot be used without naming its source (attribution).
Tim Berners-Lee, one of the creators of the internet, suggests a more advanced definition and distinguishes five levels of open data, which may sound terribly technical. Does this mean that open data is an obscure niche topic for a few technology enthusiasts? Indeed, the awareness of open data and an understanding of it has been very limited to date in international development. But open data is rapidly gaining in importance. In fact, open data is nothing less than a paradigm shift in development cooperation and in society as a whole. It is high time development professionals became familiar with this new concept.
Defining open data as an important shift is a bold claim. There are three indicators that support this claim:
The first indicator is the growing commitment at the political level to ensure that governments and institutions are "open". Just a few years ago there was not a single "open government data" website anywhere in the world. Today it is hard to keep up with the new open data sites going online. Countries like Moldova, Timor l'Este, Albania and Kenya publish government information in more accessible ways than France, Germany and Austria. But where national governments lag behind, cities are moving ahead. Paris, Berlin, Vienna and many other city administrations around the world publish detailed data relevant at the local level.
At the international level the European Commission has recognised the political and economic value of public sector information and passed the European Public Sector Information Directive back in 2003 to harmonise the regulatory framework of public sector information re-use. Six years later the common European position on open data was strengthened by the Visby Declaration which calls for more open data: "EU member states and community institutions should seek to make data freely accessible in open machine-readable formats, for the benefit of entrepreneurship, research, and transparency. Access to and reuse of public sector information and data should be improved among EU Member States." (Visby declaration 2009, Presidency conclusions).
According to a recent study commissioned by the European Commission, the market for products based on open data was worth about 32 billion Euros in 2010. The release of more open data holds enormous potential. This is the impetus behind the Open Data Strategy the European Commission will adopt at the beginning of December 2011. International initiatives like the Open Government Partnership strengthen this strong trend towards open data at the political level.
Citizens' demands for transparency: The second indicator that supports the rise of open data is the growing demand for transparent and accessible information by citizens. Open data responds to this political demand for more transparency, accountability and civic participation. At an international level the International Budget Partnership, the African Network for Social Accountability and the Open Knowledge Foundation are good examples that illustrate the level of demand. At a national level the work of the Sunlight Foundation in the USA, MySociety in the UK, the Open Data Network in Germany and countless civic online platforms brought together by organisation like the Technology for Transparency Network demonstrate that citizens around the world are demanding better information from public institutions and getting involved in public decision-making based on such information.
The third driver of the open data movement is technology. The publication of open data by itself can neither create economic growth nor satisfy political demands. Tools are needed to access, update, analyse, visualise and share data. These tools exist and their number is growing by the day. Hackathons like the one organised by Oxfam, hackdays, boot camps and app contests are mushrooming and they produce all kinds of new services, tools and ideas based on open data. The World Bank's Apps for Development contest in 2010, the OpenDataChallenge from the European Commission in 2011 and the ongoing Apps4Germany contest are some examples of the many initiatives around the world. The sharing of these new applications through open data camps and on the internet further stimulates new ideas and developments. Most of these applications are open source, so they can be used and adapted by other programmers for different contexts.
One example of such an open source tool is the Ushahidi Crisis Management Platform that has been used nearly fifty times around the world to manage a whole range of situations, such as the 2007 political violence in Kenya, the 2010 Earthquake in Haiti , managing stocks of medicines in several African countries and reporting crimes in Costa Rica. Also using geographic data and mapping have become common place technologies, so now geodata is not only used for military purposes or traffic control, but also to find the nearest restaurant, hairdresser or public toilet. Pioneers in mobile technology like Mobile Active and Trac FM are also bringing the benefits of open data to areas without reliable internet access.
The relevance of many of these technologies for development is obvious. Development cooperation is a vast sector with multiple stakeholders and rife with insufficient information flows. There has been a lot of talk about transparency in development cooperation in the last few years. The outcome document from the recent High Level Forum on Aid Effectiveness in Busan stresses the importance of transparency for aid effectiveness. Given the strong demand for more accessible information, the intense political support for open data and immense economic potential of open data, any efforts to increase transparency in development cooperation should include open data. In the near future the notion of transparency will be inherently linked to open data. The recently published GIZ transparency policy should be reviewed in this respect.
Some stakeholders like the World Bank, the Food and Agriculture Organisation, the UNDP and DFID have already embraced the new paradigm of open data. The World Bank launched its comprehensive Open Data Initiative in 2010 and released vast amounts of raw data to the public in open formats. The International Aid Transparency Initiative (IATI) is the result of the aid effectiveness debate and the Paris-Accra Process. The data standard developed by this initiative is the first open data standard in development cooperation. Donors like the World Bank, the development banks in Africa, Asia and Latin America, the Scandinavian countries and the European Commission have all signed IATI along with some non-governmental donors. Once all current IATI signatories implement this open data standard, more than 80% of all official development aid (ODA) flows will be available in an open format. Comprehensive, standardised and current aid information in open formats will represent a significant contribution to coordination, increasing effectiveness and fighting corruption in aid.
To achieve this, however, more work is required to develop tools that meet concrete information needs in the development sector. For example, the Dutch AKVO platform provides project-level information based on the IATI standard and encourages feedback from beneficiaries, thus potentially closing the feedback loop so often singled out in development as a critical factor for better quality. The Open Aid Partnership launched by the World Bank Institute, IATI and other stakeholders will strongly promote the mapping of aid activities.
However, despite some promising initiatives, the potential for innovation based on open data is still huge, particularly in the growing number of transparency initiatives. The Construction Sector Transparency Initiative, funded by DFID and the World Bank, focuses on publishing contract data in construction projects to curb corruption. While the basic idea is good, the information is published as PDF documents, making it cumbersome to access and impossible to manage using software programmes. As long as data is not open, this initiative will not benefit from the voluntary efforts of the Random Hacks of Kindness and from potential of software applications.
Similarly, the Medicines Transparency Alliance, also funded by DFID and the World Bank, is dedicated to publishing information on the entire delivery chain for medicines with the goal of ensuring reliable supplies, reduced costs and safe medicines. Again, the idea promises to have a strong economic and social impact, but MeTa is limited to doing surveys about the availability of data. The cost of searching the actual data and making it easily available to citizens in the concerned countries is prohibitive for journalists and watchdog organisations.
One final example is the Extractive Industries Transparency Initiative. This initiative seeks to reduce corruption and increase accountability in resource-rich countries by publishing financial flows in extractive industries. However the EITI reports contain only highly aggregated data in PDF format. Publishing raw data about extractive industries in open formats would increase its impact dramatically. Watchdog organisations would be able to use this data in their work, link it to other data sources and make it more accessible to the wider public. The financial gain of making this information easily available to all stakeholders could easily surpass the volume of aid flowing into these countries.
Open data may seem like a niche topic for technology enthusiasts. It may appear just a "a nice to have". However, in view of the real challenges presented by information blockages, corruption, and lack of coordination in development cooperation, the need for open data is obvious. In an information society and knowledge economy, open data is not an option. It is a requirement.