Introduction

How long do you wait to get an appointment with your doctor? How much certain drugs cost and how do you have access to them? Who is paying for these services and how? Who is going to be held accountable if I don’t get the treatment I need? These are simple questions often posed by health systems users across the developing and developed world; questions that do not have clear answers because health systems are complex institutions with numerous actors performing varied roles.

The United Nations recently approved the Sustainable Development Goals (SDGs). The health-related goals are to achieve universal health coverage, including financial risk protection; access to quality essential health-care services and access to safe, effective, quality and affordable essential medicines and vaccines for all by 2030 (WHO 2016). The World Bank (2017) estimates that globally, individuals and businesses pay 1.5 trillion per year in bribes, more than 10 times the value of development assistance. Therefore, the United Nations’ Sustainable Development Goal 16, which is calling for more transparent, accountable and participatory institutions at all levels of government, is an equally important goal for the achievement of universal health coverage.

With the emergence of the open data community in the development field, it is appropriate to ask if and how data initiatives can improve health services. In this paper, I explore three projects in Latin America, focusing on Peru, Mexico and Uruguay, focusing on the question Can open data initiatives improve health service delivery, and if so, how? The projects were analysed as part of the Latin American Open Data Initiative research agenda. They demonstrate different approaches by civil society and government institutions to develop and support transparency and accountability in health service delivery, offering a comparative view of the potential and limitations of open data initiatives.

First, I situate these cases in the context of health service delivery and transparency. I also provide a brief explanation of the open data movement and provide the rationale for the case studies selected. Second, I describe the three cases following a common framework. Third, I provide a detailed analysis about the lessons learned from these cases. Finally, I conclude by providing a set of lessons that could act as a preliminary hypothesis for further exploration of the field.

Despite the considerable resources governments and donors invest in health service provision and accountability, there is little clarity about what works in this field. The World Bank (2004) first initiated the idea that citizen feedback can contribute to improve public services in the development field. The so-called ‘short route’ to accountability would allow citizens to exercise their voice, speak out against corruption and contribute to the improvement of public services. This rationale comes from two different ideological standpoints: the new public management agenda and the participatory politics approach (Joshi 2011). However, a recent review, commissioned by United States Agency for International Development (Edstrom 2015), notes how difficult it is to find evidence of initiatives in the field and ‘rigorous’ evidence on impact.

The relatively simple question of whether citizen engagement can improve health service delivery does not have a simple answer. Projects aimed at strengthening citizen oversight usually need political buy-in, an enabling environment, and dynamic civil society organisations able to exercise their voice (Edstrom 2015).

Citizen engagement projects in this field involve three types of activities:

• Information dissemination (such as citizen charters);
• Consultation and participation (for example, health or users’ groups, citizens scorecards, third party monitoring); and
• Citizen empowerment (such as health clinics management committees).

The question of whether citizen engagement can improve health service delivery becomes more complex when we consider the accountability interventions (Joshi 2011). It is not a given that accountability initiatives will expose corruption, because the issues of transparency and accountability are not necessarily linked (Fox 2007). Accountability initiatives might not necessarily elicit a response from the concerned authorities. Also, while accountability initiatives might contribute to enhancing citizenship by improving the ability of the poor to claim their rights, it is not very clear how to achieve this or how to include the voices of the poor in such claims (Joshi 2011).

Transparency initiatives in the health sector encounter similar problems. A scoping study collected several examples of Information and Communication Technologies (ICTs) for good governance in the health sector, and clustered them into four conceptual categories: better information for transparency, usable data for decision-making, citizen mobilisation for accountability, and process automation for fraud prevention (Holeman et al 2016).3b880325521d The study concluded that: “Innovative digital approaches are increasingly being used to facilitate good governance in the health sectors of LMICs but evidence of their effectiveness is still limited. More empirical studies are needed to measure concrete impacts, document mechanisms of action, and elucidate the political and sociotechnical dynamics that make designing and implementing ICTs for good governance so complex.”

The present study contributes to the existing knowledge, revealing mixed evidence about the impact of ICTs in the health sector. The findings suggest that ICT projects can indeed address several structural problems of a health system by collecting, collating, digitising and publishing information, and enable citizens to make informed choices, and to denounce corruption. However, macro factors such as political institutions and internet access contribute to project outcomes. The evidence presented here shows that this is an uncertain and complex field and that there is yet more room to test what does and does not work in the use of open data for health.

1.1 What is the role of open data in health service delivery?

Open data refers to data (datasets) that is available in open formats for anyone to freely use and reuse. There is a legal and a technical aspect embedded in this definition. Initially advocated by Sir Tim Berners Lee (2006) as a way to promote the evolution of the web, the field gained traction as a movement demanding open data from governments, multilateral organisations, and the private sector. In the early days, the mantra was “give us raw data” (Pollock 2007). Governments built open data portals for citizens to use, and apps using data for transport services and urban issues emerged (Whitman 2010). To encourage people to take an interest and participate, advocates demanded a “flood of data from governments” (Eaves 2010).

Open data can be conceived in several, often ambiguous, ways (Yu 2010). Open data is an empty signifier; just like with the word games which can refer to very different games from chess to football (Wittgenstein 1939), open data can mean different things depending on the context:

In this paper, I consider open data as a problem-solving approach, led by civil society organisations, to improve health service delivery. I argue that citizens can seek to control, engage and co-produce services with the state and/or the private sector from the demand side. By control, I mean the traditional exercise of voice by users in order to improve transparency and accountability. Engage means the use of data to seek more transparency, accountability and the improvement of a public service. Co-production is the voluntary participation of different civil society organisations in co-producing and co-innovating services using open government data. This type of co-production assumes an active role of public servants, but it also challenges the orthodoxy that public servants are completely in charge of the process (Pestoff 2006). This is because once data is released actors can potentially reuse the data in unexpected ways, beyond the direct control of government.2dfec72cdb26

While open data portals are good repositories, demand seems so far to be low, and there are still constraints on the provision of data. The constraints include availability, quality, timeliness of the provision, and political will (Web Foundation 2016). Further, not all countries have the capacity to release open data in meaningful ways, and not all have developed the necessary legal framework (Davies et al. 2015).

Some studies suggest that open data is associated with positive outcomes such as transparency, accountability efficiency, and efficacy in service delivery (Davies 2010). Some practitioners have argued that data is public digital infrastructure like roads, bridges, or rails, free for all to use as a “living infrastructure” (Moncecchi 2012, Scrollini 2015). They have argued that open data is a public good that will improve and increase in value as more people use it. Developing and adapting its publication and use leads to more accurate and useful data, which can improve the whole system (Scrollini 2015). However, the use of open data is rarely straightforward.

In assuming that the use of data for political, social, and economic outcomes is valuable, government agencies and the private sector across the developing world have begun to promote the use of data through events such as hackathons and datathons. Such events would gather technologists and policy experts to tackle complex issues and showed the potential for establishing new types of communities and exploring new ideas, and even promoted political mobilisation. However, despite expectations, these efforts did not support sustained innovation (Mochinaki 2015).

1.2 Methodology and approach

The Latin American Open Data Initiative (ILDA) is a research initiative seeking to understand and promote the use of open data in Latin America. Avina Foundation hosts the initiative, and the International Development Research Centre (IDRC) also supports the initiative as part of its Open Data for Development programme (OD4D). ILDA developed a set of strategic initiatives to test and explore the value of open data, and set out to conduct basic research on this topic. Several Latin American open data conferences considered health service delivery a strategic area that required further research, forming the basis for the projects explained in this paper.

ILDA explored the case of Ojo Publico in Peru. Ojo Publico is a collection of young journalists who engage in novel practices of data journalism in Latin America, becoming foremost practitioners in this field. They developed an investigative reporting programme on the Peruvian health sector, which mostly operates based on private contractors. To complement the investigative reporting process, they developed Cuidadosintensivos.com (Intensive Care), a website where users could find information about the performance of the Peruvian health system. The author undertook interviews with key informants to understand the results of this platform on the Peruvian system.

As part of its strategic initiative on health service delivery, ILDA also engaged the Ministry of Health of Uruguay and DATA, a civic tech and open data association.1c9da91a0528 The intention was to develop a programme that could publish and use performance data. The project developed a tool, Atuservicio.uy, details of which I explain further below.

ILDA also engaged Sonora Ciudadana and Civica Digital in Mexico to explore how open data standards could contribute to improving accountability in health service delivery. Sonora Ciudadana is an NGO working on transparency and accountability issues in the Mexican state of Sonora. They developed the programme “La rebelión de los enfermos” (my translation: “The Rebellion of the Sick”) to raise awareness about the difficult conditions Mexican hospitals face and provide a channel for grievances. Codeando Mexico is a civic tech organisation exploring the creation and use of civic application in Mexico and is one of the pioneers of open data use. As the initiative seeks to explore the value and use of open data in this field, it adopted a Participatory Action Research approach (PAR). Action research assumes the researcher works alongside members of a given community. The researchers work with counterparts to generate evidence and knowledge, and to develop possible new approaches or solutions to tackle practical issues (Herr and Anderson 2005). In this case, ILDA agreed on the objectives with selected interested organisations. They then developed an intervention to achieve these goals and enable reflection by feeding back the knowledge created collectively to relevant partners.

The three chosen cases exhibit various ways of using open data to improve health service delivery. The following table provides a summary of the key elements these cases illustrate:

• Aim (what did the actors aim to achieve with the project)?
• Context (does the country or sector provide an enabling environment to carry forward the initiative)?
• Open data infrastructure (what kind of data and tools are available in a given context)?
• Results (did the project deliver the expected results)?

The next section explores the case studies, showing how they evolved as well as their achievements, potential and limits.

2. Cuidados Intensivos: checking on the Peruvian health system

2.1 Context

Peru is one of the fastest growing economies in Latin America, with an average of 5.3% GDP growth and a rapidly growing middle class (OECD 2016). Such growth has put pressure on the public sector to deliver better social outcomes. However, while open government efforts have been part of Peru´s agenda for at least 20 years, there are several challenges such as corruption in public procurement and lack of access to information (OECD 2016). A member of the Open Government Partnership (OGP), Peru has committed to the Open Government Principles. However, the lack of a unified Open Government policy (Casas 2012) led to a fragmentation of open government initiatives and poor implementation. There is little (or no) room for dialogue around the open government process.

The health system in Peru is fragmented. There are five entities coordinating health service delivery including the Ministry of Health, EsSalud (National Health Insurance system), the Armed Forces, the Police and the private sector (WHO 2016). Together, EsSalud and the private sector cover around 40% of the population. EsSalud both contracts out part of its services and provides its own clinic services. In recent years, private providers have become critical for the provision of health services (Torres and Huacles 2015).

Ojo Público, as discussed in the previous section, is an independent digital media outlet carrying out investigative journalism in Peru. They decided to do research on health service provision because of a lack of transparency in the health sector and the powerful corporate interests operating in the private sector. They combined traditional journalism with use of freedom of information laws and data journalism practices to explore the complex subject.

2.2 Aim

Ojo Publico´s aim was to expose the complex situation of the health system in Peru. The International Consortium of Journalists (ICJ), the Knight Foundation, Hacklabs and Hivos supported the initiative.

Their work uncovered three key issues: lack of regulation of the private sector to prevent potential abusive practices, lack of transparency and enforcement of fines for breaching terms of service, and lack of an effective redress mechanism for grievances.

Their research identified a monopoly in the private sector where eight companies formed a cartel where they provided insurance, clinical services and drugs, with little control and oversight of the authorities (Torres and Huacles 2015). The research also established that none of the clinics paid fines imposed for regulatory breaches, and thus owed the State around 10 million soles (approximately USD 3 million).

A website dedicated to publishing the information released helped to raise awareness of the issues covered in the investigative reporting pieces, while an app with various features was the ‘actionable’ item of Ojo Publico’s strategy. The website would allow users to see who had permission to practice in the health sector and track any fines accumulated. In this manner, Ojo Publico was able to obtain and use data about health providers to provide more transparency and promote better choices.

2.3 Open Data Infrastructure

The Peruvian situation presented a number of challenges, chief of which was that the office of e-government had little influence and the data could not be obtained through them because many government departments were still operating a manual system. However, there was an opportunity to do something because 56% of the population have access to the internet (Internet World Stats 2017).

Ojo Publico therefore decided to follow an eclectic strategy to obtain the data. First, they filled 52 freedom of information (FOI) requests directed to several institutions including: the National Competition Authority, the Copyright Authority and the National Authority on Health (SuSalud, EsSalud, the Ministry of Health and the Police). They also obtained reports that were already online. Most of the replies to the FOI requests were on paper. They compiled a database and systematised the information before combining them with databases from the private and public sectors, such as the National Health Institute, the Peruvian Stock Exchange, the Tax Authority, the Judiciary and the media. In some cases, Ojo Publico had to pay for the data so they created their own data infrastructure by joining disparate databases or obtaining data through different legal or social processes. They used four key pieces of data to build an app with data on private clinics, data on insurance companies and data on public health investments (Torres and Hidalgo 2016).

Ojo Publico collected information on 61,372 doctors, 9,920 clinics and 21 insurance companies. This information was not available in a centralised and open format before this initiative. They encountered many problems with the official data. Most of the data was outdated, the format was not open and the team had to correct mistakes by hand and extract date from the internet by scraping (parsing) pdf files. Hard copy data (on paper) was crucial in order to ensure reliability, as authorities were not digitising files. They also had to deal with complex official jargon, which had impacts on the presentation of data. For instance, when official reports said that a clinic had an operational risk level of 6%, what it meant was that it did not meet the 94% of the care standards (Torres and Hidalgo 2016).

Ojo Publico therefore built its own data infrastructure collecting, treating, establishing categories and eventually using the data. By combining different sources of information, Ojo Publico ended up with a data set that was unavailable before the project started. It is important to note that Cuidados Intensivos was developed using free/open source tools.

2.4 Results

The health system is one of the main challenges facing Peru. Thus, the publication of the story brought the issues into the spotlight. The app allowed access to an unprecedented amount of information about private actors in the health sector in Peru and this improved their performance. The example showed the potential of data journalism where journalists are willing to utilise traditional and non-traditional tools. Ojo Publico has continued to develop platforms on other topics following a similar model, such as stolen works of art and tax avoidance.

Cuidados Intensivos increased the national visibility of Ojo Publico, and established its reputation as a dynamic, professional news outlet. It also helped Ojo Publico to reach out to users’ rights organisations working in this field. Ojo Publico collected stories from these organisations and gave voice to issues around malpractice and abuse in the health system.

Accordingly, there was some evidence that the project improved accountability in the Peruvian health system. Accountability here is defined in a broad way that emphasises that public authorities need to explain their actions (Mulgan 2000).A narrower conception is that public authorities are subjected to external public scrutiny leading to a particular outcome such as a resignation, sanctions or a change in practice and procedure.

Because of the project, authorities were obliged to release information to Ojo Publico. This in turn enabled more people to monitor the performance of health providers in the country. This was the first time that health system information was systematised in such a way that it could be used for accountability purposes.

3. La rebelión de los enfermos: engaging the Mexico Health System

3.1 Context

Mexico’s health system is complex and fragmented. The state has invested massive amounts in public health to ensure universal health coverage, but a significant part of Mexican population is still not covered (OECD 2016). There are six institutions delivering health services in Mexico, each with its own independent network of doctors, clinics, hospitals and pharmacies (Mannat Jones 2015). This creates a bulky structure that presents several implementation problems.

Mexico has a good ranking on the Open Data Barometer (Web Foundation 2016), but the implementation of the open data policy is difficult because of several challenges (OECD 2016). For instance, only 56% of the Mexican population has access to the internet (International Telecommunications Union 2016).

Sonora Ciudadana is an NGO based in the state of Sonora (northern Mexico). It undertakes various initiatives to promote transparency at national and state level. In 2008, Sonora Ciudadana received a complaint from Abel Montenegro who could not access medicines and other services at ISSSTESON9ae83f488866 (one of the six Mexican institutions). The institution discriminated against him based on his previous health condition. Sonora Ciudadana embarked on a litigation strategy, taking the case to the Supreme Court of Justice. Recognising that this kind of discriminatory practice was systemic, they filed more cases.

It is important to note that Sonora Ciudadana is not part of the open data community. It is a traditional human rights organisation that also works on issues of transparency. Accordingly, parallel to the court cases, they launched access to information requests to obtain data on ISSSTESON’s performance. Interestingly, ISSSTESON did not deny the information provided. However, the information provided was in paper format, which made the process difficult for all parties involved (Michel 2015).

3.2 Aim

Eventually, after collecting significant evidence, Sonora Ciudadana launched a campaign under the banner “La Rebelion de los enfermos”. The campaign aimed to highlight the difficulties users faced in accessing the system and to document cases of injustice.

As the campaign advanced, Sonora Ciudadana set up a website where users could document their complaints. The website was designed with the dual purpose of soliciting information and allowing people to make official complaints. It used some of the information obtained through FOI requests and datasets obtained from the federal government. This allowed people to map and understand the methods of work and performance of clinics and hospitals across Mexico. The data compilation was a strategic approach to inform their chosen strategy of litigation. Nonetheless, the tool was designed to cover all the health providers, not just ISSSTESON the defendant.

3.3 Open Data Infrastructure

Sonora Ciudadana had to invest significantly in the digitalisation of the information, as most of the information availed from FOI requests was recorded manually on paper. Sonora Ciudadana therefore decided to collaborate with ILDA to explore how the data could be structured and eventually used.

ILDA worked with Codeando Mexico, a civic-tech organisation, to explore how Sonora could access data from the public sector. The process led to the identification of significant problems in the data, such as lack of structure, reliability and formatting issues. Thus, while data was available, there was a significant cost associated with processing it. Codeando conducted an extensive review of the data infrastructure of ISSSTESON and other Mexican authorities, developing a template for publishing data about medical institutions, their basic infrastructure, service metrics, human resources metrics and cost metrics (Codeando 2015). The programme was designed to structure information in a manner that could foster comparison. It also had to be simple and user-friendly enough for adoption by health institutions. While the programme remains available to all interested parties, lack of resources and difficulties in collecting the data prevented the project from scaling up.

3.4 Results

La Rebelion de los enfermos demonstrates a traditional campaign that delivered on citizen’s rights. Sonora Ciudadana managed to establish a significant legal precedent that effectively changed practices at one of Mexico’s largest health providers. The decision to embark on a digital journey was a significant change for a traditional accountability- oriented organisation whose main strategy had been litigation. One of the main outcomes was the creation of an open-access programme template that other organisations can improve down the road. Crucially, the project illustrates that simply providing better data does not guarantee improved monitoring.

4. Atuservicio.uy: Co-producing a health application

4.1. Context

A series of major reforms in the Uruguayan national health system have led to almost full coverage for the population. As a result, every February, Uruguayans can choose whether they will change their health service provider. Significant amounts of public funding are invested in the system, which offers a mix of public, semi-public and private providers. Funding for the providers depends on the number of users they have. Although this was not the only factor behind the funding, it was a major one, so providers competed for healthcare users.They developed media marketing campaigns and illegally offered cash to users to incentivise them to switch providers. In 2008, the Ministry of Public Health, worried about such practices, started publishing information in excel tables on their website about the system performance. The press used these tables to produce news pieces about the system. However, users rarely retrieved the tables because the language was difficult to understand, the information was confusing, and it was difficult to compare healthcare providers. Further, the information tables that assumed the users would have access to some form of proprietary software. The Ministry estimated that there were just 400 visits to the resources per year, while the raw data published in the open data portal received 2,000 visits.

In 2013, the civil society organisation, DATA, seized the opportunity to work with the data available from the Ministry. They collaborated with a local online media outlet and developed a tool to visualise and rank providers according to user preferences. The tool’s apt name was “Window Transfer,” referring to the short time frame Uruguayans had to choose their provider. DATA extracted the datasets from the ministry websites, cleaned them and designed an interface to enable users to understand the data easily. The online media outlet helped to spread the word and the website received approximately 6,000 visits in February 2014 alone. The project was open-source software based, so the treatment data received was transparent.

In the context of the Uruguayan Open Government Partnership process,ccacdeaed56c DATA and the Ministry of Public Health explored the possibility of an alliance. The Ministry intended to create a similar website to the one DATA had created, but it was not possible to find a suitable provider. DATA had the expertise to carry forward this mission but only had a basic understanding of the technical and policy nuances of health data. Eventually, the Ministry and DATA set up a formal partnership to create and produce the app, with both entities committing human and financial resources to its development. DATA insisted on developing this work using open-source software to make the work transparent and to allow eventual replication of the process. It is important to consider that according to the International Telecommunications Union (ITU), 70% of the Uruguayan population has access to the internet.

4.2 Aim

DATA engaged with a group of mid-level managers who had political support. The process was emergent and made an effort to be bottom-up. DATA and the Ministry’s team held meetings defining the scope of the information to include. DATA pushed for the inclusion of more information, whereas the ministry wanted to be more cautious about what to publish. The Ministry initially classified part of the information they released as “reserved” under the access to information law provisions. DATA and the Ministry negotiated over which information should be included in the application. Certain types of information proved controversial, for instance, the number of affiliates of each institution.

DATA had an initial bias towards user choice and thought the more people who exercised their right to switch provider, the better. However, the Ministry disagreed with this and argued that switching was not the ultimate goal as it could jeopardise the stability of the system. The ultimate goal for the Ministry was to improve the system and enable the users to voice their concerns to the Ministry.93c62759ddef To enable the development of the project, DATA agreed to work within the framework of the broader policy objectives set up by the Ministry. This discussion affected several decisions about how to represent the information and what users would eventually be able to see on the website.

4.3 Open Data infrastructure

Once there was agreement on what information to include, the Ministry’s team identified the sources of information. The team found out that most of the information was not in an easily accessible format and was compartmentalised across the Ministry. Furthermore, processes were often manual. The Ministry also discovered that some of the data sources were of poor quality and that there were conflicts between different sources. Despite these challenges, the process of collecting data helped the Ministry to understand and better organise its own information.

The DATA team initially developed the back-end of the application used to import and process the data. Initial tests were run with datasets from the Ministry to ensure compatibility. This was a lengthy and challenging process for both parties and problems with data standardisation haunted the project until its first launch.

As the project evolved, DATA developed the first drafts of the website and started the validation process with the Ministry team. Middle managers working on the project were usually on board with the design choices. The process also involved managers and political appointees who were data providers, most of them wanted to ensure that the data was not misrepresented on the website. The discussions showed the asymmetry of technical knowledge between managers and technologists. To ensure that all the parties were on the same page, members of DATA would act as “translators”.

DATA and the Ministry made basic decisions about which data to display, and how. On the home screen, users would see data about waiting times, prices, user’s rights, the location of services and performance targets. Users would then be able to compare up to three providers from their administrative jurisdiction, allowing users to delve extensively into the data. Finally, all the standardised data were added to the Uruguayan national open data portal.

4.4. Results

When the data was published, more than 35,000 Uruguayans were able to access the information during the first year. In the second year, Atuservicio’s audience increased to 60,000 users. The information from the website was used in public debates by several media outlets (Sangakoya et al 2016). Politicians from the government and the opposition also used data from Atuservicio.uy to debate health policy in Parliament. The project survived a change of government and responsible Ministry team in 2016.

The following table provides a set of results on different intended beneficiaries of the project according to Sangakoya et al (2016) and Scrollini (2016).

5. Lessons from Open Data and Health Service Delivery

5.1 Context matters

Running an open data project in the health sector is a challenging endeavour. The basic assumption that releasing data in open formats, will quickly fix a complex (and often dysfunctional) sector, is, at best, naive. Nevertheless, the use of open data as a problem-solving approach can yield some benefits.

First, it is important to note that context matters when using ICTs for accountability (Davies and Fumega 2014). In this analysis, context means the willingness and capacity of the government to go digital and adopt open data. In countries where there is a strong will to engage with open government practices and a basic open data infrastructure, it is likely that more sophisticated open data projects could emerge. In settings where one of these elements is missing, an open data approach can still add value, but will be more limited. When there is no capacity to work with open data, an open data approach can highlight areas for development and improvement. When there is no willingness to engage on the government side, an open data approach could shed light on new practices that could enhance transparency and accountability efforts.

Secondly, actors (organisations, partners, funders) need to set clear objectives and agree on methods for the open data initiative. The examples in this paper range from control/confrontation between government and civil society to co-production. Since success will be context-dependent on the country and policy arena, it is important to define appropriate objectives for the initiative, for instance, some might require a significant amount of building government capacity and some might not. Appropriate objectives will ensure that the set targets and expected results are meaningful for all the actors involved.

One of the problems that can arise is managing the expectations of the various actors involved and how to frame the interventions in a manner that can be understood and embraced by all. There is a tendency to emphasise the benefits of a given technology or approach without properly assessing what can realistically be achieved with the available resources and capacities. Open data should not be regarded as a silver bullet in a field that is dominated by pilots and hence patchy- evidence. It can nonetheless, contribute in the following ways:

• It provides a new type of citizen participation tool that also enhances collaboration between the government and civil society.
• It enhances access to information by opening up information, which while not necessarily secret, was previously difficult to access because of manual systems. It can also help to create new types of information that can be availed to the public.
• Enhances media engagement with service delivery and policy and improves civic engagement through citizen feedback.
• It can contribute to changes in government process and policy depending on context and resources, by showing how to do things better. It can lead to an improvement in open data infrastructure and data management practices.
• It can help to change the government’s attitude to open data in positive ways.
• The data can enhance shared policy objectives between government and civil society and produce long-term policy change.

Regarding the management of expectations, the following outcomes are probably not immediately achievable through such open data initiatives:

• Changes in public health policy.
• Changes in health indicators such as mortality and morbidity rates.
• Prosecution of government officials or issuing of sanctions (unless specifically incorporated as part of a larger effort or specific campaign to “dig up dirt”).
• A sustainable open data infrastructure for new projects.

5.2 Co-production of open data

Co-production is a rare phenomenon in health service delivery and the open data field. In order to be successful, co-production practices need to consider four factors namely, context, shared policy objectives, partners’ capacity and positive feedback loops (Scrollini 2016).

The co-production process in the open data field offers an opportunity to pool resources towards the creation of public value. It usually involves private not-for-profit actors working with governments, often backed by public or philanthropic resources. It is important to note that co-production necessarily involves an active government on board with the initiative. A mantra of “give us the data” will not be enough. The state needs to engage if open data, and in particular co-production processes, are going to deliver significant value in this field. In the cases in this paper, the State played a key regulatory and data collection role. Better and more effective results are more likely where the state is actively engaged (Peixoto and Fox 2016).

The project examples above used open data as a means to solve a problem, and did not simply focus on the data itself. The possibility of using information to foster social change was a major motivation factor for the actors. In other words, open data is an entry point for enhanced collaboration, citizen engagement and better policies, not an end in itself. In all the cases, the process of putting together and publishing the data resulted in a better understanding of the policy arena. The shared concern about the manner in which the data was constructed and released also contributed to better alignment of policy objectives, not just about health, but also about access to information.

A common critique of the use of ICTs for development is the lack of offline access. Even in Uruguay, where almost 70% of the population has access to the Internet, this is a serious problem. Accordingly, it is important to consider these challenges from the pilot stage of a project. Sonora Ciudadana was able to leverage traditional offline methodologies with new tools. Moreover, it is important to consider whether these interventions can end up benefiting the already powerful, and marginalising the poor even further. Thus, it is important to consider the “ripple effect” a technological intervention can have and to ensure that it will promote inclusion.

6. Conclusion and recommendations

The initial research question of this paper was Can open data initiatives improve health service delivery, and if so, how? The answer emerging from these selected cases is a ‘qualified’ yes. To different degrees, the projects contributed to improving transparency and accountability in health service delivery. Open data activates new avenues to influence the delivery of health services. Nevertheless, context and sustainability matter. Most notably, the public sector (as a regulator or provider) must be a willing partner as they are the custodians of the data and the ones with the power to implement change. Private health providers’ cooperation is also important. Without the government’s willingness to make data more accessible, and to embrace and act on issues disclosed by the data, the scope for improvement is very limited. Moreover, lessons from the health sector might apply to other service delivery sectors (Bathley and Mcloughlin 2015).

Open data infrastructure (the data, tools, technologies and standards that allow the extraction, publication, opening and use of data) is a work in progress by all the concerned parties. The infrastructure is a “living one” where the movement of data leads to the improvement of data structure, quality, and use. It is therefore important to incorporate feedback loops in these projects in order to create virtuous circles of data collection, publication and use. A necessary pre-requisite is therefore that the capacity of traditional public service organisations to use data needs to be developed. In addition, in order to scale up these projects, we need a better understanding of how to sustain sector open data infrastructure. Another key question is of course, how to protect and safeguard the privacy of patients whilst opening certain types of health data. In all these projects patient information was off limits as the main goal was to improve accountability through data that was already public. Projects dealing with data for good or “data giving” will find data governance arrangements more challenging.

Health service delivery is complex, and it is often difficult to ensure evidence-based policy and action. Open data can contribute to addressing this gap. However, in order for open data to generate tangible changes, the project theories of change and causal mechanisms must be clear. In other words, when setting up these projects it is important to refine the expected outputs, outcomes and impact the project will deliver. Even though reality is far more complex than an input-output matrix, it would certainly help donors and practitioners to agree on what they seek to achieve, and to identify the key actors they would like to influence. Further, it might also be worthwhile to consider which implementation strategy to adopt, in terms of an “incremental” or “radical approach.” Lastly, co-production is not a panacea and requires a deeper understanding of the incentives and practices that would make such an endeavour successful.

Finally, the substantial question is whether these projects can change development outcomes. For example, can more information about the health care system deliver a positive change in child mortality rates? Can more information about prices of medicines deliver more efficient and equitable access to medicines? These questions provide the core motivation to engage with ICT and open health data. While it is probably true that open data can influence some macro-variables of a health system, we need more evidence on the link between open data and other factors and determinants of health. To explore these questions in depth requires time, resources and clearly identified variables. Pilot projects are common in this field and yet they only last for a short time. However, some pilots, such as the example presented in this paper, last long enough for a proper impact evaluation. Therefore, the examples here are prototypes with some level of success, but limited scalability.

In short, the encounter between open data and health service delivery shows real promise. Further exploration on how the linkage could work even better would be very welcome.