Clinical trial opacity undermines global health

Clinical trials are a key driver of medical innovation and progress, but the existing evidence base on drugs and vaccines is incomplete and systematically biased due to the opacity of clinical trials. This may have a substantial negative effect on global health, and directly threatens progress on seven of the targets set out by Social Development Goal 3.

US$ 85 billion out of the US$ 170 billion spent on health research globally every year goes to waste because research results are not reported.4d1b8f506ab8 Incomplete, inaccurate, misleading and slow reporting of results causes funders and scientists to repeatedly explore the same dead ends, which drives up the time and cost required to develop new vaccines, treatments and cures.

In addition, the opaqueness of clinical trials creates a strong information asymmetry between those generating trial data, who often have strong vested financial or career interests in presenting positive results, and the public and private purchasers of drugs, devices and treatments. This lack of transparency creates an environment in which selective reporting, evidence distortion, and outright fraud are difficult to detect and deter, opening the door to corrupt practices.89fcefbbb23f

Because the evidence base underlying medicine is fragmentary and distorted, global health bodies, donors and health agencies, including those in the Global South, cannot reliably determine the comparative cost-effectiveness of different treatment options. This leads to suboptimal allocation of scarce public health funds, undermines pandemic preparedness, and directly harms individual patients.

This policy brief provides an overview of the issue, based on a more extensive study published by Transparency International, Cochrane, CRIT and TranspariMED in 2017.011204e6acb1 In addition, it outlines simple and cost-effective steps that donors can take to curb research waste, accelerate medical progress, and improve public health in the Global South.

Failures to report trial results and evidence distortion are widespread

There is a vast literature documenting the large scope and scale of evidence distortion in medical research.014a4f6bb7ef Around half of clinical trials never report their results.daefbe6666ca Because trials with positive outcomes are far more likely to report their results, the current evidence base systematically overestimates the efficacy of drugs and underestimates their harms.5ea54dccac906bba9a6378c0

In addition, trials with negative outcomes are often misreported as having positive outcomes. Such evidence distortion is widespread and takes many forms, including spin, statistical manipulation, and selective reporting of partial results. For example, in 2015–2016, a team of researchers found that only nine trials out of a cohort of 67 published in the world’s top five medical journals had been accurately reported.a0cb21f5b534

The chart below, based on a widely cited analysis of a cohort of 74 clinical trials of antidepressant drugs,fe7a7ffacc88 illustrates how non-publication and evidence distortion can combine to create a severely misleading picture of the effectiveness of drugs. All but one of the 38 positive clinical trials were published. Of the 36 negative trials, 22 remained unpublished, and a further 11 were misleadingly published as having had positive outcomes. As a result, the scientific literature suggested that only three clinical trials of these drugs had negative outcomes.

Importantly, these problems are not limited to clinical trials funded or run by pharmaceutical companies. In fact, numerous studies show that trials funded or run by public health bodies, universities and the third sector are even less likely to report their results.148b2dddcebf1b66f88e61e4 Evidence distortion too is widespread beyond industry due to a combination of perverse incentives, weak institutional oversight and a lack of effective deterrents.

Source: Chart based on the findings of Turner et al. (2008).

Weak legal and regulatory frameworks exacerbate opacity

The medical research sector’s inability to self-regulate is exacerbated by a lack of state action. The United Nations,6d5c680c5bc5 the World Health Organization (WHO)e809be84b5c5 and other stakeholdersc040c45a4990 have repeatedly called for greater clinical trial transparency. However, governments worldwide have so far failed to ensure that the results of all trials are promptly and accurately reported. In the United States, over 40% of applicable trials are in violation of a 2007 transparency law, but $670 million in fines remain uncollected.aa5aeea7cd6c In the Europe, 49% of applicable trials violate European Union disclosure regulations,add7a4fe0787 but no sanctions have ever been imposed. Even in wealthy nations, laws and regulations are typically limited in scope, and compliance is insufficiently monitored and enforced. At the same time, Clinical Study Reports, documents held by regulatory agencies that contain the most detailed information on the safety and efficacy of drugs on the market, are often not shared with other public bodies and independent scientists.eb38c9ea8c36

Global consequences of clinical trial opacity

The current lack of access to complete, unbiased and undistorted evidence of the benefits and harms of drugs, medical devices and treatments harms patients, prevents public health agencies from making informed decisions, and wastes public health funds. The negative consequences of weak trial transparency are well-documented in developed countries. For example, an estimated 100,000 patients died in the U.S. alone because trial results that could have alerted doctors to the risks of the heart drug Lorcainide remained unpublished. In the UK, the National Health Service spent 0.5% of its entire 2009 budget on the drug Tamiflu based on limited evidence; after previously hidden trial results came to light, many scientists concluded that the drug was ineffective. In total, 96 countries had stockpiled enough of the drug to treat 350 million people.c3e653316249 A 2015 study of 300 clinical trials showed that only 11% of publications in scientific journals provided a complete and consistent account of all the serious adverse events experienced by patients.455ab8f3cce5

Impacts on public health in the Global South

The problem of unreliable evidence affects medical research and clinical practice across all disease areas and regions, including on health issues of particular concern to the Global South. For example, at the outset of the 2014 Ebola outbreak in West Africa, scientists discovered that potentially life-saving information from past clinical trials was missing. This led to major inefficiencies in testing for Ebola therapeutics. A 2017 review of Ebola trials on the world’s largest trial registry Clinicaltrials.gov found that not a single one had posted its summary results there, despite 30 trials having already exceeded the 12 month disclosure timeframe set by the WHO.b6c9ea3c5d35 In some cases, the results of these trials had not been published in the academic literature either and therefore remained inaccessible to the scientific community.

Threat to targets set out by Social Development Goal 3

The level of transparency of clinical trials will directly affect the extent to which seven out of the 13 health targets set out under Social Development Goal 3 will be achieved. Most obviously, the non-reporting of trials undermines Target 3 (ending the epidemics of AIDS, tuberculosis, malaria by 2030) and Target 11 (supporting the research and development of vaccines and medicines). In addition, as long as the evidence base of medicine remains severely distorted, Target 9 (access to safe and effective medicines and vaccines) will be difficult to achieve because public health agencies are currently unable to reliably evaluate the safety and effectiveness of drugs. This, combined with the overall slower pace of scientific progress, will also hinder the achievement of Targets 1, 2, 4, and 5.

Practical steps to end evidence distortion in medical research

A coalition of health integrity groups led by Transparency International recommended in 2017 that:

“As a first step, political decision-makers should require all public research funding bodies within their jurisdiction to adopt and expand on the provisions of the recent WHO-brokered ‘Joint Statement’ by research funders, and ensure that they are fully implemented. In future, to help ensure that public funding for medical research actually benefits the public, government funders should only give taxpayers’ money to institutions and individuals that verifiably comply with best practices in clinical research. Taking this simple first step would deliver significant transparency gains at minimal cost.”0784cbc6acd6

Signing up to the 2017 Joint Statement commits clinical trial stakeholders to ensuring that all clinical trials they fund, co-fund, sponsor or support adhere to global best practices in clinical trial transparency.9d351c091f45 These include prospective trial registration, the posting of summary results on registries within 12 months of trial completion, monitoring of grantee compliance, and the publication of monitoring reports. As the Joint Statement itself emphasises, “[t]he resource allocation, public health and scientific benefits – together with the need to meet ethical imperatives – far outweigh the costs” of the proposed monitoring mechanisms.

While joining the initiative is voluntary, its commitments are specific and time-bound, and compliance is externally verifiable, prompting a variety of stakeholders to strongly welcome it.f4414acec70e595a64c66aa6 However, a year after the Joint Statement was published, a policy audit found that signatories’ implementation of its provisions had been uneven.0e7b0ae1af4f

Donor options for promoting clinical trial transparency among funding partners

In May 2017, the UK’s Department for International Development (DFID) became the first bilateral donor agency to sign up to the Joint Statement. Several organisations that receive international development funding, including Médecins Sans Frontières, the TB Alliance, and Medicines for Malaria Venture have also signed up.

Other bilateral donors could promote clinical trial transparency by following the lead of DFID, sign up to the Joint Statement, and fully implement its provisions. In addition, donors could require all grantee institutions involved in funding or conducting clinical trials, including universities at home and abroad, to sign up to the Joint Statement and publish the stipulated annual monitoring reports as a precondition for receiving future grants. In addition to strengthening the global evidence base on drugs and vaccines and accelerating medical progress, this would mitigate the fiduciary risk of taxpayers’ money contributing to the US$ 85 billion in research waste generated annually through the non-reporting of medical research results.

Furthermore, donor agencies could encourage public research funders within their home countries to sign up to the Joint Statement. The largest public research funding institutions in Australia, Canada, China, Germany, Italy, Japan, Spain and the United States have yet to sign up.

Rescuing clinical trial results from the global pile of research waste

Clinical trials typically cost several million dollars to set up and run. In contrast, posting their summary results onto registries costs only around $2,000;bd3fd8852cd8 the cost of publishing their results in academic journals is similarly low. Donors could fund systematic efforts to retrospectively publish the results of clinical trials concluded in the past that have failed to report their results. The medical knowledge generated by these trials will be lost forever unless their results are rescued soon, before the researchers involved retire and the underlying datasets disappear.

For example, donors could commission a review of all completed clinical trials of Neglected Tropical Diseases that have not posted summary results onto trial registries to identify those that have also failed to publish their results in the academic literature. Donors could then fund research institutions or medical writing companies to rescue these ‘lost’ trials by retrospectively posting their summary results onto trial registries and publishing their outcomes in medical journals. While the data generated by some of these trials will already have been irretrievably lost, experience shows that rescuing ‘lost’ trials is practically feasible.9e77b60c3390

As the cost of making trial outcomes publicly available is minimal compared to the original cost of conducting the trials themselves, this would be an extremely cost-effective global health intervention.

Technical assistance for strengthening clinical trial governance in the Global South

A growing number of clinical trials are being conducted in countries of the Global South whose legal and regulatory frameworks pertaining to trial transparency may be even weaker than those in donor nations. Donors could provide technical assistance to Ministries of Health in partner countries to enable them to strengthen those frameworks and effectively monitor and enforce compliance. Transparency International has recently set out transparency benchmarks to guide the development of sound clinical trial governance systems.cb2f50e3623d

In addition, donors could assist public research agencies in the Global South in implementing the provisions of the Joint Statement, as the Indian Council of Medical Research has already started to do. Donors could provide technical assistance to other agencies in major developing countries in the setup of appropriate monitoring systems and facilitate peer-to-peer knowledge exchanges.

Supporting research and advocacy on clinical trial transparency

Over 700 professional medical associations, patient groups and other civil society actors have formally expressed their support for clinical trial transparency by signing up to the principles of the global AllTrials campaign.811b586a6b98 Donors could help CSOs and patient groups to turn this passive support for more transparency into effective advocacy by supporting the replication of successful approaches.493ee7113a1a

The experiences of recent transparency initiatives strongly suggest that making institutional performance visible in and of itself can incentivise significant improvements at that level. For example, a widely noted1aa4b56e0c5d investigation by the medical news outlet STAT News in 2015 analysed Clinicaltrials.gov data and revealed that commercial companies, universities and even government agencies were “routinely violating” a key U.S. trial transparency law.ab26f6c960de In 2018, a follow-up investigation found that many of the institutions highlighted as the worst performers three years earlier had taken the biggest steps to improve their performance.9493e495a8ee Recent transparency initiatives by EBM Data Lab and TranspariMED have similarly yielded tangible results.65cd08a9bb36

While past efforts have focused on the institutional level only, there is also considerable scope for research and advocacy focused on specific disease areas, especially Neglected Tropical Diseases,312fc49490ab and for efforts that compare transparency performance at the national level. Research indicates that the impact of such initiatives could be strengthened by coupling them with direct outreach to the parties responsible for reporting the results of specific trials.caf3037707ad

In addition, donors could support think tanks in conducting policy research and advocacy around relevant legal and regulatory frameworks,6e5bb66c6818 and in integrating clinical trial transparency concerns into mainstream global and national health policy debates.