Ben Goldacre: What the Tamiflu saga tells us about drug trials and big pharma

We now know the government's Tamiflu stockpile wouldn't have done us much good in the event of a flu epidemic. But the secrecy surrounding clinical trials means there's a lot we don't know about other medicines we take

Today we found out that Tamiflu doesn't work so well after all. Roche, the drug company behind it, withheld vital information on its clinical trials for half a decade, but the Cochrane Collaboration, a global not-for-profit organisation of 14,000 academics, finally obtained all the information. Putting the evidence together, it has found that Tamiflu has little or no impact on complications of flu infection, such as pneumonia.

That is a scandal because the UK government spent £0.5bn stockpiling this drug in the hope that it would help prevent serious side-effects from flu infection. But the bigger scandal is that Roche broke no law by withholding vital information on how well its drug works. In fact, the methods and results of clinical trials on the drugs we use today are stillroutinely and legally being withheld from doctors, researchers and patients. It is simple bad luck for Roche that Tamiflu became, arbitrarily, the poster child for the missing-data story.

And it is a great poster child. The battle over Tamiflu perfectly illustrates the need for full transparency around clinical trials, the importance of access to obscure documentation, and the failure of the regulatory system. Crucially, it is also an illustration of how science, at its best, is built on transparency and openness to criticism, because the saga of the Cochrane Tamiflu review began with a simple online comment.

In 2009, there was widespread concern about a new flu pandemic, and billions were being spent stockpiling Tamiflu around the world. Because of this, the UK and Australian governments specifically asked the Cochrane Collaboration to update its earlier reviews on the drug. Cochrane reviews are the gold-standard in medicine: they summarise all the data on a given treatment, and they are in a constant review cycle, because evidence changes over time as new trials are published. This should have been a pretty everyday piece of work: the previous review, in 2008, had found some evidence that Tamiflu does, indeed, reduce the rate of complications such as pneumonia. But then a Japanese paediatrician called Keiji Hayashi left a comment that would trigger a revolution in our understanding of how evidence-based medicine should work. This wasn't in a publication, or even a letter: it was a simple online comment, posted informally underneath the Tamiflu review on the Cochrane website, almost like a blog comment.

Cochrane had summarised the data from all the trials, explained Hayashi, but its positive conclusion was driven by data from just one of the papers it cited: an industry-funded summary of 10 previous trials, led by an author called Kaiser. From these 10 trials, only two had ever been published in the scientific literature. For the remaining eight, the only available information on the methods used came from the brief summary in this secondary source, created by industry. That's not reliable enough.

This is science at its best. The Cochrane review is readily accessible online; it explains transparently the methods by which it looked for trials, and then analysed them, so any informed reader can pull the review apart, and understand where the conclusions came from. Cochrane provides an easy way for readers to raise criticisms. And, crucially, these criticisms did not fall on deaf ears. Dr Tom Jefferson is the head of the Cochrane respiratory group, and the lead author on the 2008 review. He realised immediately that he had made a mistake in blindly trusting the Kaiser data. He said so, without defensiveness, and then set about getting the information needed.

First, the Cochrane researchers wrote to the authors of the Kaiser paper. By reply, they were told that this team no longer had the files: they should contact Roche. Here the problems began. Roche said it would hand over some information, but the Cochrane reviewers would need to sign a confidentiality agreement. This was tricky: Cochrane reviews are built around showing their working, but Roche's proposed contract would require them to keep the information behind their reasoning secret from readers. More than this, the contract said they were not allowed to discuss the terms of their secrecy agreement, or publicly acknowledge that it even existed. Roche was demanding a secret contract, with secret terms, requiring secrecy about the methods and results of trials, in a discussion about the safety and efficacy of a drug that has been taken by hundreds of thousands of people around the world, and on which governments had spent billions. Roche's demand, worryingly, is not unusual. At this point, many in medicine would either acquiesce, or give up. Jefferson asked Roche for clarification about why the contract was necessary. He never received a reply.

Then, in October 2009, the company changed tack. It would like to hand over the data, it explained, but another academic review on Tamiflu was being conducted elsewhere. Roche had given this other group the study reports, so Cochrane couldn't have them. This was a non-sequitur: there is no reason why many groups should not all work on the same question. In fact, since replication is the cornerstone of good science, this would be actively desirable.

Then, one week later, unannounced, Roche sent seven documents, each around a dozen pages long. These contained excerpts of internal company documents on each of the clinical trials in the Kaiser meta-analysis. It was a start, but nothing like the information Cochrane needed to assess the benefits, or the rate of adverse events, or fully to understand the design of the trials.

Read more here: http://www.theguardian.com/

Graham