Do Clinical Trials Work?

[JohnnyD]

Fascinating opinion article. Ampligen comes to mind, and perhaps Retuximab....

http://www.nytimes.com/2013/07/14/opinion/sunday/do-clinical-trials-work.html?pagewanted=all&_r=2&


July 13, 2013

Do Clinical Trials Work?

By CLIFTON LEAF


EVERY spring, some 30,000 oncologists, medical researchers and marketers gather in an American city to showcase the latest advances in cancer treatment.

But at the annual meeting of the American Society of Clinical Oncology last month, much of the buzz surrounded a study that was anything but a breakthrough. To a packed and whisper-quiet room at the McCormick Place convention center in Chicago, Mark R. Gilbert, a professor of neuro-oncology at the University of Texas M. D. Anderson Cancer Center in Houston, presented the results of a clinical trial testing the drug Avastin in patients newly diagnosed with glioblastoma multiforme, an aggressive brain cancer. In two earlier, smaller studies of patients with recurrent brain cancers, tumors shrank and the disease seemed to stall for several months when patients were given the drug, an antibody that targets the blood supply of these fast-growing masses of cancer cells.

But to the surprise of many, Dr. Gilbert’s study found no difference in survival between those who were given Avastin and those who were given a placebo.

Disappointing though its outcome was, the study represented a victory for science over guesswork, of hard data over hunches. As far as clinical trials went, Dr. Gilbert’s study was the gold standard. The earlier studies had each been “single-arm,” in the lingo of clinical trials, meaning there had been no comparison group. In Dr. Gilbert’s study, more than 600 brain cancer patients were randomly assigned to two evenly balanced groups: an intervention arm (those who got Avastin along with a standard treatment) and a control arm (those who got the latter and a placebo). What’s more, the study was “double-blind” — neither the patients nor the doctors knew who was in which group until after the results had been assessed.

The centerpiece of the country’s drug-testing system — the randomized, controlled trial — had worked.

Except in one respect: doctors had no more clarity after the trial about how to treat brain cancer patients than they had before. Some patients did do better on the drug, and indeed, doctors and patients insist that some who take Avastin significantly beat the average. But the trial was unable to discover these “responders” along the way, much less examine what might have accounted for the difference. (Dr. Gilbert is working to figure that out now.)

Indeed, even after some 400 completed clinical trials in various cancers, it’s not clear why Avastin works (or doesn’t work) in any single patient. “Despite looking at hundreds of potential predictive biomarkers, we do not currently have a way to predict who is most likely to respond to Avastin and who is not,” says a spokesperson for Genentech, a division of the Swiss pharmaceutical giant Roche, which makes the drug.

Read the rest of the article here

 

[wdb]

I got the impression that the author doesn't really grasp what he is talking about and has got everything backwards. He is implying that if a treatment is shown to be effective in pilot studies but then a larger study is negative that the treatment probably does work; perhaps only for a subgroup and that there is some methodological problem with the larger study leading this effect not being picked up. The complete opposite is true, if a large blinded RCT is negative it means the treatment doesn't work, never did work, is no better than prescribing sugar pills. It is the pilot studies that are at fault and produced false positives due to placebo, bias and chance.

 

[JohnnyD]

I think you must have missed the point that Avasten definitely works for some patients. There is a responder group (which is why ampligen comes to mind)... And after 400 trials, on various cancers, they have no idea of why it works, or who the responders are.

"Except in one respect: doctors had no more clarity after the trial about how to treat brain cancer patients than they had before. Some patients did do better on the drug, and indeed, doctors and patients insist that some who take Avastin significantly beat the average. But the trial was unable to discover these “responders” along the way, much less examine what might have accounted for the difference. (Dr. Gilbert is working to figure that out now.)

Indeed, even after some 400 completed clinical trials in various cancers, it’s not clear why Avastin works (or doesn’t work) in any single patient. “Despite looking at hundreds of potential predictive biomarkers, we do not currently have a way to predict who is most likely to respond to Avastin and who is not,” says a spokesperson for Genentech, a division of the Swiss pharmaceutical giant Roche, which makes the drug."

 

[wdb]

Yes and placebo or no treatment at all definitely works for some patients, that is why you need proper controlled trials. You can't claim there are responders in a negative trial the fact the outcome was negative would mean that there would have been as many apparent responders in the non-treatment group. It would be like me claiming to be able do a rain dance and saying all the days it did rain proves that it is sometimes effective.

 

[JohnnyD]

Again, I think you misunderstand the gist of the article. This isn't about the placebo effect. The gold standard of clinical trials - the randomized, doubled blind, placebo controlled trial - says Avastin does not work. Doctors and patients insist that Avastin does work - and to a significant degree. The drug has been around for 16 years, studied in 400 trials, 6 billion in annual sales. No one - not doctors, not patients, not the FDA --- is calling for it to be removed from the market. Why? Because it works for a sub group of patients.

 

[taniaaust1]

Of the 26 randomized, controlled studies that were followed up by larger trials (examining the same therapy in a bigger pool of patients), the initial finding was wholly contradicted in three cases (12 percent). And in another 6 cases (23 percent), the later trials found the benefit to be less than half of what was first reported.

It wasn’t the therapy that changed in each case, but rather the sample size.

I think there was one big thing which is being missed in all this which could help explain why so many times there negative results for things which were previously positive.

Could a big part of this issue also be that often even when drugs show promising results that the results are often published while the negative results of smaller studies usually arent even published. This leads to biased towards favourable small study results being seen in publications, one could end up seeing a lot of these due to the biased and hence think a drug is great but then have a big drug trial which is probably more likely to get accepted for publication go disproving it. Drug companies which sponser so much research, also pick and choose what may be put forward for publishing (they arent going to push to publish much negative research on the drugs they make money from. I see the big drug companies as no different to car salesmen).

I personally think the whole research and publication system needs to be changed.. with "failure" drug studies, small or not given equal importance to the positive ones, maybe there even should be a binding journal agreement to publish study results, before the results are even found no matter what results. The way things currently go.. with the publication biased, I can hardly trust anymore what I see in research journals. The whole thing is often very misleading (and we arent even getting into the bad study thing).

 

[Little Bluestem]

The negative outcome could have been because the number of responders was so small that it was not statistically significant. The chance of winning a very large lottery is not statistically different from zero, yet someone does win.

I was running an errand this morning and in and out of the car during the Diane Rehm show. The subject this morning was prescription drugs and doctors. The guest, Dr. Sid Wolfe of Public Citizen, said that some drugs are recommended to not be approved by mid-level research reviewers at the FDA, but are approved by upper level decision makers to appease the pharmaceutical companies.

 

[Waverunner]

"For subtypes of disease that are already known, it may be feasible to design small clinical trials and enroll only those who have the appropriate genetic or molecular signature."

I've been demanding this over and over again. All these huge clinical trials can be and often are a huge waste of money and resources because we don't take the heterogeneity of humans into account. Molecular medicine and the sequencing of genes, tumors, microbiome etc. are the solution to all these problems but we need more time and much faster implementation.

What is the status quo? The status quo are completely flawed randomized clinical trials which are like playing lottery. You always have to hope, that you react the same way to the approved drug as did the people who got helped in the study. But what happens if the drug doesn't help you? Well, you have to switch to another drug. However, the big question is, are other drugs available, what are their risk profiles and will they work for you? As long as medicine doesn't take genetics into account, this is a pure gamble and the only chance to increase your chances of winning is to increase the number of bought lottery tickets (=approved drugs). The current system however does everything to keep the numbers of new drugs very small so the number of winning tickets is kept small as well. By doing this, a very rational approach to deal with drugs (-> buy more tickets) is destroyed and this perfectly explains why many ill patients are not helped by currently approved drugs.

 

[wdb]

Again, I think you misunderstand the gist of the article. This isn't about the placebo effect. The gold standard of clinical trials - the randomized, doubled blind, placebo controlled trial - says Avastin does not work. Doctors and patients insist that Avastin does work - and to a significant degree. The drug has been around for 16 years, studied in 400 trials, 6 billion in annual sales. No one - not doctors, not patients, not the FDA --- is calling for it to be removed from the market. Why? Because it works for a sub group of patients.

Again you misunderstand the numerous logical fallacies in the article. You cannot make the statement something is known to work when the strongest evidence to date suggests that it does not work. If the weak evidence is positive and the strong evidence is negative then the only reasonable rational position to take is the negative one, to take the opposite view that the weak evidence should take precedence and therefore RCTs don't work is BS.

Now the trial was only on brain cancer so that doesn't mean that it doesn't work for other cancers so no need for call to take it off market. Incidentally there was a similar situation regarding breast cancer and it was taken off the market for that condition.

At one point bevacizumab was approved for breast cancer by the FDA, but the approval was revoked on 18 November 2011.[3][4] The approval for breast cancer was revoked because, although there was evidence that it slowed progression of metastatic breast cancer, there was no evidence that it extended life or improved quality of life, and it caused adverse effects including severe high blood pressure and hemorrhaging. In 2008, the FDA gave bevacizumab provisional approval for metastatic breast cancer, subject to further studies. The FDA's advisory panel had recommended against approval.[5] In July 2010, after new studies failed to show a significant benefit, the FDA's advisory panel recommended against the indication for advanced breast cancer. Genentech requested a hearing, which was granted in June 2011. The FDA ruled to withdraw the breast cancer indication in November 2011. FDA approval is required for Genentech to market a drug for that indication. Doctors may sometimes prescribe it for that indication, although insurance companies are less likely to pay for it.[3] The drug remains approved for breast cancer use in other countries including Australia.[6]

 

[Firestormm]

wdb tell me if I am wrong and I do need to read the full intriguing article, but:

In two earlier, smaller studies of patients with recurrent brain cancers, tumors shrank and the disease seemed to stall for several months when patients were given the drug, an antibody that targets the blood supply of these fast-growing masses of cancer cells.

But to the surprise of many, Dr. Gilbert’s study found no difference in survival between those who were given Avastin and those who were given a placebo.

Forgive me, but isn't the article at least saying, that the drug has been shown to shrink tumors - so that is it's effect - but in the long-run the drug does not improve the chances of survival - because brain cancer is terminal.

So for the patient I would imagine, being on the drug might improve his quality of life, decrease some symptoms perhaps (I haven't looked into it); but that the drug is no cure and has never been sold as such.

The title of the article is misleading. All trials work to some extent. Because they all have aims in mind. Comparing them is difficult.

Unless this RCT-double-blind trial was also looking for example at what the drug did in terms of shrinking tumors and 'seeming to stall the disease' for a time.

Am reading the full article now:

In the RCT:

"Some patients did do better on the drug, and indeed, doctors and patients insist that some who take Avastin significantly beat the average.

But the trial was unable to discover these “responders” along the way, much less examine what might have accounted for the difference. (Dr. Gilbert is working to figure that out now.)"

I would suggest that the Trial was not designed to identify these responders - hence why Dr Gilbert is now analyzing the results.

You can't - as you know - expect all the answers from a Trial unless the Trial was designed to specifically answer them. This Trial has led to more questions - which is what happens.

Indeed, even after some 400 completed clinical trials in various cancers, it’s not clear why Avastin works (or doesn’t work) in any single patient. “Despite looking at hundreds of potential predictive biomarkers, we do not currently have a way to predict who is most likely to respond to Avastin and who is not,” says a spokesperson for Genentech, a division of the Swiss pharmaceutical giant Roche, which makes the drug.

This is perhaps being seen as 'remarkable' but it is normal, right? I mean who the hell knows if taking an aspirin is going to fix a headache? What are the annual sales of aspirin - or what were they prior to generic manufacture?

Any drug only increases the chances. We don't know why a drug might help one person with X disease and not another - same goes for non-drug treatments: although we know even less about them.

Ah, OK and reading the rest of the piece. Maybe the title isn't as misleading as I thought. I think it is in relation to this Trial; but not in relation to the remainder.

Clinical Trials are not, cannot be the be all and end all. We simply don't know enough about individuals. They are the best that we can achieve with what we have - and some are better designed than others: but all are limited by many factors: motivation and cost among them.

Provocative piece though. Good stuff. Will come back later

 

[wdb]

In two earlier, smaller studies of patients with recurrent brain cancers, tumors shrank and the disease seemed to stall for several months when patients were given the drug, an antibody that targets the blood supply of these fast-growing masses of cancer cells.

This is the issue that I have, taking the position that we already know something works before it has been properly tested. These were small uncontrolled trials, they gave reason to suspect that the treatment might be effective and that a larger study was justified but they did not demonstrate anything conclusive in themselves.

In the RCT:

"Some patients did do better on the drug, and indeed, doctors and patients insist that some who take Avastin significantly beat the average.

But the trial was unable to discover these “responders” along the way, much less examine what might have accounted for the difference. (Dr. Gilbert is working to figure that out now.)"

How do they know they were responders to the treatment, this is an assertion made on anecdotal evidence. In any trial you'd expect some participants to do significantly better than average and some to do worse than average by chance alone. In very large trials you might expect a small number to massively beat the average and there is no way of determining whether they are responders or just outliers on the bell curve. If significantly more in the treatment group had responded than in the control group then the outcome would not have been negative.

So either there were just as many responders in the control group, or tumour shrinkage has no bearing on mortality. Either way I don't see any of it as evidence controlled trials don't work.

I guess one other possibly would for every person helped by the treatment there was an equivalent amount or harm done to other participant in that group cancelling it out.

I don'r know the validity of this but it was posted on another forum
Barbara July 15, 2013

Our Oncology unit participated in the Avastin in glioblastoma study (AvaGlio). We enrolled 12 patient of which two did very well (still alive after 2+ years). Roche unblinded all patients in February 2013 and both patients were on placebo.

 

[JohnnyD]

I agree, it is a provocative article that raises many more question than it answers. Here is another New York Times article written in June on the same trial (actually two trials).

http://www.nytimes.com/2013/06/03/b...ffectiveness-against-a-rare-brain-cancer.html

They pulled Avastin for breast cancer, from the article despite an outcry from patients and doctors.

In 2011, after subsequent studies showed that Avastin did not help women with breast cancer live longer, the agency revoked the drug’s approval for that disease, provoking an outcry from some patients and doctors who said that the drug had helped them.

Revocation seems less likely for brain cancer treatment because fewer options are available than for breast cancer.

“The vast majority of patients with glioblastoma in the United States get Avastin at some point in their illness,” said Dr. Mark R. Gilbert of the University of Texas M.D. Anderson Cancer Center, the lead investigator in one of the studies, which were presented at the annual meeting of the American Society of Clinical Oncology.

Dr. Howard A. Fine of the New York University Langone Medical Center, who was chosen to comment on the studies at the meeting, said removal of the approval would be “a travesty.”