(PACE Trial) Response to FoI request: Numbers within normal ranges for CFQ and SF36 PF at baseline

[Tom Kindlon]

Dear Mr. Smallcombe,

Thank you very much for your reply. It appears that I did not ask my question well enough however. Would it be possible to ask how many participants were in the normal range in either the SF-36 or Chalder Fatigue Scale at the beginning of the trial, ie one or the other?

Thank you very much for your time.

Regards,

<name>

-------------------------

(Queen Mary, University of London) FOI Request: 2013/F42

On Mon, Mar 11, 2013, <foi-enquiries@qmul.ac.uk> wrote:

Dear <name>

Thank you for your email of 12th February.

Please see the attached for the information you requested.

If you are dissatisfied with this response, you may ask the College to conduct a review of this decision. To do this, please contact the College in writing (including by fax, letter or email), describe the original request, explain your grounds for dissatisfaction, and include an address for correspondence. You have 40 working days from receipt of this communication to submit a review request. When the review process has been completed, if you are still dissatisfied, you may ask the Information Commissioner to intervene. Please see www.ico.gov.uk for details.

Yours sincerely

Paul Smallcombe
Records & Information Compliance Manager

 

[Simon]

Great work, <name>!

Because no one else has yet, let me state the obvious: 3 patients were 'normal' at the start of the trial and met the recovery criteria for fatigue and function. A further 82 patients met the recovery criteria for fatigue or function at baseline. In all, 85 patients, 13%, met either fatigue or function recovery criteria, or both, before the trial began. Nice.

 

[Bob]

So, in terms of physical function, 20 (12%) of the CBT group, and 15 (9%) of the GET group, already had a level of physical function that was considered 'recovered', on entry into the trial.

And for Chalder fatigue, 3 (2%) of the GET group, already had a level of fatigue that was considered 'recovered', on entry into the trial.

 

[Mark]

Is it possible to determine whether those that were 'recovered' before they started 'treatment' were the same people who were included in the figures for those 'recovered' after the 'treatment'?

I guess even if they weren't, that would mean they were people who were 'recovered' before the treatment, but not after the treatment. So from that point of view, those figures should be subtracted from the 'recovery' statistics whichever way you slice it.

 

[Esther12]

Is it possible to determine whether those that were 'recovered' before they started 'treatment' were the same people who were included in the figures for those 'recovered' after the 'treatment'?

I guess even if they weren't, that would mean they were people who were 'recovered' before the treatment, but not after the treatment. So from that point of view, those figures should be subtracted from the 'recovery' statistics whichever way you slice it.

The new criteria for 'recovery' has additional components, such as requiring that patients not fulfil the criteria for CFS, so patients could not have fulfilled that criteria at the start of the trial. I think that added this as a way of artificially avoiding the absurdity of patients being 'sick' and 'cured' at the same time, but without needing to tighten the requirements for levels of fatigue or disability.

 

[Peter Kemp]

This is great, Tom. I would be curious to know if this data was generated in response to the FOI, or, if it was already available, when was it produced? You can probably guess where my dirty little mind is going! Did they generate this as part of the process of deciding where to put 'normal range'? The info above can only have been derived from the raw data to produce Likert scores correlating to 'normal range'. I imagine it was a simple enough database inquiry to produce, but still, if they did it specially for the FOI that was very nice of them.

 

[Valentijn]

Is it possible to determine whether those that were 'recovered' before they started 'treatment' were the same people who were included in the figures for those 'recovered' after the 'treatment'?

I guess even if they weren't, that would mean they were people who were 'recovered' before the treatment, but not after the treatment. So from that point of view, those figures should be subtracted from the 'recovery' statistics whichever way you slice it.

I think it just means things are a bigger mess and it's even more impossible to make any sense of the results You could subtract the ones that were normal on both CF and PF from the results (table 1 at journals.cambridge.org/psm/White) easily enough (-1 each to APT, SMC, and GET).

But it doesn't look like they break down CF and PF in the "recovery" table. You could subtract the combined number of baseline CF-normal plus PF-normal from each group. But then you get negative 4 recovered patients in the SMC recovery group based on the CF and PF combined criteria (22 PF and CF "recovered" at the end, 26 either PF or CF normal at the start). If carrying through that subtraction from the final total of "recovered" patients and assuming the worst (that the recovered patients had normal PF and/or CF at the start), you get a theoretical recovery rate that looks like:
APT: -9%
CBT: 2%
GET: 3%
SMC: -13%

Which is just another indication that this trial and the data presented are a bunch of meaningless babble.

 

[Bob]

I'd forgotten all about this thread and these figures.

So, in terms of physical function, 20 (12%) of the CBT group, and 15 (9%) of the GET group, already had a level of physical function that was considered 'recovered', on entry into the trial.

Thinking out loud...

So does this mean that it was only necessary for physical function to improve in 10% of the participants in the CBT group in order for a 22% 'recovery' rate to be recorded?

And taking the SMC group into account, in which 7% were reported to have 'recovered', does it mean that only an extra 3% of participants needed to improve in the CBT group, compared with SMC, for a 22% recovery rate to be recorded for the CBT group?

(I'm not sure if either of these calculations are appropriate, considering that 16% of the SMC group had physical function in the normal range at baseline.)

In any case, it isn't helpful that 12% of the CBT group were already in the recovery range, for physical function, at the start of the trial.

 

[Enid]

Big sigh Bob - guess I'm plus 3% one day and minus the next - what crashing idiots these people who try to fit into their stats are.

 

[alex3619]

I missed this as I was in hospital. Question: does this justify a charge of scientific fraud?

 

[biophile]

Question: does this justify a charge of scientific fraud?

Not necessarily.

However, on a wider note, the British Medical Association have just voted that non-publication of trial results is now to be considered research misconduct, and that doctors involved in withholding results should be referred to the GMC.

http://www.alltrials.net/2013/bma-v...ion-of-trials-results-is-research-misconduct/

The motion agreed in full:

“That this meeting states that:

i) Selective non-publication of unflattering trial data is research misconduct;

ii) Registered medical practitioners who give grounds to believe they have been involved in such conduct should have their fitness to practice assessed by the GMC.”

I am not sure if that only means the failure to publish any data at all because the trial was unsuccessful, or if it also includes the failure to publish any (significant) "unflattering trial data".

 

[Simon]

Not necessarily.

However, on a wider note, the British Medical Association have just voted that non-publication of trial results is now to be considered research misconduct, and that doctors involved in withholding results should be referred to the GMC.

http://www.alltrials.net/2013/bma-v...ion-of-trials-results-is-research-misconduct/

I am not sure if that only means the failure to publish any data at all because the trial was unsuccessful, or if it also includes the failure to publish any (significant) "unflattering trial data".

Selective non-publication of unflattering trial data is research misconduct

The key word is selective, so I think publishing postive data while withholding unfavourable data would be covered by the BMA motion.

 

[alex3619]

Hmmm, if I believe this data, and without checking definitions yet again, then around twelve percent could never have had CFS under Fukuda or other definitions. If they were within normal function range they did not have a loss of fifty percent of functional capacity necessary for most definitions of CFS. Oxford based ineptitude strikes again. Now where is handy definition of the Oxford definition ...

PS Still looking for a good version of Oxford, but I think it requires disabling fatigue. Anyone who is normal does NOT have even Oxford CFS. So twelve percent of the patients should not have been there at all. Or their definition of normal is wrong.

PPS http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1293107/pdf/jrsocmed00127-0072.pdf

Clause (c) of the definition:

"The fatigue is severe, disabling, and affects physical and mental functioning."

 

[Bob]

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1293107/pdf/jrsocmed00127-0072.pdf

Clause (c) of the definition:

"The fatigue is severe, disabling, and affects physical and mental functioning."

Interesting. If, at baseline, 12% of the CBT group had a level of physical functioning that was within the 'recovery' range, then can it be considered that the fatigue affected their physical functioning (if their physical functioning was considered 'recovered')?

 

[biophile]

The key word is selective, so I think publishing positive data while withholding unfavourable data would be covered by the BMA motion.

AllTrials is about both scenarios, but especially no data at all. This however leaves some ambiguity in what qualifies as "withholding unfavourable data". No doubt PACE would claim not to have done so, and regulators might still believe them. It is also more difficult to assert that unfavourable data is being withheld too long if we do not know enough about the unpublished data to judge whether it is unfavourable or not, seems more of a hindsight judgement, such as when the Nijmegen CBT school withheld unfavourable objective data for several years.

It has been previously argued that the PACE employment/welfare data was withheld and turned out to be unfavourable, but also counter-argued that everything cannot be squeezed into a single paper. However, they sure took their sweet time publishing it about 18 months after the initial Lancet fanfare. PACE have recently fended off a FOI with claiming that the requested "deterioration" rates will be published soon, but I doubt that they would have published it soon without prompting.

 

[Sean]

So twelve percent of the patients should not have been there at all.

If I recall correctly, 12% is getting close to the net number of patients who benefit in any way, even under the most generous definition of 'benefit'.

 

[SOC]

Not necessarily.

However, on a wider note, the British Medical Association have just voted that non-publication of trial results is now to be considered research misconduct, and that doctors involved in withholding results should be referred to the GMC.

http://www.alltrials.net/2013/bma-v...ion-of-trials-results-is-research-misconduct/



I am not sure if that only means the failure to publish any data at all because the trial was unsuccessful, or if it also includes the failure to publish any (significant) "unflattering trial data".

You mean this wasn't considered research misconduct before?

Geez, publishing all the data, in favor of your hypothesis or not, was beaten into me from day one. Even trying to spin the data would get you a field-wide dirty look and a bad reputation.

Medical research manages to have it's own definition of science, doesn't it? Or is it just the psychs? Probably not. I suspect pharma-sponsored research has some similar problems.

 

[alex3619]

If I recall correctly, 12% is getting close to the net number of patients who benefit in any way, even under the most generous definition of 'benefit'.

About 15% appeared to benefit, but that was after comparison to the SMC control group of 15%, iirc. The claimed result was, iirc, 30%. So we can't be sure that this wasn't the baseline "improvement" from the SMC. Nor can we be sure that these patients remained well at the end - other patients may have improved, and they might have got worse.

What we can say is that the data is inadequate to support their claims. This can be used to push for full disclosure of data.

It may not have been clear from my post though that the alternative is that they were indeed unwell, but that criteria for normal are stuffed ... something we have noted before.

 

[alex3619]

You mean this wasn't considered research misconduct before?

Geez, publishing all the data, in favor of your hypothesis or not, was beaten into me from day one. Even trying to spin the data would get you a field-wide dirty look and a bad reputation.

Medical research manages to have it's own definition of science, doesn't it? Or is it just the psychs? Probably not. I suspect pharma-sponsored research has some similar problems.

This problem was highlighted in big pharma research. Though they did tend to publish all the data from a study, they also liked to do a bunch of studies and publish only the favourable study. I do not recall details of this though.

A related problem in CBT/GET research for CFS is the abandoning of obtaining objective data as it did not correlate with subjective data. If you had to abandon one of them, surely it would be the subjective data? In PACE this took the form of abandoning actometer measurement. All psychogenic hypotheses trials need objective data ... there is too much uncertainty in the diagnosis/theories for subjective data to have value.

 

[Bob]

You mean this wasn't considered research misconduct before?

Geez, publishing all the data, in favor of your hypothesis or not, was beaten into me from day one. Even trying to spin the data would get you a field-wide dirty look and a bad reputation.

Medical research manages to have it's own definition of science, doesn't it? Or is it just the psychs? Probably not. I suspect pharma-sponsored research has some similar problems.

I read somewhere, the other day, that good science is to create a hypothesis and then spend the rest of your time trying your hardest to disprove it.