PACE Trial and PACE Trial Protocol

[Bob]

Yay! You got there first Dolphin!!!

Happy 1000th post everyone!



I think everyone should take a moment to give themselves a good pat on the back for all the enormous amount of effort put into the subject.
:Sign Good Job::Sign Good one:

I'm optimistic that our community's combined efforts will make a big difference in the medium term, and the Trial will be seen for what it is, at least by the people who matter.

 

[Esther12]

I thought I'd write an easy-to-read message for the 1000th message.

Also, a bit of pacing advice: if you have just started reading this thread and have got to here, it might be time to have a little break.

lol. I got burnt-out around page 60. I've been barely taking it all in since then.

 

[WillowJ]

PACE set a threshold of 'normal' fatigue as a CFQ score of 18 or less, based on a mean of 14.2 and SD of 4.6, which were taken from a 2010 study, Measuring fatigue in clinical and community settings. But these figures may well not be representative of the working age population.

The way they selected the participants is complex, and the underlying data was collected published in 1994: Population based study of fatigue and psychological distress (T Pawlikowska, T Chalder, S R Hirsch, P Wallace, D J M Wright, S C Wessely), and this is where the holes start appearing.

Crucial bit is point 4 if you're pressed for time.
...

4.Data from the original study indicate this is an unhealthy cohort.
According to Pawlikowska, 38% of patients had a score about the original Chalder bimodal cut off of 3 (as used in the PACE protocol) and 18.3% of patients were substantially fatigues for 6 months or longer. Whoa, that looks unhealthy, esp as the paper quotes a 1990 paper that found only 10% of GP practice patients had fatigue for one month or more. I think there are some US studies indicating fatigue of over 6 months in the population is much less than 18%.

So, I'm pretty fatigued now, and so are you if you've read this far. But it looks like PACE have been using highly unsuitable 'normative' data. Again.

ETA: should mention that the Cella CFQ data is not normally distributed and therefore, like the SF36 data, is not suitable for use with parametic stats, such as the 'within 1 SD of the mean' formula used by PACE:

I think I am a long ways back on the thread, but if you still need data on fatigue in the normal population, it is about 5%.

If about 5% of the
population has 6 or more months of fatigue, and about
half of this is due to clear medical or psychiatric reasons
[31], then the critical question is how many of the remaining
2.5% have CFS.

31. L. A. Jason, J. A. Richman, A. W. Rademaker, K. M.
Jordan, A. V. Plioplys, R. Taylor, et al., A Community-
Based Study of Chronic Fatigue Syndrome, Archives
of Internal Medicine, Vol. 159, 1999, pp. 2129-
2137.

Jason LA, Evans M, Brown A, Brown M, Porter N, et al. "Sensitivity and Specificity of the CDC Empirical Chronic Fatigue Syndrome Case Definition." Psych. 2010 Apr; 1(1):9-16

 

[WillowJ]

Patients whose main symptom is fatigue may be developing multiple sclerosis:

http://www.medpagetoday.com/MeetingCoverage/CMSC/26848

Months or even years before demyelinating events are recognized, patients later diagnosed with multiple sclerosis often turn up in their doctors' offices complaining of fatigue, a researcher said here.

Among patients receiving MS diagnoses after 2005, records of nearly 30% indicated that they had reported fatigue to their physicians beforehand, said Joseph Berger, MD, of the University of Kentucky in Lexington, in a presentation at the annual meeting of the Consortium of Multiple Sclerosis Centers (CMSC).
...
Claims data for those patients during the years before MS diagnosis indicated that 1,534 (29%) had ICD-9 codes for either chronic fatigue syndrome or malaise, which were taken to indicate a patient complaint of fatigue. All but 73 of these patients were coded only for malaise.

Among these patients, records indicated that 31% experienced only fatigue without any other symptoms that might be associated with MS, Berger indicated. These included peripheral neuropathies or movement problems, visual or speech disturbances, or dizziness.

And, in the other patients with fatigue who also had other symptoms that could have been related to MS prior to formal diagnosis, records of 38% indicated that fatigue was the first symptom reported to their physicians.

Nearly one-third of the patients with fatigue had seen their physicians with the problem two to three years before diagnosis, and another quarter had reported the symptom one to two years before.

The researchers also examined drug prescriptions following these visits prior to MS diagnosis. They indicated that only 10% of patients received prescriptions for agents often given for fatigue, such amphetamines or modafinil (Provigil).
...
He also suggested that the study's numerical findings were almost certainly underestimates of the frequency of fatigue, insofar as many physicians, including himself, rarely record it with a specific ICD-9 code on their patients' charts.

 

[WillowJ]

Your suggestion for GET being affected by response bias on the 6MWD is certainly possible, but I'm not sure if it's any more plausible than GET having a small but real effect on walking capability. And I would have predicted that SMC would have a smaller response bias than APT, but that doesn't appear to be the case. There were only 5 sessions of SMC with relatively low expectations and satisfaction, compared with 13 sessions of APT (plus 3 of SMC), with a strong therepuetic alliance and high satisfaction. APT is set up for response bias.

It appears from the data that something is going on, but i'm struggling to see a clear, compelling explanation for what it is. Help welcome.

I might have a crack at a deluxe graph in due course.

I would expect that getting SMC where they were offered antidepressants, pain control, and sleeping pills would help. Treating pain and sleep and, when present, depression, is basic. We see this huge jump right at the beginning, and I expect that's from having basic symptom control for pain and sleep and, if needed, depression. That's a huge improvement over "nothing much is wrong with you" or "you should get out more" or antidepressants alone.

 

[Dolphin]

Happy 1000th post everyone!



I think everyone should take a moment to give themselves a good pat on the back for all the enormous amount of effort put into the subject.
:Sign Good Job::Sign Good one:

I'm optimistic that our community's combined efforts will make a big difference in the medium term, and the Trial will be seen for what it is, at least by the people who matter.

Yes, fingers crossed. The results aren't impressive as they thought they would be/some previous smaller RCTs were.

 

[Dolphin]

"When and how can endpoints be changed after initiation of a RCT?"

(Probably not that important)

I just read the following paper:

Free full text: http://clinicaltrials.ploshubs.org/article/info:doi/10.1371/journal.pctr.0020018
When and how can endpoints be changed after initiation of a randomized clinical trial?

to see whether the PACE Trial might qualify.

To me, they don't seem to fit the criteria. I started a thread on it at http://forums.phoenixrising.me/show...ndpoints-be-changed-after-initiation-of-a-RCT where I underlined some bits I thought were of interest (as I said, I partly read it with the PACE Trial in mind)

 

[oceanblue]

Wow, over 1,000 posts. I'm not sure if that's cause for celebration or tears.

I think I am a long ways back on the thread, but if you still need data on fatigue in the normal population, it is about 5%.

Thanks for that. Interestingly, Lenny Jason also used the Chalder Fatigue scale (CFQ) for that study but doesn't say what threshold was used to define 'fatigued' - and it may be that 'chronic fatigue' cases were actaully assessed by other questions on fatigue:

Interviewees who reported that they had severe fatigue, extreme tiredness, or exhaustion for 6 months or longer were asked additional questions on the CFS Screening Questionnaire...

Is this based on a CFQ score or on the answer to other questions? This makes it hard to compare fatigue rates - and this is true across fatigue studies generally, they tend use different measure of fatigue making it even harder to pin down the true rate of chronic fatigue in the population.

 

[Esther12]

I started a thread on it at http://forums.phoenixrising.me/show...ndpoints-be-changed-after-initiation-of-a-RCT where I underlined some bits I thought were of interest (as I said, I partly read it with the PACE Trial in mind)

I only skimmed your highlights, but it looked interesting. Thanks. I wonder how widely accepted these sorts of concerns about changing measures are? It seems obviously problematic to me, but not many other than CFSers seem concerned with regards to Pace. Is that because no-one cares about CFS patients being misled, no-one really cares about Pace, or because we've misunderstood how widespread and accepted these sorts of shenanigans are?

 

[Dolphin]

Thanks for that. Interestingly, Lenny Jason also used the Chalder Fatigue scale (CFQ) for that study but doesn't say what threshold was used to define 'fatigued' - and it may be that 'chronic fatigue' cases were actaully assessed by other questions on fatigue:

Interviewees who reported that they had severe fatigue, extreme tiredness, or exhaustion for 6 months or longer were asked additional questions on the CFS Screening Questionnaire...

Is this based on a CFQ score or on the answer to other questions? This makes it hard to compare fatigue rates - and this is true across fatigue studies generally, they tend use different measure of fatigue making it even harder to pin down the true rate of chronic fatigue in the population.

They have their own screening questionnaire but as you point out, it doesn't seem to be clear from the main paper how they used it.

The reference in the 1999 Jason et al. paper is:

28. Jason LA, Ropacki MT, Santoro NB, et al. A screening scale for chronic fatigue syndrome: reliability and validity. J Chronic Fatigue Syndrome. 1997;3:39-59.

They used Likert scoring for the Chalder Fatigue Questionnaire:

The first part of the phase 1 screening questionnaire also contained the Fatigue Scale30 and other questions assessing quality and duration of fatigue.

The Fatigue Scale provides a continuous distribution of fatigue scores. Despite its brevity, this scale was reliable and valid, possessing good face validity and reasonable discriminant validity. The 11-item scale has responses rated on a 4-option continuum; total scores range from 0 to 33 (with higher scores signifying greater fatigue).

The definition of caseness is bimodal score (4 or more). So like you, I'm not sure how they used it. I might keep an eye out - they published lots and lots of papers based on that study so perhaps it gets a mention somewhere.

 

[Dolphin]

I started a thread on it at http://forums.phoenixrising.me/show...ndpoints-be-changed-after-initiation-of-a-RCT where I underlined some bits I thought were of interest (as I said, I partly read it with the PACE Trial in mind)

I only skimmed your highlights, but it looked interesting. Thanks. I wonder how widely accepted these sorts of concerns about changing measures are? It seems obviously problematic to me, but not many other than CFSers seem concerned with regards to Pace. Is that because no-one cares about CFS patients being misled, no-one really cares about Pace, or because we've misunderstood how widespread and accepted these sorts of shenanigans are?

This registering trial protocols seems fairly new in general to medicine. So then not following them is gong to be a bit new also.

I haven't heard much talk about not following protocols in the ME/CFS world before the last year or so.
Indeed, even talking about outcome measures is fairly newish. For most of the 17 years I've been reading ME/CFS literature, the criticism has been on criteria used and maybe occasionally questionnaires don't give all the answers - but very little focus on particular questionnaires, etc.

One problem with the PACE Trial is how much there is to read: the protocol paper itself is quite long and then if you're not familiar with the questionnaires, it can be easy to forget what is involved. I think to really get on top of it, one probably needs to read the full PACE protocol paper which is around 200 pages - and I'm not sure how many people have done that. One good thing about that is that it includes all the questionnaires - for example, even though I've heard talk of the WSAS in various studies before, I'd never seen it before I think. It's much easier to get a grip of the questionnaires, etc. if one reads the questions.

One of the reasons it seems particularly egregious to me is that they actually published their protocol in a medical journal.

 

[Esther12]

It's much easier to get a grip of the questionnaires, etc. if one reads the questions.

Absolutely.

As you say though, it's a lot of work. It's too easy to assume the questionnaires are measuring some abstract, rather than measuring the answers to the questions presented. I find myself still doing this, particularly with research that reinforces my own beliefs.

It is all a lot of work though, and difficult to divide up between different people, so especially hard for ill patients to keep up with. We should be given a bright healthy person to do this for us!

 

[Sean]

One problem with the PACE Trial is how much there is to read:

And take on board generally about the science and history of this disorder. I can understand how otherwise intelligent, qualified, decent people can be seriously misled if they only do a relatively quick perusal of the literature, or have a quiet off-the-record chat with one or two of the 'world experts'. That might be fine for getting up to speed with the latest in setting fractured femurs, but it is arguably worse than wilful ignorance when it comes to ME/CFS.

As for changing endpoints during the trial, it may well be legitimate in principle to do that in some circumstances. But in PACE they way they changed things is not okay. It was clearly to try to salvage a minimal result from a serious disaster.

 

[Dolphin]

Absolutely.

As you say though, it's a lot of work. It's too easy to assume the questionnaires are measuring some abstract, rather than measuring the answers to the questions presented. I find myself still doing this, particularly with research that reinforces my own beliefs.

Astute point

 

[Dolphin]

And take on board generally about the science and history of this disorder. I can understand how otherwise intelligent, qualified, decent people can be seriously misled if they only do a relatively quick perusal of the literature, or have a quiet off-the-record chat with one or two of the 'world experts'. That might be fine for getting up to speed with the latest in setting fractured femurs, but it is arguably worse than wilful ignorance when it comes to ME/CFS.

Yes. This is one of the reasons I think it is important to challenge, ideally "on the record" i.e. in the published literature. Otherwise, research and researchers can seem to have covered all the bases. It was unfortunate how things worked out with the Lancet editorial accompanying the Lancet paper but I think there were other reasons apart from them getting lots of letters (I imagine anyone who has a letters page gets quite a few letters - certainly newspapers get lots and lots).

I think it's important to put doubt in readers mind so they know they need to hear both sides - that they can't assume the "establishment" position is correct.

 

[Andrew]

I don't know if this has already been posted. It is PACE page that has the manuals

http://www.pacetrial.org/trialinfo/

 

[biophile]

Sorry I have been out of commission for over a month.

This is probably in the wrong thread since I can't find it now, but it was a drafted answer to a question someone posted about the effect sizes in the PACE trial about a week ago (?), someone wanted to know if there were additional ways of explaining how these effect sizes were small.

Anyway, another way not mentioned in the question would be from the perspective of odds ratios. From memory and a quick check, none of the ORs given in the PACE trial reach 2.5 (or equivalent 1/x inverse), which in Cohen's rules of thumb is the threshold for "moderate".

In the original protocol they talk about an OR of 2-3 being the threshold for clinical significance, in the results these are around 2 more or less, so barely scraping in at best.

 

[Bob]

hi biophile,

Great to have you back with us.

I think you might be answering my question, when I asked if anyone is aware of any way of challenging the word 'moderately' (i.e. "moderately effective").

Thanks for your info... it's very interesting... I'm not familiar with 'Cohen', but I'll do some googling.

Thanks,
Bob


BTW, Dolphin, thanks very much for the info that you posted in answer to my question.

 

[Bob]

OK, now I need a statistician to explain Cohen's rules of thumbs, and to re-interpret the PACE Trial results, in language that I can easily understand!

biophile? Graham?

Here's helpful links re Cohen's rules of thumbs:
http://web.me.com/rsbalkin/Site/Research_Methods_and_Statistics_files/Effect Size outline.pdf
http://staff.bath.ac.uk/pssiw/stats2/page2/page14/page14.html

 

[oceanblue]

I only skimmed your highlights, but it looked interesting. Thanks. I wonder how widely accepted these sorts of concerns about changing measures are? It seems obviously problematic to me, but not many other than CFSers seem concerned with regards to Pace. Is that because no-one cares about CFS patients being misled, no-one really cares about Pace, or because we've misunderstood how widespread and accepted these sorts of shenanigans are?

These sorts of shenanigans are widespread (which is why the article got written, and why the Lancet has guidelines on them...). Pharmaceutical companies have been the trailblazers here but researchers in many fields have been happy to adjust endpoints etc to make their results look good.