Is it About Getting Better or NOT Getting Worse? (Nov 7 blogpost)

[Tom Kindlon]

This November 7 blog post covers:

- PACE – little difference in treatments groups at follow-up

- Do Recovery Techniques Matter?

- Preventing Relapse

- Tips for Preventing Relapses

https://tipsforme.wordpress.com/2015/11/07/is-it-about-getting-better-or-not-getting-worse

[Comment: She mentions in passing a group CBT study. The fitness improvement mentioned was only achieved as the investigators excluded some values that didn't suit (e.g. two people not in the CBT group who had normal scores on the fitness test). If one includes these values, there was no statistically significant improvement in fitness. I submitted a comment on this which can also be read here: http://www.ncbi.nlm.nih.gov/pubmed/17014748#cm17014748_3673 (copied below)]

Additional information on/context for the walking test results (from the full text):
The abstract doesn't put the walking test results in context.

In the full text, the authors say:
"The ISWT, used as a physical performance measure, has normative reference data described by Taylor and colleagues.7 Their sample of 122 healthy subjects (mixed gender and age) walked a mean of 67 × 10-m shuttles."

Contrast this figure of 67 with the results achieved:

The CBT group started at: 24.3. After six months, were at 28.5 and at 12 months were 28.9.

The changes are thus minimal compared to normal functioning.

The full text also refers to outliers being excluded:
"Five clear outlying observations were omitted from the analysis of shuttles walked. Three were very low values (0 or 2) and two were amongst the highest values (60 and 75), but were from a patient with a low baseline score (9). If these outliers were retained, the SEs increased and difference between CBT and SMC was no longer statistically significant (p = 0.17)."

Whatever about excluding the low values, I question excluding the values of 60 and 75 which are the type of scores one sees in the normal population. It is not widely accepted that nobody with CFS ever gets back to normal functioning (I don't even think the authors believe this either). It is understandable to exclude scores that don't look like they represent the real values, but there is no evidence that the scores of 60 and 75 are anything other than the real, or valid, values.

Also, the wording, "two were amongst the highest values", suggests that there were other high values but these weren't excluded.

 

[Jenny TipsforME]

(Re my blog post, thanks for starting a thread @Tom Kindlon)

Is this study I was part of little known? Perhaps it wasn't positive enough about CBT? I'm happy to answer questions about the experience but it was a long time ago.

An oddity I'm not sure about is I didn't attend any of the GP surgeries mentioned as how they recruited the participants. I attended a trial at that time in that place and the protocol matches what happened. I was aware that the group I was in was some sort of control group (but I'm a psychology graduate so may sniff these things out more easily than most!). It may be that my results were excluded? Don't remember how my walking test went. Also my life was a little chaotic at that time with moving and stuff so I may have missed a final follow up or something. Details are hazy more than 10 years on with brainfog then and now!

 

[Sea]

"Five clear outlying observations were omitted from the analysis of shuttles walked. Three were very low values (0 or 2) and two were amongst the highest values (60 and 75), but were from a patient with a low baseline score (9). If these outliers were retained, the SEs increased and difference between CBT and SMC was no longer statistically significant (p = 0.17)."

Whatever about excluding the low values, I question excluding the values of 60 and 75 which are the type of scores one sees in the normal population. It is not widely accepted that nobody with CFS ever gets back to normal functioning (I don't even think the authors believe this either). It is understandable to exclude scores that don't look like they represent the real values, but there is no evidence that the scores of 60 and 75 are anything other than the real, or valid, values.

It seems that they've excluded the high values because they are so far removed from the entry score of that person. I guess they're admitting they don't believe their treatments can bring that big an improvement.

 

[Tom Kindlon]

It seems that they've excluded the high values because they are so far removed from the entry score of that person. I guess they're admitting they don't believe their treatments can bring that big an improvement.

Given the change in the p values, dipping just under the p<0.05 threshold, it looks like they excluded the scores of the people who didn't have therapy i.e. a competitor to their favourite therapy in the trial.

Longitudinal studies show some people with CFS, esp. in the early stage of the illness, do get better e.g. those who had EBV. The point of having a no therapy group in a trial is that one controls for natural changes that can occur with time: these changes can include some people getting better simply due to time.

 

[Jenny TipsforME]

@Tom Kindlon @Sea
Also this wasn't quite the same as "no therapy" in the comparison group. There was some organised content on the same themes as the CBT Sessions but in our sessions we were allowed to talk freely after 20mins and believe what we liked.

I think this is really key because some benefits of CBT may be nothing to do with false illness beliefs etc but everything about the nonspecific effcts of talking about different aspects of illness experience. I found it physically hard to get to the sessions but I enjoyed talking to other people and it was the first experience of group support I'd had.

It made me smile to read the protocol now though (obviously we didn't see it at the time) as the researchers must have been dying to redirect our conversation! We went off on tangents like the alcohol one. We were moaning about not being able to drink and one woman volunteered that a G&T was OK for her. In between sessions we tried drinking gin and reported back that it was better than wine etc. I doubt the CBT group got to try that one!

Also you have the ongoing issue that it's higher functioning pwme who take part in trials like this. Probably more people who feel on the way up so prepared to spend energy going to sessions that may be placebo. If you are in a phase of cutting down activity you wouldn't agree to it (drug treatment may work the other way around?). I can't say much in case it identifies individuals but on average my group was functioning better than typical patients I meet online. I was working part time and 2 of the group were in full time work (but still getting symptoms).

 

[Tom Kindlon]

The full paper of the O'Dowd study is available here:
http://www.journalslibrary.nihr.ac.uk/hta/volume-10/issue-37#hometab0

This is longer than most normal papers (around 100 pages if you read all the detail, which I have)

 

[Tom Kindlon]

@Tom Kindlon @Sea
Also this wasn't quite the same as "no therapy" in the comparison group. There was some organised content on the same themes as the CBT Sessions but in our sessions we were allowed to talk freely after 20mins and believe what we liked.

Yes, I know. There were three arms of the trial:

The three interventions were group CBT incorporating graded activity scheduling, education and support group (EAS) and standard medical care (SMC).

 

[Jenny TipsforME]

Ah yes, so the excluded data was from the standard medical care? No cognitive spoons left to check doc today.

 

[Tom Kindlon]

Ah yes, so the excluded data was from the standard medical care? No cognitive spoons left to check doc today.

I presume so given SMC is mentioned:

If these outliers were retained, the SEs increased and difference between CBT and SMC was no longer statistically significant (p = 0.17)."

 

[Jenny TipsforME]

I've had a read through of the large report today. I have a few thoughts on pros and cons in the methodology too.

Positives

Control groups They attempted 2 types of control - a plain control group (standard medical care - which probably means some anti depressants and otherwise being ignored!) and the placebo therapy group

Objective measures were used. This is important where treatment may be interfering with illness beliefs and willingness to report symptoms. The shuttle walk seemed like a good test of walking ability from my memory of it. For some reason it is meant to allow less bias than the 6 minute walk test although I didn't read the reference to find out why. A battery of neurocognitive tests were also administered by a computer (presumably the computer lacks bias!). It is worth noting that there was no significant difference between groups with these tests.

Sampling: It is claimed the participants were representative of patients. Patients baseline scores were lower than referenced prior studies (virtually all studies exclude Severe ME which may be the most worthwhile group to study...).

Free thinking: the authors note that participants in EAS group naturally brought up the same topics as CBT. The qualitative experience of this was better than having other people's beliefs imposed on you or your own beliefs questioned though. When not backed into a corner pwme are nuanced enough to know that there's some relationship between health, thoughts and behaviour.

Transparent data: they provide data upfront.

Null hypothesis: The researchers are clear that the 3 primary outcomes were not met (although they included significant findings that weren't the primary outcomes), they also seem to be fairly forthcoming on their perception of the limitations of the study and they haven't tried to reinvent the aims etc. This report complies with research writing format as I was taught at uni.

Ethics: In the implications they engage with the ethics of managing patient expectations. Since CBT failed to return participants to normal they see it as unethical to claim it cures if used with subsequent patients. It would however be OK to say that they could see some improvement.

Negatives/Queries

Outliers - Tom has covered the stats side and that's never been a strength of mine so I didn't pay much attention to it. Excluding normal values does seem odd.

Double blind? The study is described as double blind, and I can see why, but in standard medical care there could be no blinding, it was clear to me that the group I was in was designed as a placebo/support group and many pwme can recognise CBT or GET when they see it (or was that not the case 2000-2002?).

Placebo? Although it was clear the EAS group was meant as a group therapy placebo, I do also think it could have had effects that are relevant to ME. We did relaxation techniques and covered things like heart rate. Since there are autonomic and parasympathetic tone issues in ME this could have had some use that's not placebo or general to any support group. In EAS we shared tips with each other and the report notes that some participants arrived with good coping skills. An interesting note is that alternative health engagement went from 30% to 65% in this group. I read that and thought it was an excellent example of how we influenced each other, but this seems to be missed in the report.

Mechanism of action: no attempt was made to measure any changes in cognition. This makes it impossible to know if positive trends in the CBT group were to do with the theoretical framework of CBT or something else.

Diagnosis: Patients weren't formally diagnosed by specialists and Fukuda was used. Mood disorder was included as a core aspect of ME/CFS which isn't the case (confused by incidence of comorbid depression). No-one with Severe ME participated because they couldn't attend the sessions.

Baseline: to start with 30% of participants were in the normal range on the SF-36. If I was conducting this research I would have excluded this group as already too well recovered to give meaningful results.

Questionnaires: I remember being frustrated by these questionnaires as they didn't provided the answers I wanted to give. They seemed to assume only negative experience compared to prior wellness in relation to mental health. For example, I wanted to say that I enjoyed some activities more (as in the experience had taught me to appreciate some things more). Also, I couldn't really remember what it was like to be well. The scales measuring that are only measuring your interpretation of memory which is influenced by current mood (I'd suggest to an extent that it would be mostly a measure of how bad/negative you feel today and little to do with how you functioned years ago when well). There may be floor effects with the SF-36 which was the main subjective measure (too many people score low for it to be sensitive to differences).

That's my two pennies worth anyway, many of these limitations are acknowledged by the researchers themselves.