FITNET’s Internet-Based Cognitive Behavioural Therapy Is Ineffective and May Impede Natural Recovery

[slysaint]

also featured on ProHealth; you can comment and also *rate
http://www.prohealth.com/library/showarticle.cfm?libid=30653

 

[Tom Kindlon]

Some people might be interested in the letters Joan Crawford and I had published in the Lancet on the original FITNET trial:

http://www.thelancet.com/pdfs/journals/lancet/PIIS0140-6736(12)61325-7.pdf

 

[Dolphin]

This paper contains a lot of useful information. I doubt I'm going to highlight all of it.

However I think it is important that I also highlight some points if I think they are incorrect or am not sure they are correct.

Here is one:

The trial’s primary goal was the recovery rate at long-term follow up (LTFU). Primary outcomes were fatigue severity, physical functioning and school attendance assessed via questionnaires. The trial concluded that after six months, i.e., at end of treatment, 60% of participants had recovered in the Internet based CBT group and only 8% in the usual care group. At LTFU (mean of 2.7 years after treatment’s start) the positive effects of FITNET were maintained, 58.9% of adolescents had recovered from CFS. Usual care led to similar recovery rates, although these rates were achieved at a slower pace [9].

The long-term follow-up paper does say:

The primary goal of this study was to determine long-term recovery rates of adolescent CFS patients who participated in the FITNET trial. In addition, factors related to recovery were assessed.

http://pediatrics.aappublications.org/content/131/6/e1788.long

However I think that may be referring to the primary goal of the 2013 paper.

The full FITNET protocol can be seen here:
https://bmcneurol.biomedcentral.com/articles/10.1186/1471-2377-11-23

At some points it is a bit unclear. However at one point it does say:

The main efficacy analysis will pertain to the data obtained after a 6 months FITNET or usual care condition.

This to me would make more sense. After this point, the control group can then do FITNET so it becomes a lot messier to judge the effects of FITNET.

Edited to add: Esther12 has now found one registry entry below which suggests the primary outcome point was at 12 months (though there is disagreement about which time point that means) but it would not be the data published in the Lancet paper. But neither the protocol nor the registry entries mention recovery as the primary outcome.

 

[Dolphin]

Treatment in the study finished after 6 months and the first follow up was at the end of it. A systemic review by Whiting et al. concluded that due to the relapsing nature of CFS, follow-up should continue for at least a further 6–12 months after treatment has ended, to confirm that any improvement observed was due to the intervention and not to the naturally occurring fluctuations of CFS [15].

15. Whiting, P.; Bagnall, A.M.; Sowden, A.J.; Cornell, J.E.; Mulrow, C.D.; Ramírez, G. Interventions for the Treatment and Management of Chronic Fatigue Syndrome—A Systematic Review. JAMA 2001, 286, 1360–1368. [CrossRef] [PubMed]

Good quote. See:

There is little evidence from the literature concerning the appropriate duration and follow-up of interventions used in the management of CFS. However, as CFS is by definition longterm, it would seem sensible to follow up participants for an appropriate period of time. The relapsing nature of CFS suggests that follow-up should continue for at least an additional 6 to 12 months after the intervention period has ended, to confirm that any improvement observed was due to the intervention itself and not just to a naturally occurring fluctuation in the course of the illness.

 

[Dolphin]

According to Prasad, Cifu and Ioannidis, the use of short-term outcome evaluation affects the credibility of clinical trial results in a negative way [17].

Good find:

The increasing use of surrogate end points and short-term outcomes has also affected the credibility of clinical trial results.

 

[Dolphin]

Also, 21% of patients had a comorbid depression and 14% a comorbid anxiety disorder [12], meaning that up to 35% of patients had comorbid mental health issues, which is almost the same as the 36% of patients classed as ‘recovered’ by the FITNET trial when only one standard deviation was used for their definition of recovery [12]. This is also the very subset of patients one would expect to respond positively to CBT as a meta-analysis by Tolin et al. found cognitive behavioural therapy to be the most effective therapy for anxiety and depression [25].

Reeves et al., co-authored by one of the FITNET investigators, concluded in 2003 that patients with a comorbid medical or psychiatric condition that may explain the chronic fatigue state should be excluded from CFS research studies because overlapping pathophysiology may confound findings specific to CFS [26]. Why the FITNET trial did not do this is unclear.

26. Reeves, W.C.; Lloyd, A.; Vernon, S.D.; Klimas, N.; Jason, L.A.; Bleijenberg, G.; Evengard, B.; White, P.D.; Nisenbaum, R.; Unger, E.R.; et al. Identification of ambiguities in the 1994 chronic fatigue research case definition and recommendations for resolution. BMC Health Serv. Res. 2003, 3. [CrossRef] [PubMed]

It is probably reasonable to raise this issue if one is being thorough in terms of possible problems. However it doesn't seem to me to be the norm that depression or anxiety are full exclusions from CFS studies. Perhaps they should be but that is not how most researchers use such criteria.

 

[Esther12]

The trial’s primary goal was the recovery rate at long-term follow up (LTFU). Primary outcomes were fatigue severity, physical functioning and school attendance assessed via questionnaires. The trial concluded that after six months, i.e., at end of treatment, 60% of participants had recovered in the Internet based CBT group and only 8% in the usual care group. At LTFU (mean of 2.7 years after treatment’s start) the positive effects of FITNET were maintained, 58.9% of adolescents had recovered from CFS. Usual care led to similar recovery rates, although these rates were achieved at a slower pace [9].

The long-term follow-up paper does say:

The primary goal of this study was to determine long-term recovery rates of adolescent CFS patients who participated in the FITNET trial. In addition, factors related to recovery were assessed.

http://pediatrics.aappublications.org/content/131/6/e1788.long

However I think that may be referring to the primary goal of the 2013 paper.

The full FITNET protocol can be seen here:
https://bmcneurol.biomedcentral.com/articles/10.1186/1471-2377-11-23

At some points it is a bit unclear. However at one point it does say:

The main efficacy analysis will pertain to the data obtained after a 6 months FITNET or usual care condition.

This to me would make more sense. After this point, the control group can do FITNET so it becomes a lot messier to judge the effects of FITNET.

I hate commenting on FITNET, as I never really properly dug into it, but this trial registration says:

Primary Outcome Measures:

• School presence [ Time Frame: one year ]
• Severity of fatigue [ Time Frame: one year ]
• Physical functioning as measured by the subscale physical functioning [ Time Frame: one year ]

https://www.clinicaltrials.gov/ct2/show/NCT00893438

This other registration saya:

Primary outcome measures
1. School presence
2. Severity of fatigue
3. Physical functioning as measured by the subscale physical functioning

Efficacy of the web-based programme will be determined after these 6 months. The follow-up measurement at 12 months is meant to define if the result.

http://www.isrctn.com/ISRCTN59878666

Then the Lancet abstract said:

FITNET was significantly more effective than was usual care for all dichotomised primary outcomes at 6 months

http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(12)60025-7/abstract

LTFU was longer than 1 year. Was the 1 year data even published?

 

[Dolphin]

3.3. Using Primary Outcomes Assessed via Questionnaires

One of the problems with outcomes assessed via questionnaires in an unblinded trial is ‘response-shift bias’. This occurs when an intervention leads individuals to change their evaluation standard with regard to the dimension measured leading the therapist (and often also the patient) to conclude erroneously that the treatment has worked [16]. This is even more of a problem when the therapy used, in this case internet-based CBT, aims to modify participants’ beliefs and perception of their symptoms.

Other important causes of bias are ‘effort justification’ where patients investing substantial time, energy and effort in an intervention often feel a psychological need to justify this commitment. There is also a tendency for patients/clients to report improvement in accord with what they believe to be the therapist’s/researcher’s hypotheses [16].

Good points

 

[Dolphin]

I hate commenting on FITNET, as I never really properly dug into it, but this trial registration says:



https://www.clinicaltrials.gov/ct2/show/NCT00893438

This other registration saya:

Primary Outcome Measures:

• School presence [ Time Frame: one year ]
• Severity of fatigue [ Time Frame: one year ]
• Physical functioning as measured by the subscale physical functioning [ Time Frame: one year ]

http://www.isrctn.com/ISRCTN59878666

Then the Lancet abstract said:




http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(12)60025-7/abstract

LTFU was longer than 1 year. Was the 1 year data even published?

That's interesting. I don't recall seeing those registry entries.

To me 12 months would refer to timepoint 2 in this diagram.

Data on Timepoints 2 & 3 were published in a second paper in 2013:
http://pediatrics.aappublications.org/content/pediatrics/early/2013/05/08/peds.2012-2007.full.pdf

Here is a summary of the results

 

[Esther12]

To me 12 months would refer to timepoint 2 in this diagram.

Data on Timepoint 3 was published in a second paper in 2013.

I'd have assumed it meant T3, but I guess that is ambiguous, and that they started providing FITNET to the control group at 6 months does mean it would be odd for them to use 1 year as their primary outcome.

I edited in another FITENT registration above, that is also a bit unclear:

Primary outcome measures
1. School presence
2. Severity of fatigue
3. Physical functioning as measured by the subscale physical functioning

Efficacy of the web-based programme will be determined after these 6 months. The follow-up measurement at 12 months is meant to define if the result.

http://www.isrctn.com/ISRCTN59878666

I don't know what I'm talking about with FITNET... the null result at LTFU gave me an excuse to not try to understand the details of it!

 

[Dolphin]

It is interesting looking at 2013 paper again. It does not appear to give the results for the primary outcome measures at either timepoint 2 or timepoint 3, just the recovery rates and results for those who didn't or did not recover as they defined it.

 

[Dolphin]

3.4. The Actometer Results

The Dutch FITNET protocol states that it will be measuring physical performance objectively by using the actometer [11]. However, the results were not reported and the reason for this was not given. Heneghan et al. refer to this as a typical example of “Outcome reporting bias” which “occurs when a study has been published, but some of the outcomes measured and analysed have not been reported”; this “significantly affects the validity” of a study [31] (p. 4).

Wiborg et al. reanalyzed three trials which had reported subjective improvements without reporting their actometer results, and found that CBT didn’t lead to objective improvements [32]. One of the authors of the Dutch FITNET trial [9,12], who is also involved in the NHS FITNET trial [10], was involved in these 3 studies [32].

It was worth highlighting that they never published any data regarding the actometer.

However technically I don't think it qualifies as outcome reporting bias as it wasn't specified as an outcome, just a possible predictor.

The 2013 paper does include:

TABLE 3 Factors Related to Recovery After FITNET Treatment at LTFU

and it should have been referred to there

 

[Dolphin]

Wood et al. in a large scale meta-analysis of over 1300 varied clinical trials, found that in unblinded trials, subjectively assessed outcomes increase the degree of bias and introducing objective outcomes reduces this [27]. Other systemic reviews of clinical trials also concluded that lack of patient blinding combined with self-reporting of outcomes leads to pronounced bias as they become prone to outside influences leading to the erroneous inference of efficacy in its absence, thus making them unreliable [16,28]. Low correlation between objective and subjective activity measurements [29] is not confined to the chronically ill but is also present in the healthy population [30].

Good. It is impressive the scholarly work involved in making such points with references

Here is reference number 27
http://www.bmj.com/content/336/7644/601

Main outcome measures Ratios of odds ratios quantifying the degree of bias associated with inadequate or unclear allocation concealment, and lack of blinding, for trials with different types of intervention and outcome. A ratio of odds ratios <1 implies that inadequately concealed or non-blinded trials exaggerate intervention effect estimates.

Results In trials with subjective outcomes effect estimates were exaggerated when there was inadequate or unclear allocation concealment (ratio of odds ratios 0.69 (95% CI 0.59 to 0.82)) or lack of blinding (0.75 (0.61 to 0.93)). In contrast, there was little evidence of bias in trials with objective outcomes: ratios of odds ratios 0.91 (0.80 to 1.03) for inadequate or unclear allocation concealment and 1.01 (0.92 to 1.10) for lack of blinding. There was little evidence for a difference between trials of drug and non-drug interventions. Except for trials with all cause mortality as the outcome, the magnitude of bias varied between meta-analyses.

This is a reference number 29:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2674446/

Health Qual Life Outcomes. 2009; 7: 29.
Published online 2009 Apr 1. doi: 10.1186/1477-7525-7-29
PMCID: PMC2674446
Clinical assessment of the physical activity pattern of chronic fatigue syndrome patients: a validation of three methods
Korine Scheeres,

1 Hans Knoop,#1 van der Jos Meer,#2 and Gijs Bleijenberg#1

Abstract
Background
Effective treatment of chronic fatigue syndrome (CFS) with cognitive behavioural therapy (CBT) relies on a correct classification of so called 'fluctuating active' versus 'passive' patients. For successful treatment with CBT is it especially important to recognise the passive patients and give them a tailored treatment protocol. In the present study it was evaluated whether CFS patient's physical activity pattern can be assessed most accurately with the 'Activity Pattern Interview' (API), the International Physical Activity Questionnaire (IPAQ) or the CFS-Activity Questionnaire (CFS-AQ).

Methods
The three instruments were validated compared to actometers. Actometers are until now the best and most objective instrument to measure physical activity, but they are too expensive and time consuming for most clinical practice settings. In total 226 CFS patients enrolled for CBT therapy answered the API at intake and filled in the two questionnaires. Directly after intake they wore the actometer for two weeks. Based on receiver operating characteristic (ROC) curves the validity of the three methods were assessed and compared.

Results
Both the API and the two questionnaires had an acceptable validity (0.64 to 0.71). None of the three instruments was significantly better than the others. The proportion of false predictions was rather high for all three instrument. The IPAQ had the highest proportion of correct passive predictions (sensitivity 70.1%).

Conclusion
The validity of all three instruments appeared to be fair, and all showed rather high proportions of false classifications. Hence in fact none of the tested instruments could really be called satisfactory. Because the IPAQ showed to be the best in correctly predicting 'passive' CFS patients, which is most essentially related to treatment results, it was concluded that the IPAQ is the preferable alternative for an actometer when treating CFS patients in clinical practice.

 

[docsimsim]

I'm reading this now. It makes a lot of good points. Thanks to the authors for the effort that they put in.

One small correction I have just come across:








Reference 47 refers to the SF-36 physical functioning subscale. That is not the same as the Physical functioning (Child Health Questionnaire).



The former has 10 questions, the latter nine:

TH

I'm reading this now. It makes a lot of good points. Thanks to the authors for the effort that they put in.

One small correction I have just come across:








Reference 47 refers to the SF-36 physical functioning subscale. That is not the same as the Physical functioning (Child Health Questionnaire).



The former has 10 questions, the latter nine:

TU for pointing this detail out

 

[docsimsim]

It was worth highlighting that they never published any data regarding the actometer.

However technically I don't think it qualifies as outcome reporting bias as it wasn't specified as an outcome, just a possible predictor.
View attachment 23265
The 2013 paper does include:

and it should have been referred to there

Hi. Is that table (where actometer is a predictor not an outcome ) from the Dutch trial? Tx

 

[Dolphin]

Hi. Is that table (where actometer is a predictor not an outcome ) from the Dutch trial? Tx

Yes, the protocol paper:
https://bmcneurol.biomedcentral.com/articles/10.1186/1471-2377-11-23

 

[docsimsim]

Yes, the protocol paper:
https://bmcneurol.biomedcentral.com/articles/10.1186/1471-2377-11-23

TY, much appreciated. Have sent you a private convo message. Hope you don't mind. TC

 

[Dolphin]

4.3. Maternal Concern

The authors stated that a factor related to recovery at LTFU was “maternal focus on bodily symptoms” and concluded that it “suggests that an intergenerational vulnerability and interaction between mother and child may exist” [9] (p. e1794). Yet it is normal for a parent to worry about (the symptoms of) her ill child when there’s no recovery or improvement and maternal concern is a variable that will itself be influenced by how ill the child is and concluding that it increases illness duration is confusing cause with effect.

I thought this was an astute point. However I have now looked the protocol paper and wonder was this a measurement at baseline in which case the argument would not apply or at least would not be a strong.

 

[Dolphin]

With regards to school attendance, presence of 90% [9] was required to be classed as recovered. School presence at the end of treatment, i.e., at 6 months, was measured by looking at the two previous weeks, which is too short a time window to make a meaningful conclusion about recovery

Good point

 

[Dolphin]

In addition, patients with CFS often have concentration and memory problems [58], so school attendance does not translate to school performance

Good point