1. I think you pointing out adherence is a good tact to take. It's funny how none of the papers bring up the actigraphy results after they talk about it in methods!
2. The issue of combining the three groups is problematic in that:
- the Prins papers says that they used CDC criteria but then not the "4/8" symptoms; they explain this away by saying 18 were diagnosed with "idiopathic chronic fatigue" in one of the letters -- why didn't they just throw out this small group? It messes up the data.
Patients were eligible for the study if they met the US Centers for Disease Control and Prevention criteria for CFS,1 with the exception of the criterion requiring four of eight additional symptoms to be present.
- the interventions are different; the Stule. paper has parents participating and states.
Both protocols differed from the treatment of adults.
- while the individual papers show a "baseline" chart to show that randomization suceeded, combining the three groups again should mean that they need to re-do a "baseline" chart (to be real picky, the hours worked in Prin (16-CBT vs. 12-13 other groups) and the duration of illness - between the groups in Knoop (72 months vs. 96 months) showed randomization might not have succeeded. Values should be similar.
http://www.consort-statement.org/consort-statement/13-19---results/item15_baseline-data/)
3. My main issue is that, for a paper about detrimental effects, the authors need to elucidate from the subjects WHY they withdrew, especially with a drop out rate of about 20%. Accounting for missing values statistically in the analysis by defaulting missing values to "deterioration" status is reasonable but doesn't cover reasons for withdrawal beyond what the authors hypothesized. For example, if people withdrew for worsening cognitive impairment, I am not sure those scales that they used would account for this.
(I haven't had a chance to review the different scales though.)
Also, no additional information is given about why people decided to not start CBT. Perhaps, despite the scales saying the people are smiliar on the baseline data, these folks, after hearing further about the trial after randomization, were concerned they could not tolerate it.
(From Consort)
Discontinuations and withdrawals due to adverse events are especially important because they reflect the ultimate decision of the participant and/or physician to discontinue treatment.
It is important to report participants who are nonadherent or lost to follow-up because their actions may reflect their inability to tolerate the intervention.
Passive surveillance of harms leads to fewer recorded adverse events than active surveillance.
4. In terms of using data from surveys, here's a supportive statement from CONSORT about that:
Authors should contrast the trial results on harms with other sources of information on harms, including observational data from spontaneous reporting, automated databases, case–control studies, and case reports.
(which is what surveys and patient anecdoates are! So don't let others discount the value of those surveys. They have their limits but they have a value too.)
5. They use the DOF and DOP to monitor people but how good are these scales? Sounds like something they used before but how well is it tested (validity, reliability, etc.)? I have to do some more reading on scales.
6. Also, in future papers, watch for how they handle missing data (people who drop out). In the Stule. and Knoop paper, they "carried forward the last observation." which means they assumed the person did not change in their rating for fatigue, etc.
This could be challenged since the person might have deteriorated instead.
In the Heins paper supposedly they corrected this with fancy statistical analysis but in the end, they did not show their data, which could also be challenged.
The data of the 3 RCT were pooled to increase statistical power.
Numbers were calculated on an intention-to-treat basis, and
missing values on the postmeasurement were replaced with estimates
derived from single imputation (missing variable analysis,
regression with baseline value as predictor) [32] . Missing data in
categorical variables were not replaced.