Severe ME Day of Understanding and Remembrance: Aug. 8, 2017
Determined to paper the Internet with articles about ME, Jody Smith brings some additional focus to Severe Myalgic Encephalomyelitis Day of Understanding and Remembrance on Aug. 8, 2017 ...
Discuss the article on the Forums.

PACE Trial follow-up: Here's the table looking at the effects of having CBT or GET after 52 weeks

Discussion in 'General ME/CFS News' started by Dolphin, Oct 29, 2015.

  1. Dolphin

    Dolphin Senior Member

    Messages:
    10,670
    Likes:
    28,172
    I think there should be some focus on the table in the Appendix that actually looked at the effects of CBT and GET after treatment so thought I would give it it's own thread:

    Sharpe 2015 Table 3 Appendix.png

    The authors suggest that it is the CBT and GET after APT and specialist medical care alone that is the reason the differences between the groups disappeared. However the table doesn't bear this out.

    Indeed those that had 10 or more sessions of CBT and GET tended to have the lowest improvements of the three groups ((i)10+ sessions of CBT and GET post-trial; (ii) 1-9 sessions of CBT or GET and (iii) no sessions of CBT or GET post trial). (We're looking at the first two columns)

    (For the Chalder Fatigue Questionnaire (CFQ), the lower the score the better the result. For the SF-36 physical functioning subscale, it's the opposite: the higher the score the better. The main thing of interest is the change scores i.e. mean difference).
     
    Last edited: Oct 29, 2015
    WillowJ, lycaena, mango and 14 others like this.
  2. Valentijn

    Valentijn WE ARE KINA

    Messages:
    14,281
    Likes:
    45,811
    Also of interest in that table is that you can calculate the mean SF36-PF scores. CBT and GET patients had a score of 61.4 on average at the 2.5 year followup. That's roughly equivalent to the score of a 75 year old.

    "Sustained recovery," my ass :rofl:
     
    Last edited: Oct 30, 2015
    Justin30, WillowJ, justy and 16 others like this.
  3. elliepeabody

    elliepeabody

    Messages:
    86
    Likes:
    401
    East of England
    Thanks for posting this table. Struggling a bit to understand it - have a couple of questions. First this quote

    Am I right in thinking that the second sentence is actually inaccurate as they have measured the outcomes of those in the SMC and APT group who then went on to have CBT / GET? (the top two and the middle two rows of the table) compared to those who haven't. And the improvements were worse for the +10 sessions of CBT / GET.

    Sorry if this is a daft question but is the mean difference the average (i.e. all added up and divided by the number of participants as opposed to say median) difference of the participants SF36 and CFQ scores at the end of the trial and then at the end of the 52 week review?

    And the researchers are saying that there was more improvement in the CBT / GET groups than SMC and APT by the end of the trial, but at the end of the 30 months this improvement had gone, and this is because the SMC and APT groups went on to have sessions of CBT / GET? You are saying that those who had 10+ sessions of CBT / GET (deemed by the researchers as adequate treatment) actually had lower improvement levels than 1-9 of CBT / GET or no CBT / GET. The 1-9 sessions improvement level is a bit confusing though as it is higher (if I have read the tables correctly) than that for 10+ sessions and no sessions.

    I'm struggling with interpreting the mean difference scores in the context that you mention. In the second row of the table (SF36+10 or sessions of CBT / GET) the mean differences are
    APT 5.4
    SMC 5.2
    CBT 14.3
    GET 2.8

    Does that mean that in this group (+ 10 sessions CBT / GET) the most improvement was in the CBT arm , and the least in the GET arm with APT and SMC in the middle?

    And for the CFQ + 10 sessions CBT / GET the most improvement was CBT (-6.4), then APT, then GET, then SMC? Or have I got it the wrong way round?

    Then looking down the table, for the SMC and APT group the mean differences for SF36 and CFQ were
    + 10 sessions of CBT / GET: 5.4 and 5.2 (CFQ -2.7 & -3.6)
    1 - 9 sessions of CBT / GET: 11.6 and 11.3 (CFQ -5.4 & -1.6)
    No post trial CBT / GET: 5.8 and 8.5 (CFQ -3.9 & -3.5)

    Does this mean that the most improvement was in the 1-9 sessions of CBT / GET on the SF36 scale and on the CFQ this group had the most improvement -5.4 for SMC?

    Thanks but no worries if you don't want to plough through all those questions :eek:
     
    Last edited: Oct 30, 2015
    Esther12 likes this.
  4. elliepeabody

    elliepeabody

    Messages:
    86
    Likes:
    401
    East of England
    Is that the average across the 3 groups?
    Really brings it home when you compare with the expected PF scores.
    But wasn't this one of the things they changed along the way, initially comparing with working age population and then with the whole population?
    Do you have a link to SF - 36 PF scores relative to age? Would be handy to have a look at.
     
    MEMum likes this.
  5. alex3619

    alex3619 Senior Member

    Messages:
    12,479
    Likes:
    35,007
    Logan, Queensland, Australia
    I don't have a link handy, but when I made the comparison some years ago the recovery threshold was set at the capacity of an 80 year old. If that's recovery I don't want it. Try telling that to a twenty year old former athlete who managed to scrape a score of 60. Mind you, that is not a fit, energetic 80 year old. That is an 80 year old with all the commonly associated health problems, and more than likely to die in the next odd few years.
     
    MEMum, helen1, Dolphin and 5 others like this.
  6. Valentijn

    Valentijn WE ARE KINA

    Messages:
    14,281
    Likes:
    45,811
    Yup. The 244 original CBT/GET patients who had 10+, 1-9, and 0 additional sessions have a median SF36-PF score of 61.4 (14,987.6 / 244). I didn't bother calculating CFQ scores, because it's a useless scale which no one else uses.
    I think the issue was that they expanded the "recovered" range by including a standard deviation. So 85+ was recovered in the protocol, but they changed it to 60+ after they started seeing the data rolling in. But since SF36-PF scores are skewed, with nearly everyone in the 95-100 (maximum) range in the general public, standard deviations are meaningless and inappropriate to use with that data. One of the basic rules of statistics is that standard deviations should only be used with normally distributed data, which comes out looking like a bell curve when graphed.
    Table 3 of http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2654814/ which can be opened separately at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2654814/table/T3/ . It's based on data from the 1996 Health Survey for England.
     
  7. Snookum96

    Snookum96 Senior Member

    Messages:
    290
    Likes:
    552
    Ontario, Canada
    I don't remember a lot from stats courses but aren't the p-values for GET kinda high? I remember the minimum we would accept was 0.05 I think.
     
    Dolphin and elliepeabody like this.
  8. Valentijn

    Valentijn WE ARE KINA

    Messages:
    14,281
    Likes:
    45,811
    Yeah, that's why they aren't claiming those results are statistically significant. Though they are playing dumb about having the data, by speculating that the data might show something which it clearly does not.
     
    Dolphin, elliepeabody and Snookum96 like this.
  9. Snookum96

    Snookum96 Senior Member

    Messages:
    290
    Likes:
    552
    Ontario, Canada
    That's ridiculous. They know that most people don't really understand what statistically significant means? What kind of idiot publishes stats that aren't statistically significant.

    Douchebags.
     
    SOC and elliepeabody like this.
  10. Valentijn

    Valentijn WE ARE KINA

    Messages:
    14,281
    Likes:
    45,811
    It's normal (and good) to publish the insignificant data. But it's dishonest for them to pretend that the gains in the original APT and SMC patients at the 2.5 year followup might be due to them getting extra CBT or GET after the trial ended.

    Their own data definitively proves that post-trial CBT and GET are not even potentially responsible for the long-term gains seen in the original APT and SMC patients.
     
    SOC, Dolphin, elliepeabody and 4 others like this.
  11. Sidereal

    Sidereal Senior Member

    Messages:
    3,095
    Likes:
    17,172
    The accompanying editorial in Lancet Psychiatry says so but no one will read it except us. The general public will only see the newspaper headlines. The damage is done.
     
    WillowJ, Keela Too, Dolphin and 4 others like this.
  12. Sidereal

    Sidereal Senior Member

    Messages:
    3,095
    Likes:
    17,172
    Just to add that the participants in the Pace trial were majority women, mean age 38 years. Yet they're scoring--after "recovery" from CBT/GET--in the range of the average disease-ridden, soon to be deceased 80-year-old.
     
    WillowJ, MEMum, Daisymay and 5 others like this.
  13. wdb

    wdb Senior Member

    Messages:
    1,377
    Likes:
    2,956
    London
    Could someone double check these graphs I made from the table, I feel like I must have made a mistake. But if I haven't then it it very much goes against the suggestion then the SMC/APT groups improved due to addition post trial CBT/GET.


    mean.png


    mean-diff.png
     
    Sea, lycaena, SOC and 8 others like this.
  14. ScottTriGuy

    ScottTriGuy Stop the harm. Start the research and treatment.

    Messages:
    1,079
    Likes:
    4,352
    Toronto, Canada
    :lol::lol::lol::lol:

    X2
     
    elliepeabody and Snookum96 like this.
  15. user9876

    user9876 Senior Member

    Messages:
    2,583
    Likes:
    18,182
    moderated with many others
    I think quoting means where group numbers are small as in some here is dodgy as can easily be skewed by outliers.
     
    elliepeabody, Snookum96 and alex3619 like this.
  16. A.B.

    A.B. Senior Member

    Messages:
    3,751
    Likes:
    23,188
    One of the points raised by Coyne is that the follow up data is uninterpretable because it's no longer gathered in a randomized clinical trial.

    For example, we can see patients who were assigned to SMC but didn't do CBT and GET had lower fatigue scores than those who did, but we don't know what this means because things are no longer randomized.
     
    Last edited: Oct 30, 2015
    WillowJ, Keela Too, SOC and 4 others like this.
  17. Valentijn

    Valentijn WE ARE KINA

    Messages:
    14,281
    Likes:
    45,811
    I printed out that page from the supplement and checked it against your graph values. It all looks accurate. I thought I saw problems a couple times, but checked again and it was correct ... looking at the data just makes anyone go a bit cross-eyed I think, especially since the order is reversed :p
     
    Dolphin, elliepeabody and wdb like this.
  18. alex3619

    alex3619 Senior Member

    Messages:
    12,479
    Likes:
    35,007
    Logan, Queensland, Australia
    That is what is often accepted, but modern calculations show that, even without bias skewing the p values, a p value of 0.05 can reflect an error rate of around 30%.

    If there is systemic and entrenched bias, deep bias, then its possible to get really low p values, under 0.01. P values are an indicator, but cannot sort out highly biased studies, nor outright fraud.
     
    SOC, elliepeabody and Valentijn like this.
  19. Simon

    Simon

    Messages:
    1,919
    Likes:
    14,452
    Monmouth, UK
    Ha, @wdb, great minds think alike! But I've done it slightly differently. Please let me know if anyone spots any errors.

    I thought this might be easier to intepret with just the key figures of mean difference for the SMC & APT patients during follow-up, by number of sessions of CBT/GET they had

    upload_2015-10-30_11-51-43.png

    upload_2015-10-30_11-51-13.png

    added: obviously 1-9 sessions is best :); ok, it's just noise, and treatment or not made no difference.
     

    Attached Files:

    MEMum, Sea, lycaena and 8 others like this.
  20. Snookum96

    Snookum96 Senior Member

    Messages:
    290
    Likes:
    552
    Ontario, Canada
    Makes sense. I would think insignificant data would be more of a note at the end of the study or a brief sentence, rather than the part that makes headlines.
     
    WillowJ, MEMum and elliepeabody like this.

See more popular forum discussions.

Share This Page