1. Patients launch a $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
Hyperparathyroidism: An Often Overlooked Differential Diagnosis to ME/CFS
Andrew Gladman puts hyperparathyroidism under the microscope, exploring what the disease is, how it can mimic ME/CFS in presentation and how it is treated.
Discuss the article on the Forums.

PACE Trial and PACE Trial Protocol

Discussion in 'Latest ME/CFS Research' started by Dolphin, May 12, 2010.

  1. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    342
    UK
    PACE used wrong data to define 'normal' fatigue

    We've already shown how the PACE definition of 'normal' for SF36 function is wrong, but it's been harder to pin down the problems with the Fatigue defintion. I'm pretty sure there I've now found data showing PACE used the wrong fatigue data to set its norms, because their 'normative' sample was very sick. Since I failed to excite anyone with my previous post,
    I'm having one last go at explaining this:
    One point of explanation. The Cella 2010 paper that was used for PACE's definition of 18 as 'normal' used a subset of a sample of 15,000 patients published in 1994. However, we know that the mean for Cella subsample was 14.2, higher than the mean of the original sample of 13.8 (a statistically significant difference) so we can conclude that the Cella sample is at least as fatigued as the original sample, hence the figures above should apply to the Cella data too. My original post has the details.

    OK, I've done my best, if no one's interested I promise I won't mention it again.
  2. Graham

    Graham Senior Moment

    Messages:
    775
    Likes:
    1,824
    Sussex, UK
    I'm interested, oceanblue, honest! We knew about the inconsistencies in "normal", which is why we focused on the fatigue scale rather than the sf36 scale in our animated graphics. The fatigue scale is so utterly wrong in so many ways. But I didn't know about the Cella paper, so I hooked back on your original posting, and thanks for that: now I understand much more about it.

    To be honest, the whole fatigue thing was rubbish. But it is more effective and easy to use their results to prove how feeble the improvement was (or non-existent rather), than to try to convince them that they had done such a bad job. It would have been much harder if they had actually come up with more impressive figures.
  3. Esther12

    Esther12 Senior Member

    Messages:
    5,086
    Likes:
    4,844
    I'm interested too... I'm just struggling to keep up with all the good points you keep making. Thanks for highlighting what you think is particularly important... I'll try to make the time to really go over it when I'm feeling better.
  4. Sean

    Sean Senior Member

    Messages:
    1,152
    Likes:
    1,660
    I am following this as closely as I can for now, OB. Don't ever think these kind of posts are ignored or unappreciated. At the very least they provide useful background info and ideas.
  5. anciendaze

    anciendaze Senior Member

    Messages:
    816
    Likes:
    726
    You may well be right, oceanblue. My take on the whole business is that, after careful study: 1) we can't figure out what they mean by CFS; 2) we believe they used a very strange definition of normal; 3) we don't believe they were serious about getting objective data; 4) we don't believe the reported improvements in objective scores were clinically significant; 5) we can't tell if reported improvements were statistically significant because of the huge difference between the number intended to treat and the number actually used in reported measures; 6) we're not even sure the measures of significance were appropriate because the distributions appear to violate prerequisites for determination of necessary parameters. For a scientific paper these lapses are damning.
  6. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    342
    UK
    Thank you all for your kind replies :D. I realise my info would have been more useful in the 2 week Lancet-letter window but I only recently managed to get my head around the nicities of the fatigue studies.
  7. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    342
    UK
    Response bias? The graph, with bells & Whistles

    New version of the graph showing SF36 scores for PACE at 52 weeks if they had stayed in line with the original WSAS data (see 'WSAS' projected line).

    deluxe1.jpg

    Second version adds error bars nb these are standard errors, about half this size of error bars for 95% confidence limits. I'm not entirely sure how to intepret these as we are looking at the trend across a series of data, not comparing two different points. As its a scatter plot there are 2 sets of error bars per data point: sf36 error and 6mwt error.

    deluxe2.jpg

    don't know why these images are so badly mangled when I upload them; the much clearer originals are availabe on request

    And I won't be the least bit offended if no one replies to this post...
  8. Bob

    Bob

    Messages:
    7,423
    Likes:
    8,550
    England, UK
    Unrecorded drop outs from the PACE Trial...

    Here's some text from Action for ME's recent edition of Interaction Magazine...

    I can't quite get my head around the wording of the text taken from the PACE Trial paper, so I can't confirm if the interpretation is correct.

    Had any of us picked up on this previously? I'd completely missed this bit of the paper.

    It's written by Prof Derek Pheby.


    --------------------------------------------------------------------------------


    Interaction magazine - Page 5 - top right hand corner...

    The analysis was ostensibly on the basis of treatment intention. This is the correct way to analyse the results of a randomised controlled trial, because if the analysis only involves those subjects who complete the trial, it disregards any participants who drop out because of adverse effects, and may therefore give an unjustifiably favourable impression of the treatment under examination.

    The authors state, though: "We excluded participants from the intention-to-treat population for whom we had no primary outcome data in the final analysis."

    In other words, this was not an intention-to-treat analysis at all, since only data regarding participants who stayed the course long enough for outcomes to be assessed were included in it.


    Here's the relevant quote from the PACE Trial paper:

    We excluded participants from the intention-to-treat
    population for whom we had no primary outcome data
    in the final analysis, which used restricted maximum
    likelihood. The per-protocol analysis excluded
    participants who were ineligible after randomisation,
    treated at a second centre, or did not received adequate
    treatment, adjusting for actual stratification factors.
    Statistical analyses were done with Stata version 10,
    SAS version 9.1, and SPSS version 18.
  9. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    342
    UK
    Thought this had been mentioned briefly, but Pheby is right. If they were going to do an intention to treat analysis they should at least have used a 'last observation carried forward' approach, though even that would be too conservative if people had dropped out due to relapse. That said, using last-observation-carried-forward would probably only affect the results (including the difference with SMC) by 5% and I'm not sure that's so important.

    I'm not sure why they couldn't get primary data on 5%: it's only 2 short questionnaires to complete by post or phone, and a carer could presumably do this if someone was in a really bad way.

    Disappointing that Derek Pheby didn't mention the PACE self-report gains were not matched by gains in objective measures.
  10. Bob

    Bob

    Messages:
    7,423
    Likes:
    8,550
    England, UK
    Ah, right, I was getting confused again! Thanks for that ocean.

    I thought that Pheby was referring to participants who were not recorded in any of the data in the paper.

    Yes, the paper did record the number of drop outs didn't it...
    When I looked at those figures, I seem to remember agreeing with you, that they wouldn't have made much difference to the outcomes.
  11. Dolphin

    Dolphin Senior Member

    Messages:
    6,432
    Likes:
    4,629
    Some data on 6 min walk vs SF-36 PF that could be plotted

    I just checked, Jason et al. (2007) does have 6 minute walk test data along with SF-36 PF data. It would be interesting to plot the points on a graph (and ideally make a regression line/make an approximate regression line visually) and see how it compares to the baseline in the PACE trial.


    from:
  12. Sean

    Sean Senior Member

    Messages:
    1,152
    Likes:
    1,660
    Nice consistency between the objective and subjective results. Exercise clearly didn't work. CBT did no better overall than pacing. And the SD for CBT outcome on the 6mwt, is more than thrice the therapeutic gain.

    Not much joy there for PACE lovers.
  13. Dolphin

    Dolphin Senior Member

    Messages:
    6,432
    Likes:
    4,629
    Thanks oceanblue.

    Regarding the slide from this paper:
    some of us dissected this graph in this thread (started a few posts in): http://forums.phoenixrising.me/show...a-should-not-be-exclusion-criteria-for-ME-CFS

    Basically, the GHQ-12 measures various symptoms incl. some physical symptoms so describing it as a measure of psychological distress is somewhat misleading. The same problem of confounding is probably related to a lot of the points made.

    ETA: Oceanblue has corrected me that I was thinking of a similar but different study. His point seems stronger: http://forums.phoenixrising.me/show...s-from-the-90s-that-keep-cropping-up&p=179419

    The Diabetes type 1 CBT/motivational interventional is not really analagous to ME/CFS: with diabetes, it is very clear what are the sort of behaviours to encourage; it is much less clear cut in ME/CFS.
  14. Dolphin

    Dolphin Senior Member

    Messages:
    6,432
    Likes:
    4,629
    I hadn't thought of that. Good point.

    Well spotted.
  15. Dolphin

    Dolphin Senior Member

    Messages:
    6,432
    Likes:
    4,629
    Where did you get the figure of 13.8 from?[/QUOTE]Oceanblue explained this (wasn't in the paper - he did some calculations) and I'm convinced the figure for Cella is higher than the Pawlikowska figure.
  16. Snow Leopard

    Snow Leopard Senior Member

    Messages:
    2,203
    Likes:
    1,534
    Australia
    I didn't bother with the audio, but I was concerned with the following:
    This hardly provides confidence in the medical model. It is also a non sequitur to then blame it on mental health. The biomolecular pathways between cognition, stress, behaviour etc and physical symptoms have not been explained at all. Basically all we have is two competing opinions (mental vs physical). The fact is that it is not scientific to decide between them until the underlying science is known.
  17. Dolphin

    Dolphin Senior Member

    Messages:
    6,432
    Likes:
    4,629
    That's only within each item. The total score is the relevant statistic in the PACE Trial.

    In Pawlikowska et al. (1994) it says:
    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2539651/pdf/bmj00432-0041.pdf
  18. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    342
    UK
    1. They quoted figures for item 5 and 9 but I'm pretty sure they are referring to the whole data as not normally distributed (eg mean skewness figure).
    2. It's the Cella data that's most relevant to the PACE trial, and that doesn't seem to be normally distributed. And did you notice that the Pawlikowska claim of normal distribution were not backed up by any data eg skewness or Shapiro-Wilk test? The graph of the data certainly looks non-normal, with a positive skew and median and mode higher than the mean. I'd put money on that not being a normal distribution.
  19. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    342
    UK
    I've started a new thread on the Wessely papers from the 90s, and have replied there re the graph:
    Could you explain a bit more about diabetes and behaviour?
  20. Dolphin

    Dolphin Senior Member

    Messages:
    6,432
    Likes:
    4,629
    This isn't important I think but anyway:
    1. I'm pretty sure they are referring to the individual items:
    2. As I think you know, they are mainly assuming normality here to work out percentiles (as far as I can see), so one is more interested in what percentiles are chosen by the figures.

    I think a problem looking at the graph is 11 & 12 are combined. 11 is for "fatigue the same as usual" (11 * 1). I would think it would have a lot higher frequency. Indeed if one looks at Table 3, in Cella & Chalder, one can see that item 11 represents 22.9% of all the scores (.866 -.637) for people who don't have CFS, while only 7.8% scored 12.

    We can also that 16.7% of the non-CFS cases scored 19 or more (that may have been pointed out before).

    Note: one thing the central limit theorem tells one is that if one adds together sufficient numbers of skewed distributions (that are the same), one gets a normal distribution. I think this also extends under certain conditions to skewed distributions that are not the same distribution (which is what causes so many things in nature to be normally distributed as far as I know). What sort of graph one would get adding together 11 graphs of equal weight with mean skewness of 0.77, I'm not sure.
    ETA: Normally the central limit theorem is expressed in terms of the distribution of the mean of the items. However, the distribution of the mean will have exactly the same distribution as the total score - it's just a scalar multiple of it.

    I don't know too much about skewness. Given the range of scores is only 0,1,2,3, perhaps one can get a good idea of the relative frequencies of the four scores using the mean skewness figure of 0.77?

See more popular forum discussions.

Share This Page