1. Patients launch $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
New Exercise Study Brings Both Illumination and Questions
Simon McGrath looks at new objective evidence of abnormal response to exercise in ME/CFS patients, and the questions that researchers are still trying to answer ...
Discuss the article on the Forums.

'Recovery' from chronic fatigue syndrome after treatments given in the PACE trial

Discussion in 'Latest ME/CFS Research' started by Sam Carter, Jan 31, 2013.

  1. Dolphin

    Dolphin Senior Member

    Messages:
    6,718
    Likes:
    5,565
    If anyone is not going to write a letter but has references that make this point, feel free to share them on the thread or to me personally. Thanks.
  2. Esther12

    Esther12 Senior Member

    Messages:
    5,268
    Likes:
    5,478
    I think that points 1 and 2 could be turned in to something, but without expansion would be better left out.

    re point 3: There are problems with the language there. I had a quick go at a re-write for you (assuming English is not your first language, it was a valiant effort! - if English is your first language, then I've just been rather rude)



    Firestormm:

    I don't think much of substance was said in the HofL debate.

    Point 1 - as you say is relevant given how results were sold and presented in the media (although I agree that these sorts of problems are not really surprising in trials).

    Point 2 - as you say, it's pretty pathetic, especially if these dramatic changes are combined with a refusal to release data in the manner laid out in their protocol as supplementary information. Just because those speaking in the House of Lords do not think it's fair for patients to want access to the results from a publicly funded trial in the manner laid out in the trial's protocol, and should be happy with however it is the researchers choose to present their results to us, doesn't mean that there's any reason to think that they are right.
  3. Marco

    Marco Old blackguard

    Messages:
    1,291
    Likes:
    975
    Near Cognac, France
    Simon likes this.
  4. Gijs

    Gijs Senior Member

    Messages:
    122
    Likes:
    121
    ------------------------------------------------------------------------------------------------------------------------------------
    Tabel 3
    age 16-24 25-34 35-44 45-54 55-64 65-74 75-84 85+
    N for each group n = 204 n =415 n =319 n =297 n = 297 n=281 n=296 n=36

    Mean (SD), 95.5 (12.1) 94.5 (13.5) 93.3 (13.4) 87.2 (20.9) 78.0 (26.3) 72.7 (26.7) 57.9 (28.6) 39.3 (31.5)
    physical function
    ---------------------------------------------------------------------------------------------------------------------------------
    Male 86.3 (22.5) n=925
    Female 81.8 (25.7) n=1117
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Bowling et. al., 1999

    Standard for recovery: ’The average age of the participants in the study of White et al, mean (SD) 38 (12), 77% of them were female. The mean (SD) scores for a demographically representative English adult population were 86.3 (22.5) for males and 81.8 (25.7) for females (Bowling et al 1999) . We derived a mean (SD) score of 84 (24) for the whole sample, giving a normal range of 60 or above for physical functioning''.

    The average age (38) in the studied group of White et.al. falls between 26 and 50 years. The standard to determine recovery should consider the average score in this group of age not the average general standard stated by White et.al., because the average age of this population falls between 16 and 85+ years table 3. The standard for normal physical function in this age group will be substantially lower because an older group of age steal a proper standard. The standard of White et. al. therefore is not representative for participants with an age between 26 to 50 years.

    ------------------------------------------------
    Example on data table 3 *
    -------------------------------------------------
    age normal range or above
    ------------------------------------------------
    (25-34) 81
    (35-44) 79.9
    (45-54) 66.3
    ------------------------------------------------
    Average score in the age from 25 and 54 years = 75.7

    *This standard is statistically not accurate because you need individual data but it is an indication to show that the standard for recovery stated by White et.al., is not correct without taking age by groups in to account.

    The standard giving a normal range need to be adjusted.
  5. Graham

    Graham Senior Moment

    Messages:
    829
    Likes:
    2,115
    Sussex, UK
    Sorry to harp on, but you are still using the mean and variance. Are you sure that the data is Normally distributed? I suspect it is strongly skewed. If we calculate the mean and variance in income of adults in the UK in 2007/8 we get £26800 and £29500, which gives us a "normal range" of income of between £56300 and minus £2700. In contrast, if we use the median (the middle value) as an average, it is a more sensible £18500, and the quartiles (i.e. the middle half of the earners) get between £11800 and £29500. I did the calculations for the Chalder Fatigue scale, and found that the data there had the same coefficient of skew as the incomes. Have you tried the median and percentiles to estimate normal range? It may be quite different.

    Recently both Simon and I followed a statistics course with a slant on psychology, and before any calculations were made, it was emphasised that the data had to be checked to make sure that it was "well-behaved". It still is common to use means and variances, but that doesn't make it right: but I have seen several studies using the median and percentiles.
    Dolphin likes this.
  6. Gijs

    Gijs Senior Member

    Messages:
    122
    Likes:
    121
    No problem Graham. If you want do good statistics you need all data of individuals. But it is clear that the standard white et. al. used is not representative for his population by age and sexe. His population is relatively young and that substantially influence the outcomes. It is not comparable.
    peggy-sue, Roy S and Dolphin like this.
  7. user9876

    user9876 Senior Member

    Messages:
    759
    Likes:
    1,825
    If you look at what data is available you will notice that the worse the scores the larger the standard deviation. I could explain this in terms of heathy people being described as a random variable hence forming some kind of distribution but the many health problems both with long term health issues and aging as being fairly randomly (and independantly distribued).

    Looking at the distibutions from Bowling's paper the basic graph looks a bit like an exponential distribution in shape (although not necessarily in spread) along with some noise created by the addition of people who are 65+, those with long term health problems and those with short term health problems (ill in last 2 weeks, visited hospital in previous year).

    Including people non healthy people thus has two effects firstly it reduces the mean and secondly it leads to considerable increases in the standard deviation.

    Looking at the Bowling paper
    22% had long term health problems (mean 52 std 29)
    16% had accute health problems
    11% were 75+
    13% were between 65 and 74

    Note that the standard deviations increase considerably with age as the means decline. My explanation would be that people are having more health problems and hence the distribution is affected by increasing numbers of people with health problems.

    The important to note that the inclusion of people with health problems in the data decreases the mean and increases the standard deviation hence giving a very wide range for normal.

    Graham is of course right in that it is wrong to characterise the data using the standard deviation and the mean is not a robust measure if we consider the distribution to be multimodal. Hence we should use the median, percentiles (or the median absolute deviation)

    http://www.archive.official-documents.co.uk/document/doh/survey96/tab5-18.htm

    Give some median and percentile figures but they note the numbers with health problems rises with aging.
    http://www.archive.official-documents.co.uk/document/doh/survey96/tab5-3.htm
    Hence the sf36-pf scores decrease.

    Even including these issues they have a 25 percentile of 75. That is only 25% of the population fall below the score of 75. When looking at the 35-44 age group (given average age in PACE of 38) the 25 percentile is 90.

    To demonstrate issues with the mean for the 35-44 age group there is a median of 95 and mean of 89 (25th percentile 90). But as the health of people declines the median moves more towards the mean (as would be expected). So the mean for 75+ is 52 with a median of 55. For the overall population the mean is 81 compared to the median of 95.
    Dolphin and Simon like this.
  8. user9876

    user9876 Senior Member

    Messages:
    759
    Likes:
    1,825
    From offical EU Guidance on changing trial protocols

    http://www.archive.official-documents.co.uk/document/doh/survey96/tab5-3.htm

    Talks about the legal needs to notify the ethics commitee and national bodies for drug trials wanting to make substantial protocol changes (Page 12 onwards) with examples of substantial (page 14,15) including:


    Also page 23 section 5.1 of the european medical agencies statistical principles for clinical trials

    http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500002928.pdf

    talks about prespecification of the analysis


    There is
    Dolphin likes this.
  9. Graham

    Graham Senior Moment

    Messages:
    829
    Likes:
    2,115
    Sussex, UK
    You are entirely right Gijs. White et al were utterly wrong to do what they did.

    Actually I would argue that to include healthy and ill people's scores in one distribution is adding together two very different types. It would be like adding adult males' heights to those of children of various ages. You generally get a Normal (or Gaussian) distribution when there are a myriad of small effects all combining (as in intelligence test scores), but I would argue that there isn't much of a distribution at all for healthy people, because the tests are not designed for that. Each illness would have its own distribution which may not at all be Normal.

    I don't think any of this "normative" sets of data have a single distribution. The only way to set targets or levels is to actually look at the scores of healthy people in the right age range and make an educated and informed decision. Statistics is a tool, not a compulsion.

    Speech over (sorry, but it's the maths teacher in me, Gijs!)
    user9876 likes this.
  10. biophile

    biophile Places I'd rather be.

    Messages:
    1,387
    Likes:
    4,586
    Tossing in some more physical function data from a general population:

    http://www.ncbi.nlm.nih.gov/pubmed/9817123

    http://share.eldoc.ub.rug.nl/FILES/root2/1998/Tranvaann/Aaronson_1998_Journal_of_Clinical_Epidem.pdf

    Mean (SD) [range]
    Age: 43.1 (18.1) [16–97].
    General population PF: 85.2 (23.1)
    [0.7% floor ie 0 and 45.0% ceiling ie 100]

    Chronic health conditions: 56%.

    No chronic health condition (all ages): 94.1 (13.2).

    1 chronic health condition: 88.5 (18.8).

    >1 chronic health condition: 70.0 (28.3).

    Migraine attack past 2 weeks PF: 84.0 (21.3).

    Metastatic stage of cancer PF: 59.2 (26.3).
    Simon and Dolphin like this.
  11. biophile

    biophile Places I'd rather be.

    Messages:
    1,387
    Likes:
    4,586
    The several CFS papers I have come across which report PF scores for healthy age-matched controls report mean(SD) scores of around 96(4). Admittedly the sample sizes were much smaller too.
  12. Dolphin

    Dolphin Senior Member

    Messages:
    6,718
    Likes:
    5,565
    While I can see why Graham criticised it, it can still be useful to show that that using the PACE Trial investigators own methods (mean - 1 SD), the threshold could be different if the the range of ages was restricted, as Gijs did.
  13. kaffiend

    kaffiend Senior Member

    Messages:
    166
    Likes:
    122
    California
    Pasted below is nice article (with references) on the validity of fatigue scales used in CFS. The article recommends using the Fatigue Severity Scale - at a glance it would appear to be very sensitive to CFS type fatigue.

    http://www.cfids.org/archives/2002rr/2002-rr4-article02.asp

    Is there any way to start a wiki-type project to pool the main criticisms of this paper? A lot of important points are being brought up in this thread.
  14. Dolphin

    Dolphin Senior Member

    Messages:
    6,718
    Likes:
    5,565
    At least a few of us hope to/plan to submit letters. We would then intend to collate letters that are published as well as those that are not published, the way we did with the Lancet PACE Trial paper:

    PACE Trial - letters that were published and authors' response (and editorial)
    http://forums.phoenixrising.me/show...ublished-and-authors-response-(and-editorial)

    PACE Trial - letters that were not accepted by the Lancet:
    http://forums.phoenixrising.me/show...-letters-that-were-not-accepted-by-the-Lancet

    A wiki would be good, but my preference is, for the moment, that people would focus on submitting letters. Having a complaint/criticism published in a journal gives it more status.
    ukxmrv and Simon like this.
  15. kaffiend

    kaffiend Senior Member

    Messages:
    166
    Likes:
    122
    California
    Has anyone identified a clear case in which the change to the Chalder Fatigue Questionnaire (CFQ) from binary scoring to Likert scale had an effect on meeting case definition criteria? Is it possible to score worse at 52 weeks and be classed as recovered? The study has a heavy reliance on this tool.
  16. Dolphin

    Dolphin Senior Member

    Messages:
    6,718
    Likes:
    5,565
    Regarding the latter, I gave an example here:
    http://forums.phoenixrising.me/inde...n-in-the-pace-trial.21628/page-11#post-332889

    First number is baseline, second is at 52 weeks:
    CFQ (Likert): 17 --> 18 (i.e. a deterioration) e.g. 22222211111-->33222111111 but now within normal range
    CFQ (bimodal): 6 --> 5 e.g. scores in last line translates to 11111100000 --> 11111000000
  17. Graham

    Graham Senior Moment

    Messages:
    829
    Likes:
    2,115
    Sussex, UK
    Too true Dolphin. I never meant it as a criticism: it was more an attempt to add to its faults. I'm not too good at differentiating between what I mean and what I actually say!

    Kaffiend, you may find our table on the Chalder Fatigue Scale useful.

    We also carried out a survey on Chalder Fatigue Scores.

    http://evaluatingpace.phoenixrising.me/aps6furtherdetai.html

    and

    http://evaluatingpace.phoenixrising.me/aps6survey.html
    Dolphin likes this.
  18. Dolphin

    Dolphin Senior Member

    Messages:
    6,718
    Likes:
    5,565
    ME Association response to PACE trial recovery paper | 15 February 2013
    by Dr. Charles Shepherd:
    http://www.meassociation.org.uk/?p=14460

    Useful letter, has a chance of being published I would have thought.

    Note: I wouldn't recommend making your letter public until the letters are published in the journal (or you are told it is not going in) but perhaps the ME Association can get away with it.
    Simon, Firestormm and Valentijn like this.
  19. Firestormm

    Firestormm Guest

    Messages:
    5,824
    Likes:
    5,975
    Cornwall England
    Dolphin, Does the journal that published the paper routinely publish letters do you know? I know that some are more amenable that others to do so - and on-line. I presume there is a process and not all submitted letters 'make the grade'. Hopefully this one will and will prompt a response. Am guessing that authors are obliged to respond?
  20. Dolphin

    Dolphin Senior Member

    Messages:
    6,718
    Likes:
    5,565
    One can get free access to Cambridge journals published in 2012 incl. this journal to see what they published. They published plenty of letters - more than some journals, but less than others.

    I don't think any journal publishes all letters. Some journals allow e-letters/rapid responses/online comments which can be a much lower standard and still go up on the site.

    Charles' letter hasn't covered all the points that could be made so I hope more letters will be submitted. I would hope they would publish more than one, given it's a £5m trial, but who knows for definite. We can collate the ones that aren't published which would be a good way of summarising points. I would hope to highlight a list of submitted letters so they got some attention.
    Firestormm likes this.

See more popular forum discussions.

Share This Page