1. Patients launch a $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
The Lipkin Microbiome Crowdfunding Campaign Launches!
An ambitious $1.27m international, patient-led fundraising campaign storms into action. Sasha invites you to join it!
Discuss the article on the Forums.

PACE Trial and PACE Trial Protocol

Discussion in 'Latest ME/CFS Research' started by Dolphin, May 12, 2010.

  1. anciendaze

    anciendaze Senior Member

    Messages:
    833
    Likes:
    780
    I've written elsewhere that distributions resulting from problems involving multiple interacting organ systems are very likely to be Lévy distributions, since efficiency of each system combines multiplicatively. A simple, obvious example is the combination of COPD and heart failure, and individuals with both problems function way below the mean, if they are still living.

    We don't really know if even the PACE authors believe the full distribution of SF-36 pf scores has a symmetrical right tail of people performing far above the mean. In the absence of such a tail the mean, mode and median values will not be the same, and common inferences about the meaning of "one-standard deviation below the mean" will be invalid. I suspect anyone familiar with psychology would know how readers who don't have the time to check all references and assumptions would interpret that criterion. Stated differently, I believe this continuing unresolved problem reflects a deliberate attempt to mislead.

    From an academic standpoint, the distribution from which samples are drawn need not be Gaussian provided the Central Limit Theorem (CLT) applies. This places a great deal of weight on the sampling process itself, plus some unstated assumptions. What we know in this case is that patients entering the trial were chosen according to subjective values applied by a small group of people. It is hard to find objective criteria for the illness definition in use. A small sprinkling of randomness will not correct inherent systematic bias.

    I think the common prerequisites for the CLT, independent identically-distributed random processes, are clearly not present. More sophisticated proofs of the CLT depend on a large number of different distributions, each possessing a mean and bounded variance. Analytic Lévy distributions do not have bounded variance. It seems quite likely they have mixed samples with organic disease and psychological problems. I do not consider two a large number. If the organic disease(s) involved combine multiplicatively to reduce performance, the main requirement for getting a Gaussian distribution out disappears entirely.

    We don't know how many times they applied sampling criteria before they got something out that looked like a set of normal distributions for each group. We do know they had so much trouble getting the numbers they wanted that they went back and revised the entry criteria. If changing these by 5 points did not invalidate results, then, a fortiori, changing a score by 5 points during the trial will not invalidate the null hypothesis that there was no benefit. This implies that modified recovery criteria should be more stringent, not less.

    Do we have any way to evaluate the number of patients considered recovered by no more than 5 points?

    In the absence of answers to the objections above, I believe we should consider the entire 5-million pound study an expression of unsubstantiated opinion.
    Valentijn likes this.
  2. user9876

    user9876 Senior Member

    Messages:
    684
    Likes:
    1,548
    Where there are multiple interacting systems I suspect observed distributions are actually the sum of a number of different (not necessarily independent) random processes and hence likely to be multi-modal and complex. Hence if you want to model them something like a Gaussian mixture model or a phase type distribution would seem appropriate. These are basically composite distributions either a sum of Gaussians or Exponential distributions that are then fitted to the data. In the case of phase type distributions they can be created using some form of Markov chain which can represent something of the structure of the problem.

    More generally there are issues around noise and error distributions on measurements. I've generally been a fan of robust statistics which tries to remove assumptions of normality in noise distributions. ftp://ess.r-project.org/Research-Reports/94.pdf
  3. anciendaze

    anciendaze Senior Member

    Messages:
    833
    Likes:
    780
    @user9876, the crucial difference in my hypothesis is that I believe measures like overall performance arise through multiplication of performance of individual organ systems. This is likely to occur in many examples of pathological situations. Too often medical statistics based on distributions of healthy people are applied to those who are clearly ill. While instrumental errors are more likely to be normally distributed, distributions connected to pathology are not.

    Remember that the classic normal biometric distribution of adult heights came from draft physicals of healthy adult males, and that independent variation in length of individual bones does combine additively to produce total height. (You don't have to look far to find statistical problems if you combine distributions for male and female heights, or apply these numbers to patients with osteoporosis.) I don't have to engage in special pleading to suggest that emphasis on problems limited to single organ systems will distort measures of health in such a way that some patients will necessarily "fall through the cracks" of the resulting system.

    In the case of ME/CFS, the problems I have currently identified are: dysautonomia, immune dysfunction, mitochondrial dysfunction, hypovolemia and reduced cardiac output. By themselves, each of these does not look terribly bad in most cases. The cumulative effects are something else. You can die while bouncing between specialists, and this is not peculiar to ME/CFS.

    Another aspect gets into subjective experience, though this is also the result of objective organic conditions which are hard to study prior to autopsy. Damage to dorsal root ganglia will produce sensory inputs which are apparent to patients, but completely invisible in terms of clinical signs. This overlaps with perception of pain and fatigue by MS patients, who have clear damage to efferent nerves that produces clinical signs visible to doctors. No ethical doctor will tell MS patients to simply push themselves past limits set by perceived effort.

    The problem for ME patients is that they have no way to distinguish boundaries between safe effort and effort which will result in a crash, before they wind up in the ER. Their nerves are always telling them something is wrong. Cognitive impairment is another indication which is generally disregarded by doctors. ME patients pay attention to these episodes as precursors of syncope, which might even involve them in falls or automobile accidents. I believe honest statistics about these potential problems will show that patients are not simply "catastrophizing". Too many have experienced such consequences.

    The assumption above of damage within the CNS is perhaps the most controversial. In addition to recent evidence of neuroinflammation, we have older results on the proteome of CFS patients cerebrospinal fluid. There are also high-resolution MRI studies using voxel-based morphometry which show progressive reductions in volume of structures within the brain as a function of time since onset.

    This evidence is considerably more impressive than repeated subjective studies by the same people based on an idée fixe about functional somatization disorders. Dodgy use of statistics dealing with primarily subjective measures does nothing to reduce suspicion of deliberate bias and obfuscation.
  4. anciendaze

    anciendaze Senior Member

    Messages:
    833
    Likes:
    780
    In the above post, perhaps I've leaned too hard on the subjective nature of PACE. To be even-handed I should give the authors their due. The most impressive objective measure of success for PACE is that they were given five million pounds to conduct the study.
    Last edited: Jun 30, 2014
    daisybell, MeSci, Valentijn and 2 others like this.
  5. Graham

    Graham Senior Moment

    Messages:
    776
    Likes:
    1,831
    Sussex, UK
    Imagine what they could have got for doing a proper job!
  6. alex3619

    alex3619 Senior Member

    Messages:
    6,662
    Likes:
    9,793
    Logan, Queensland, Australia
    @anciendaze, the way I put what you describe as a multiplication effect years ago is like this. Most diseases require a loss of substantive organ function before you see pathology. In the case of the heart it can be as much as 80% loss of function. Yet what if you have seven interdependent organs or systems with a loss of 20% function? The resulting function across the systems is then only 21%. I found it easier to explain using numbers.

    My point about non-Gaussian was primarily aimed at the general population, not patient response to treatment. Its about making comparisons and the use of SD to define SF-36 cutoff points in the general population. Patient SF-36 data distribution is probably much more like a Gaussian distribution, though primarily as an artifact of the study. Not too sick, not too well, just right for the study. The selection process creates artificial boundaries.
  7. anciendaze

    anciendaze Senior Member

    Messages:
    833
    Likes:
    780
    @alex3619 :

    My point about distributions within individual groups is that these could look like Gaussian distributions, if you did not have many data points, and none you had were extreme. The researchers certainly had the opportunity to exclude any patients they didn't want. We also don't know how many times researchers rolled the dice to get these sample distributions, but we do know they had trouble meeting original criteria.

    One aspect of working with distributions that lack bounded variance is that sample numbers and bounds completely determine sample variance. We know that these authors manipulated criteria to change bounds and increase numbers when they ran into trouble. A less obvious manipulation turns up in the criteria for adverse events. These were changed as well, making it harder to register an adverse response, and virtually impossible to register a serious adverse response if patients did not die. Taken together, these changes, and dropping objective measures promised in the proposal that got funding, strongly suggest the result was preordained, not the result of experiment.

    The discrepancy between the press release and actual numbers tells an interesting story all by itself. Professional psychologists and psychiatrists can hardly claim to be naive about the likely press interpretation of what they said.

    Incidentally, I'm not arguing against the exponential distributions people are talking about above. The tail of a Lévy distribution is essentially exponential. I'm going beyond raw data to suggest a mechanism based on combining distributions multiplicatively. I think the clue shows up when performance of an individual organ system goes to zero. No matter how well other organs perform total patient performance will be zero, the patient being dead.

    (I also need to caution people about how they may be combining deviations from the mean additively. I made the same mistake many years ago as a student, considering only extreme cases. The expected value of the variance of Gaussian distributions combines in a less dramatic way. )

    Your point about subjective criteria is well taken. I think PACE could become a case study in the number of ways you can present subjective choices as objective data.
    Last edited: Jul 2, 2014
    ukxmrv, alex3619 and Valentijn like this.
  8. anciendaze

    anciendaze Senior Member

    Messages:
    833
    Likes:
    780
    @Graham, I strongly suspect they did exactly what they were expected to do, show that the patients were to blame for their illness. This reduces demands for disability payments, and it has already been shown that most patients are highly averse to proposed therapies, which reduces visible costs in a second way. Arguing that you need government assistance because you are not competent to make your own decisions is a losing proposition when dealing with people who already feel they are competent to make decisions for everyone else.
    ukxmrv and Valentijn like this.
  9. anciendaze

    anciendaze Senior Member

    Messages:
    833
    Likes:
    780
    Just a note for people who wonder how a travesty like PACE can be accepted as "evidence-based medicine". The BBC has an article on medical doctors understanding of risks reported in statistics. Since doctors are very much herd animals, who are strongly influenced by peer pressure, the fact that the majority misunderstand statistics in simple situations should be a serious caution.

    Don't expect mere exposure to mathematics to change this. Consider what happened to major banks in 2008. This article mentions what Gigerizer found when he talked to people at Goldman-Sachs about risk management after the crash.
    Cheshire, Valentijn, alex3619 and 2 others like this.
  10. alex3619

    alex3619 Senior Member

    Messages:
    6,662
    Likes:
    9,793
    Logan, Queensland, Australia
    @anciendaze , nice article links on Gigerenzer. :) I think we should all read them.
    Last edited: Jul 7, 2014
  11. Tom Kindlon

    Tom Kindlon Senior Member

    Messages:
    234
    Likes:
    718
    Someone sent me this after reading my letters and thought I might find it interesting but wanted to stay anonymous if I was going to re-use it.
    -----

    [​IMG]
    Last edited: Jul 9, 2014
    MeSci, Sasha, Sean and 3 others like this.
  12. Dolphin

    Dolphin Senior Member

    Messages:
    6,470
    Likes:
    4,749
  13. alex3619

    alex3619 Senior Member

    Messages:
    6,662
    Likes:
    9,793
    Logan, Queensland, Australia
    If we can get a graph of the likely PACE scores, based on what we do know (inferred from the averages and SD) then we can do a composite indicative graph (not accurate of course) with different ranges superimposed.
    MeSci likes this.
  14. Dolphin

    Dolphin Senior Member

    Messages:
    6,470
    Likes:
    4,749
    MeSci likes this.
  15. Graham

    Graham Senior Moment

    Messages:
    776
    Likes:
    1,831
    Sussex, UK
    I'm sure that all of you in the UK have been overjoyed to see that our economy has now recovered. Which just goes to show that it isn't only psychiatrists who can define their own version of recovery.

    Mind you, the economy had been depressed.
    Last edited: Jul 25, 2014 at 6:29 PM
    MeSci and OverTheHills like this.
  16. Dolphin

    Dolphin Senior Member

    Messages:
    6,470
    Likes:
    4,749
    In a reply to another paper, Frank Twisk mentioned the PACE Trial
    http://www.rehab.research.va.gov/jour/2013/509/pdf/letterstotheeditor509.pdf

    Valentijn and Sean like this.

See more popular forum discussions.

Share This Page