• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Data or Dogma?

Daisymay

Senior Member
Messages
754
http://www.meactionuk.org.uk/Data-or-Dogma.htm


Data or Dogma?

Margaret Williams 16th August 2014


A significant paper on the nature of ME/CFS was recently published: it concluded: “These data are consistent with evidence of multisystem dysregulation in CFS and implicate the involvement of DNA modifications in CFS pathology” (de Vega WC, Vernon SC, McGowan PO. DNA Methylation Modifications Associated with Chronic Fatigue Syndrome. PLoS ONE 9(8):e104757. doi:10.1371/journal.pone.0104757; published 11th August 2014, volume 9, Issue 8).


The accompanying press release states that the researchers found evidence of distinct epigenetic profiles in immune and other physiologically relevant genes, explaining that epigenetic modifications affect the way genes are turned on or off without changing the inherited gene sequences: “These types of changes would be expected to affect immune cell function in ME/CFS patients….it is further demonstration of the indisputable biological basis of ME/CFS” (Solve ME/CFS Initiative: Breaking News: Chemical Changes in Immune Cell DNA from ME/CFS Patients).

Immediately apparent is the significant discrepancy between this evidence and the selectively published results of the PACE trial, as the cohort was essentially the same age group as PACE participants (in this study, ME/CFS patients’ age averaged 41.1 years and controls averaged 39.7 years, compared with the average age of 38 years in the PACE trial).

Comparison of the physical function scores of this study with the PACE trial is illuminating: in this study, the mean physical function score for patients was 38.9 but for the healthy age-and-sex matched controls it was 95.0. The standard error of the mean (similar to a standard deviation) was 1.5, giving a normal range of greater than 93.5.

In contrast, the PACE trial Investigators did not use healthy age-and-sex matched controls and decreed that a physical function score of 60 or higher represented the normal range (which meant that it was possible for a PACE participant to enter the trial with a physical function score of 65, then deteriorate physically over the course of the trial, yet still be reported as having “recovered” with CBT and GET).
 

Snow Leopard

Hibernating
Messages
5,902
Location
South Australia
While some interesting points are made, I cannot help but think the whole article could be communicated a bit more effectively. Few except perhaps those already convinced of MW's point are going to read all 4000 words.

Rather than 4000 words, perhaps 400 words with some appendices for any additional background.

I'm saying this as I'd like to see a wider audience.
 

Sasha

Fine, thank you
Messages
17,863
Location
UK
I haven't read this particular paper yet but I think that there's a general tendency in authors who do all this hard work for us to document things in very great detail without providing a short overview that is likely to actually be read. People - especially PWME, busy doctors, and people who hold the opposing view aren't going to read anything long.

It's actually pretty hard and skilled work to produce something concise. It's an extra job once you've gathered all the info, but as you say, @Snow Leopard, it's important to do it if you want a wider audience.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
This problem with detail/brevity I hope to address in my book. Margaret Williams will look short compared to what I am hoping to do. However I hope to thematically structure it, then provide detailed summaries and introductions, and publish the summaries separately as a more readable text, from which someone can go to my book for more substantiating detail, topic by topic as it interests them. We need the capacity to see both the forest and the trees.

I am also considering an alternative structure entirely, but its all on hold for now.
 

A.B.

Senior Member
Messages
3,780
Comparison of the physical function scores of this study with the PACE trial is illuminating: in this study, the mean physical function score for patients was 38.9 but for the healthy age-and-sex matched controls it was 95.0. The standard error of the mean (similar to a standard deviation) was 1.5, giving a normal range of greater than 93.5.

Could someone explain this in plain English?
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Comparison of the physical function scores of this study with the PACE trial is illuminating: in this study, the mean physical function score for patients was 38.9 but for the healthy age-and-sex matched controls it was 95.0. The standard error of the mean (similar to a standard deviation) was 1.5, giving a normal range of greater than 93.5.
Could someone explain this in plain English?
I'm not a statistician, but I think that's an incorrect interpretation of the "standard error of the mean".
In simplistic terms, I think the "standard error of the mean" indicates how precisely the mean is known, so that as a sample size becomes larger, the "standard error of the mean" becomes lower, and the figure given for the mean is considered more reliable.
The "standard error of the mean" is not used to define normal ranges, as far as I understand.

A mean SF-36 physical function score of roughly 95 for healthy individuals is commonly reported.
The PACE trial used a mean score of 84 as a population norm, to define the normal range, and recovery, however their mean score included people with chronic illness and adults of all ages. Their method of defining a normal range was inappropriate for a number of reasons, as indicated by the fact that a participant could theoretically have a worse physical function score after treatment (with CBT/GET) and then be declared 'recovered'.
 
Last edited:

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
The net effect of the PACE claim for normal range is they are claiming that its normal for a twenty year old to have the functional capacity of an average (not especially well or fit) eighty year old. Who in their right mind would accept that claim? They are hiding behind the problem that most people do not look up the appropriate reference ranges, and behind an erroneous use of statistics.
 

MeSci

ME/CFS since 1995; activity level 6?
Messages
8,231
Location
Cornwall, UK
Could someone explain this in plain English?
Comparison of the physical function scores of this study with the PACE trial is illuminating: in this study, the mean physical function score for patients was 38.9 but for the healthy age-and-sex matched controls it was 95.0. The standard error of the mean (similar to a standard deviation) was 1.5, giving a normal range of greater than 93.5.

What I take from this last bit - which I don't think was very well-worded - is that the control patients all had a physical function of 93.5 or more. The standard error of 1.5, with a mean of 95.0, means that it ranged from 93.5-96.5 (1.5 lower and 1.5 higher than 95.0).

So if (any of) the patients scored less than 93.5, they were worse than any of the controls.

I think I got that right!
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
@MeSci, I think you aren't a statistician either? :)
I don't think your interpretation is right.
The standard error of the mean indicates the reliability of the mean.
It does not indicate the range of values that were used to calculate the mean.
 

MeSci

ME/CFS since 1995; activity level 6?
Messages
8,231
Location
Cornwall, UK
@MeSci, I think you aren't a statistician either? :)
I don't think your interpretation is right.
The standard error of the mean indicates the reliability of the mean.
It does not indicate the range of values that were used to calculate the mean.

Oh dear. I have admitted elsewhere that stats are my weak point! :redface:
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
Its probably easier to think of it as 95 plus or minus 1.5. Old school use of plus and minus. Stats are also a weak point for me - I understand some of the underlying principles, but I am definitely not statistically minded.
 

MeSci

ME/CFS since 1995; activity level 6?
Messages
8,231
Location
Cornwall, UK
Its probably easier to think of it as 95 plus or minus 1.5. Old school use of plus and minus. Stats are also a weak point for me - I understand some of the underlying principles, but I am definitely not statistically minded.

So I was right? :confused: Or are you saying that the mean is between 93.5 and 96.5 rather than the range of values is? My brain hurts...maths always does that to it. My brother got all the maths genes.

(I thought that reliability was indicated by the 'p='/'p<' value.)
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Or are you saying that the mean is between 93.5 and 96.5 rather than the range of values is?
Yes, that's what Alex means. I don't think it's a precise explanation, but it's a good way of understanding it.

My brain hurts...maths always does that to it. My brother got all the maths genes.
lol, I think my sister would sympathise with you there! :)

(I thought that reliability was indicated by the 'p='/'p<' value.)
"The P value gives the probability of any observed difference having happened by chance."

That's my best explanation. Over to someone else if you need a better explanation!

Look at the third result in this google search list (i.e. the PDF), if you'd like to see a very helpful, and easy-to-understand book re medical stats:
https://www.google.co.uk/search?q=medical statistics made easy harris taylor&sourceid=ie7&rls=com.microsoft:en-GB:{referrer:source}&ie=UTF-8&oe=&rlz=1I7GZAZ_en&gws_rd=ssl
 

MeSci

ME/CFS since 1995; activity level 6?
Messages
8,231
Location
Cornwall, UK
"The P value gives the probability of any observed difference having happened by chance."

Yes - that's what I understood about it. I did have to do stats for my degrees, but it took very hard work and brain exhaustion (I already had ME a year into the first degree). The maths was my least favourite part of my science studies as I find it so hard.

Thanks for the link, but I won't be able to retain the info. Brain says no! I'll leave it to you stats wizzes.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
I think that like many things in statistics, things like the plus minus explanation are approximations, as its involves probabilities. Its a calculated probability range, not a certainty.

Probability is not intuitive. In a recent test of people who used statistics every day, namely economists, 97% failed a basic grasp of probability in the real world. Our brains are really bad at intuitively grasping this stuff, which is why we have math. Yet the math has to be translated to and from the real world, and this is where it often goes horribly wrong.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
My understanding is also that the P value is not always accurate. Its, again, a calculated probability and uses assumptions that are not always valid. I forget nearly all the explanations for this kind of stuff, my brain does not retain it and I have to relearn it again and again. Grrr ....

Further, let us recall that not likely being due to chance does not rule out it being due to bias, fraud, etc. Chance is just one risk, so is bias and fraud and just bad methodology. With publication bias causing increased publication of positive findings, this is a huge one to consider and is external to any study. So it is not even reflected in the P value.
 

Leopardtail

Senior Member
Messages
1,151
Location
England
I haven't read this particular paper yet but I think that there's a general tendency in authors who do all this hard work for us to document things in very great detail without providing a short overview that is likely to actually be read. People - especially PWME, busy doctors, and people who hold the opposing view aren't going to read anything long.

It's actually pretty hard and skilled work to produce something concise. It's an extra job once you've gathered all the info, but as you say, @Snow Leopard, it's important to do it if you want a wider audience.
The 'Quick overview' is really the job of the abstract @Sasha - If the paper itself is too short or does not give convincing detail, it's nor really of much use to other academics and can neither be challenged, nor confirmed.
 

Sasha

Fine, thank you
Messages
17,863
Location
UK
The 'Quick overview' is really the job of the abstract @Sasha - If the paper itself is too short or does not give convincing detail, it's nor really of much use to other academics and can neither be challenged, nor confirmed.

You don't generally see abstracts outside of academic papers published in scientific journals (although you might see something similar referred to as an executive summary). I think that if we want to do a good job of persuading people, though, we've got to have some sort of concise overview.
 

Leopardtail

Senior Member
Messages
1,151
Location
England
I'm not a statistician, but I think that's an incorrect interpretation of the "standard error of the mean".
In simplistic terms, I think the "standard error of the mean" indicates how precisely the mean is known, so that as a sample size becomes larger, the "standard error of the mean" becomes lower, and the figure given for the mean is considered more reliable.
The "standard error of the mean" is not used to define normal ranges, as far as I understand.

A mean SF-36 physical function score of roughly 95 for healthy individuals is commonly reported.
The PACE trial used a mean score of 84 as a population norm, to define the normal range, and recovery, however their mean score included people with chronic illness and adults of all ages. Their method of defining a normal range was inappropriate for a number of reasons, as indicated by the fact that a participant could theoretically have a worse physical function score after treatment (with CBT/GET) and then be declared 'recovered'.
Basically the statistical term 'sample mean' is not 'the mean of the sample' but the mean of multiple variables each of which itself is a mean. Lets call those variable Member_Means [1..x]

Standard error comes in because each of those separate variables (Member_Mean [1..x] ) may not average the same number of samples. Standard Error is a standard deviation not of the values but of the number of samples in each value.

E.g. mean 1 = 25 [contains 5 samples]. mean 2 = 25 [contains 1 sample] mean 3 = 10 [contains 2 samples]
True mean is 21.25
Sample mean is 20

Standard Error is the standard deviation of the set 5,1,2 that being 1.7
It's a guess about how likely to the data is to be wrong when attempting to combine averages simply.

In this case it does look wrong..