• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Data or Dogma?

Leopardtail

Senior Member
Messages
1,151
Location
England
So I was right? :confused: Or are you saying that the mean is between 93.5 and 96.5 rather than the range of values is? My brain hurts...maths always does that to it. My brother got all the maths genes.

(I thought that reliability was indicated by the 'p='/'p<' value.)
What they are saying (I think in the new paper) is "we have taken some short cuts working the mean out" and "allowing for any error that our short cut introduced the mean may be between 93.4 and 96.5". So yes your 'gut hunch' was correct.

By contrast the p value means 'the probability that our conclusion might be wrong is'. The estimation of p values is something that often gets criticised when research is published since authors tend to treat themselves too kindly.
It's supposed to allow for things like the reliability of blood tests results (e.g. the result of a single blood test being accurate to +- 10%). It's a way of bringing together all the experimental error and giving an indication of how reliable the source data is collectively, the effect of any missing data and so on....

E.g. Prolactin and TSH are both stimulated by TRH. TRH cannot be measured accuruately without great expense. Low levels of Testosterone, or high levels of Dopamine can suppress TSH production.

So if you are gauging hypothyroidism based upon TSH without measuring the factors that would cause suppression there is some doubt as to your conclusion. The p value in this scenario would give your estimate of how often something might suppress TSH that has not been measured combined with an estimate of how reliable the TSH measurement is itself.
E.g.
If TSH is measured to an accuracy of +- 5% and within your sample set 5% have suppression then your p value is then the change your conclusion is right is 95%* 95% = 90%. Your p value 0.1.

In effect the p value deals with missing data, or alternative conclusions from the same data.

Where standard error deals with lazy maths when averages of averages produce possible error. It's only really excusable when the sample size is too large to reliably compute in one go (such as an average of millions of large numbers too large for a computer to handle).

Hope that clarifies it, not exactly easy to explain... with ME brain.
 
Last edited:

Leopardtail

Senior Member
Messages
1,151
Location
England
You don't generally see abstracts outside of academic papers published in scientific journals (although you might see something similar referred to as an executive summary). I think that if we want to do a good job of persuading people, though, we've got to have some sort of concise overview.
With you now, did not originally realise you were dealing with the response rather than the paper.... You mean a brief paragraph is needed at the start such as 'What this paper shows is that the SFC figure for wellness should be 95 not the 60 used in the PACE trial and a such the PACE trial has falsely reported recovery.'
?????
 

Leopardtail

Senior Member
Messages
1,151
Location
England
My understanding is also that the P value is not always accurate. Its, again, a calculated probability and uses assumptions that are not always valid. I forget nearly all the explanations for this kind of stuff, my brain does not retain it and I have to relearn it again and again. Grrr ....

Further, let us recall that not likely being due to chance does not rule out it being due to bias, fraud, etc. Chance is just one risk, so is bias and fraud and just bad methodology. With publication bias causing increased publication of positive findings, this is a huge one to consider and is external to any study. So it is not even reflected in the P value.
Too true.... researchers do tend to want to think they must be right!! But also it's hellishly difficult to predict all the factors that may influence the reliability of the data, and sometimes that information needed to calculate that risks is not available. E.g. there is no reliable large data set that would allow me to gauge the frequency with which boderline hypothyroid patients have high dopamine (see other example for relevance).
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
Its also very much an issue that large data sets do not take into account rare phenomena. In other words, outliers are ignored, or not captured due to rarity, and yet to an individual who has an outlier issue its very much a real problem. Its also the case that statistical methods don't work very well or are invalid for most non-Guassian data, though I think there are caveats to that which I have not investigated yet.
 

Leopardtail

Senior Member
Messages
1,151
Location
England
Its also very much an issue that large data sets do not take into account rare phenomena. In other words, outliers are ignored, or not captured due to rarity, and yet to an individual who has an outlier issue its very much a real problem. Its also the case that statistical methods don't work very well or are invalid for most non-Guassian data, though I think there are caveats to that which I have not investigated yet.
True though I suspect that would be too large a shift in medical culture, medics work on probability.....
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
... and probability on certain problems is a nonsense method. GIGO. Its why so much of economics is stuffed. Its why so many do not predict stock market crashes. The 2007 crash occurred when, according to one expert, a once in 10,000 year random event occurred. The next day, I think, another one occurred. Then another. Then his company went bust. In the PACE trial they inappropriately used statistical methods to get a normal range, and that range is mathematically undefined. People doing statistical analysis on complicated data domains need to be in contact with professional mathematicians.

This is further complicated when professionals who use probability fail testing on basic probability, including doctors and economists, with fail rates of between 80 and 97% That's nearly all of them.
 

Sasha

Fine, thank you
Messages
17,863
Location
UK
With you now, did not originally realise you were dealing with the response rather than the paper.... You mean a brief paragraph is needed at the start such as 'What this paper shows is that the SFC figure for wellness should be 95 not the 60 used in the PACE trial and a such the PACE trial has falsely reported recovery.'
?????

Sorry, I seem to have been spectacularly unclear! I was just making a general point - not just about this 4,000 paper (a response to PACE which I still haven't even looked at) but any lengthy piece of advocacy (including scientific advocacy) - that it's no good expecting people to read the whole tome and that authors should provide a short summary/abstract at the beginning. 250 words, maybe; or for a major document, something a bit longer with bulleted points. Something that will remove the burden of work from the reader.
 

MeSci

ME/CFS since 1995; activity level 6?
Messages
8,231
Location
Cornwall, UK
Sorry, I seem to have been spectacularly unclear! I was just making a general point - not just about this 4,000 paper (a response to PACE which I still haven't even looked at) but any lengthy piece of advocacy (including scientific advocacy) - that it's no good expecting people to read the whole tome and that authors should provide a short summary/abstract at the beginning. 250 words, maybe; or for a major document, something a bit longer with bulleted points. Something that will remove the burden of work from the reader.

I understood, so I don't think you were being unclear. :)
 

Jonathan Edwards

"Gibberish"
Messages
5,256
What I take from this last bit - which I don't think was very well-worded - is that the control patients all had a physical function of 93.5 or more. The standard error of 1.5, with a mean of 95.0, means that it ranged from 93.5-96.5 (1.5 lower and 1.5 higher than 95.0).

So if (any of) the patients scored less than 93.5, they were worse than any of the controls.

I think I got that right!

The standard error is a measure used by scientists very widely to create their 'error bars' on data plots. It is sneaky because it looks very small when the uncertainty is much bigger. In this case the normal range would be two standard deviations from 95.0. If I remember rightly standard error is one standard deviation divided by a number related to sample size (I think the square root). As far as I know it is always smaller than one standard deviation so looks rather nice on the error bars. So if standard error here is 1.5, the standard deviation might have been 6 (for a sample of 16 cases) and two standard deviations 12 so we are really looking at a normal range of 83-107. (If 100 is the maximum score then this is not a Gaussian situation so will be different but it doesn't really matter.) That would still make the idea that 60 was a recovery ridiculous. The author's argument is basically sensible. The maths are not quite right but it is very likely that the message is the same with the right maths!
 

Leopardtail

Senior Member
Messages
1,151
Location
England
The standard error is a measure used by scientists very widely to create their 'error bars' on data plots. It is sneaky because it looks very small when the uncertainty is much bigger. In this case the normal range would be two standard deviations from 95.0. If I remember rightly standard error is one standard deviation divided by a number related to sample size (I think the square root). As far as I know it is always smaller than one standard deviation so looks rather nice on the error bars. So if standard error here is 1.5, the standard deviation might have been 6 (for a sample of 16 cases) and two standard deviations 12 so we are really looking at a normal range of 83-107. (If 100 is the maximum score then this is not a Gaussian situation so will be different but it doesn't really matter.) That would still make the idea that 60 was a recovery ridiculous. The author's argument is basically sensible. The maths are not quite right but it is very likely that the message is the same with the right maths!
Standard error is confusing to put it mildly (stats was one of my A-levels) it actually covers un-even distribution of the data used to form averages (e.g. averaging % deviation from mean of two groups of samples when the two sample groups are not the same size). I have picked up the science uses the term incorrectly but commonly as you indicate. True Standard error is the standard deviation of sample sizes, hence it only produces notable numbers in very large sample sizes.

Referring to SF-36, the vast majority of the population should score 100 and 95 is the bottom of the normal range, not the average. A lot here depends on whether you include those over 60 in the range which massively distorts the figures and makes them nearly useless. A forty year old who function likes a 70 or 80 year old would be in need of rapid medical help :lol:
 

MeSci

ME/CFS since 1995; activity level 6?
Messages
8,231
Location
Cornwall, UK
A forty year old who function likes a 70 or 80 year old would be in need of rapid medical help :lol:

A typical 40-year-old ME sufferer could well be described as functioning like a 70- or 80- year-old. Does such a person need rapid medical help? We certainly need medical help.

I guess you are talking about a 'healthy person' though, but then s/he wouldn't be healthy if s/e was functioning like a 70- or 80-year-old...
 
Last edited:

Leopardtail

Senior Member
Messages
1,151
Location
England
A typical 40-year-old ME sufferer could well be described as functioning like a 70- or 80- year-old. Does such a person need rapid medical help? We certainly need medical help.

I guess you are talking about a 'healthy person' though, but then s/he wouldn't be healthy if s/e was functioning like a 70- or 80- year-old...
Smiles, I think most of us do need rapid medical help. and there used to be times I did think they should be around....
We are thinking fairly alike I suspect......

I was quite definitely thinking that many middle aged ME sufferers have 80 year old or worse levels of function....
 

Cheshire

Senior Member
Messages
1,129
Smiles, I think most of us do need rapid medical help. and there used to be times I did think they should be around....
We are thinking fairly alike I suspect......

I was quite definitely thinking that many middle aged ME sufferers have 80 year old or worse levels of function....

Well, my 70 year old father in law, just before undergoing a heart operation, could walk longer and faster than me.
His doctors were worrying, mine said why not making another appointment with a mental health professional?:eek: