• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

How to cheat in CFS studies to get the answer you want

G

Gerwyn

Guest
Good ways to slant the data in your direction--quite innocently of course!

* Throw all your data into a computer and report as significant any relation where P<0.05

* If baseline differences between the groups favour the intervention group, remember not to adjust for them

* Do not test your data to see if they are normally distributed. If you do, you might get stuck with non-itemmetric tests, which aren't as much fun - this is a common one in psych studies

* Ignore all withdrawals (drop outs) and non-responders, so the analysis only concerns subjects who fully complied with treatment

* Always assume that you can plot one set of data against another and calculate an "r value" (Pearson correlation coefficient), and assume that a "significant" r value proves causation another common one

* If outliers (points which lie a long way from the others on your graph) are messing up your calculations, just rub them out. But if outliers are helping your case, even if they seem to be spurious results, leave them in - a psych favourite

* If the confidence intervals of your result overlap zero difference between the groups, leave them out of your report. Better still, mention them briefly in the text but don't draw them in on the graph—and ignore them when drawing your conclusions

* If the difference between two groups becomes significant four and a half months into a six month trial, stop the trial and start writing up. Alternatively, if at six months the results are "nearly significant," extend the trial for another three weeks

* If your results prove uninteresting, ask the computer to go back and see if any particular subgroups behaved differently. You might find that your intervention worked after all in Chinese women aged 52-61

* If analysing your data the way you plan to does not give the result you wanted, run the figures through a selection of other tests
- Practically every psych study has a variation of this
 

Mithriel

Senior Member
Messages
690
Location
Scotland
At university we were told about a study into the rhythm method of birth control which had a very good success rate but a high drop out rate. It turned out most of the people who dropped out did so because they were pregnant. :Retro smile:

I think the same principle gave the success in GET trials.

Statistics are so difficult to follow at this level. I am sure they sure so complex nowadays because you can type into a computer and get a number out. When we did experiments and worked out the errors using a column of figures with paper and pencil we had to understand what we were doing.

Mithriel
 

Hysterical Woman

Senior Member
Messages
857
Location
East Coast
At university we were told about a study into the rhythm method of birth control which had a very good success rate but a high drop out rate. It turned out most of the people who dropped out did so because they were pregnant. :Retro smile:

I think the same principle gave the success in GET trials.

Statistics are so difficult to follow at this level. I am sure they sure so complex nowadays because you can type into a computer and get a number out. When we did experiments and worked out the errors using a column of figures with paper and pencil we had to understand what we were doing.

Mithriel

Hi Mithriel,

Loved your story about the pregnant "drop outs". Statistics are difficult to follow at a high level. And of course, there is a huge incentive for them to be "manipulated" since there are millions of $$$ at stake. Sigh.

Take care,

HW
 

starryeyes

Senior Member
Messages
1,558
Location
Bay Area, California
At university we were told about a study into the rhythm method of birth control which had a very good success rate but a high drop out rate. It turned out most of the people who dropped out did so because they were pregnant.

:tear: That is so funny and illustrates just how flimsy the "science" behind a lot of health studies is.