Review: 'Through the Shadowlands’ describes Julie Rehmeyer's ME/CFS Odyssey
I should note at the outset that this review is based on an audio version of the galleys and the epilogue from the finished work. Julie Rehmeyer sent me the final version as a PDF, but for some reason my text to voice software (Kurzweil) had issues with it. I understand that it is...
Discuss the article on the Forums.

How to cheat in CFS studies to get the answer you want

Discussion in 'General ME/CFS News' started by Gerwyn, Feb 2, 2010.

  1. Gerwyn

    Gerwyn Guest

    Good ways to slant the data in your direction--quite innocently of course!

    * Throw all your data into a computer and report as significant any relation where P<0.05

    * If baseline differences between the groups favour the intervention group, remember not to adjust for them

    * Do not test your data to see if they are normally distributed. If you do, you might get stuck with non-itemmetric tests, which aren't as much fun - this is a common one in psych studies

    * Ignore all withdrawals (drop outs) and non-responders, so the analysis only concerns subjects who fully complied with treatment

    * Always assume that you can plot one set of data against another and calculate an "r value" (Pearson correlation coefficient), and assume that a "significant" r value proves causation another common one

    * If outliers (points which lie a long way from the others on your graph) are messing up your calculations, just rub them out. But if outliers are helping your case, even if they seem to be spurious results, leave them in - a psych favourite

    * If the confidence intervals of your result overlap zero difference between the groups, leave them out of your report. Better still, mention them briefly in the text but don't draw them in on the graph—and ignore them when drawing your conclusions

    * If the difference between two groups becomes significant four and a half months into a six month trial, stop the trial and start writing up. Alternatively, if at six months the results are "nearly significant," extend the trial for another three weeks

    * If your results prove uninteresting, ask the computer to go back and see if any particular subgroups behaved differently. You might find that your intervention worked after all in Chinese women aged 52-61

    * If analysing your data the way you plan to does not give the result you wanted, run the figures through a selection of other tests
    - Practically every psych study has a variation of this
  2. Mithriel

    Mithriel Senior Member

    At university we were told about a study into the rhythm method of birth control which had a very good success rate but a high drop out rate. It turned out most of the people who dropped out did so because they were pregnant. :Retro smile:

    I think the same principle gave the success in GET trials.

    Statistics are so difficult to follow at this level. I am sure they sure so complex nowadays because you can type into a computer and get a number out. When we did experiments and worked out the errors using a column of figures with paper and pencil we had to understand what we were doing.

  3. Hysterical Woman

    Hysterical Woman Senior Member

    East Coast
    Hi Mithriel,

    Loved your story about the pregnant "drop outs". Statistics are difficult to follow at a high level. And of course, there is a huge incentive for them to be "manipulated" since there are millions of $$$ at stake. Sigh.

    Take care,

  4. starryeyes

    starryeyes Senior Member

    Bay Area, California
    :tear: That is so funny and illustrates just how flimsy the "science" behind a lot of health studies is.

See more popular forum discussions.

Share This Page