Review: 'Through the Shadowlands’ describes Julie Rehmeyer's ME/CFS Odyssey
I should note at the outset that this review is based on an audio version of the galleys and the epilogue from the finished work. Julie Rehmeyer sent me the final version as a PDF, but for some reason my text to voice software (Kurzweil) had issues with it. I understand that it is...
Discuss the article on the Forums.

Dr Avindra Nath (NIH intramural study) to give Solve webinar, 21 April

Discussion in 'General ME/CFS News' started by Sasha, Mar 11, 2016.

  1. viggster

    viggster Senior Member

    Messages:
    437
    Likes:
    3,703
    Hmm...you're right I don't understand the basis of your complaint. They'll compare patients to healthy controls, and separately, to the asymptomatic Lyme patients. So I don't see how adding the Lyme controls has any bearing on the findings of patients versus healthy controls.

    Edit: I have a friend with a PhD in biostats from Berkeley...I just asked her to explain this issue to me.
     
    Last edited: May 2, 2016
    Roy S likes this.
  2. Valentijn

    Valentijn Senior Member

    Messages:
    14,281
    Likes:
    45,850
    Any time a comparison is made between a group of patients and a control group, it's possible that abnormal results between those groups are due to random chance. Basically, it might just be a meaningless fluke, or "false positive". So p-values are used to set a basic threshold to determine if those results are probably meaningful or not. A typical p-value is 0.05 or 0.01.

    When multiple comparisons are made between the patient group and control group, each comparison introduces an additional opportunity for a false positive. So the p-value is "corrected for multiple comparisons", and there has to be a more drastic difference on any of the comparisons for it to satisfy the threshold of probably being meaningful.

    So if someone is just comparing blood glutamate levels between patients and controls, it can be pretty easy to show that there is a meaningful difference, even if it's a fairly small difference using a small group of patients and a small group of controls. But if the researcher decides he wants to look at all of the amino acids and adds in another 19, you're going from making 1 comparison to making 20 comparisons. So now there is a big chance that modest differences are due to random chance, and the threshold for showing a statistically significant difference in the levels of any one of the amino acids becomes much much higher.

    Similarly, if an extra control group is added, extra comparisons are being made. This might not be a problem if just looking at glutamate levels in ME patients versus healthy controls versus sedentary controls, for example. That's just 2 comparisons: ME glutamate versus one group, then ME glutamate versus the other group.

    But if using two control groups to compare ME results for all 20 amino acids, that's 20 comparisons between ME patients and each group, resulting in 40 comparisons being made total. That's a lot of opportunity for false positive results, which makes it much harder to meet the requirements for statistical significance ... the differences now must be pretty dramatic to be considered meaningful. So if only really interested in glutamate, it's probably best to refrain from testing the other 19 amino acids and to seriously reconsider the usefulness of the extra control group.

    (This actually happened in a study which Dr Mark Hallett and Dr Silvina Horovitz co-authored, albeit with a single control group. 22 patients with Focal Hand Dystonia and 22 controls. The study is all about debunking an earlier study showing significant results regarding GABA levels, according to the abstract. But if you read the paper they added in 19 other metabolites which they mostly weren't interested in or even discussing, and dutifully corrected for the excess comparisons. They found a 10% drop in GABA levels in patients, but it needed to be 30% or greater. They would have needed 80 patients and 80 controls for the 10% difference to be statistically significant, according to their own analysis. But the study is still used as a null result, to support the claim that GABA levels are not different in FHD patients versus controls.)

    This means that if someone is going to make more than a few comparisons between controls and patients, they need a decent sample size of both patients and controls. If they're making dozens of comparisons, they probably need a pretty big sample size of both patients and controls. And if they're going to then add multiple control groups, the problem pretty much explodes into something completely unmanageable.
     
    Last edited: May 2, 2016
    voner, geraldt52, duncan and 2 others like this.
  3. Bob

    Bob

    Messages:
    9,872
    Likes:
    34,218
    England (south coast)
    And that's why most genetic studies are worthless - because there are such a huge number of variables and there's so much noise in the data that you need large numbers of subjects to make sense of the stats. And that's why the UK's 'omics' study is planned to include thousands of participants. For the NIH study, if it were to be done to full scale, it would also need thousands of participants because they are making a huge number of measurements. But I've always understood that phase 1 of the NIH study is an initial observational study that is simply looking for clues upon which to base the next phases of the research, and to dig deeper, rather than an attempt to immediately define ME in phase 1. I thought they were looking for small groupings of patients showing clear abnormalities, that will lead them on to the next stage. If they don't find clear abnormalities in a small number of patients (e.g. like Davis has reportedly found with mitochondria) then I think it might not be such a useful study. They've said that the second phase will include more participants. But maybe I don't know enough about research methodology to interpret it properly.
     
    Last edited: May 2, 2016
    voner, Kati, duncan and 2 others like this.
  4. Amaya2014

    Amaya2014 Senior Member

    Messages:
    215
    Likes:
    543
    Columbus, GA
    Hi @Gingergrrl so sorry you were offended. That wasn't my intent. I'm in prayer for yours, mines, and all our recovery!
     
  5. viggster

    viggster Senior Member

    Messages:
    437
    Likes:
    3,703
    That's my understanding as well. And when Nath says he's calculated sample sizes needed to find abnormalities driving the disease, I think he's talking about exactly the kind of stats that Valentijn mentions.

    Also, the problems with whole genome scan studies that associate SNPs with disease is the opposite - so many comparisons are made that if you set a p value of 0,5 for significance, some falsely associated SNPs will pop out. In follow-up studies, many SNPs that at first look associated with diseases fall apart.

    Edit: I also want to add that Nath and others have communicated to me they are confident they can find the immune abnormalities driving the illness. They are not deliberately setting this up to fail. I guess if Valentijn thinks they are, or thinks they're incompetent, there's little reason to continue the discussion.
     
    Last edited: May 2, 2016
    aimossy, ukxmrv, Comet and 3 others like this.
  6. Riley

    Riley Senior Member

    Messages:
    156
    Likes:
    389
    Louisiana
    I love to hear that!
     
    A.B., Comet, Blue and 1 other person like this.
  7. Bob

    Bob

    Messages:
    9,872
    Likes:
    34,218
    England (south coast)
    I don't think Nath is setting it up to fail (i trust him from what I've seen of him) but I do understand why there is a major lack of trust with other players at the NIH.
     
    Last edited: May 2, 2016
    Hutan and Comet like this.
  8. duncan

    duncan Senior Member

    Messages:
    2,085
    Likes:
    4,521
    Ok, @viggster, the flaws that might result in failure could be inadvertent. A failure would still be a failure, yes?.

    Just to be safe, wouldn't it seem prudent to identify the potential flaws - already done, in large measure - and remedy them?
     
    ukxmrv and Valentijn like this.
  9. Valentijn

    Valentijn Senior Member

    Messages:
    14,281
    Likes:
    45,850
    Do you know where the exact quote for that is?
     
  10. Bob

    Bob

    Messages:
    9,872
    Likes:
    34,218
    England (south coast)
    I remember Nath saying something like that (i.e. that his stats colleagues have looked at, and approved, his study design), and i think it's in the Solve ME/CFS Initiative webinar transcription, Val.

    Edit: But I can't find it. (And it's possible that I've mis-remembered.) Sorry - I'm not being any help here!
     
    Last edited: May 2, 2016
  11. Bob

    Bob

    Messages:
    9,872
    Likes:
    34,218
    England (south coast)
    In the webinar, Nath discusses some of the issues we've been taking about...
     
    Kati likes this.
  12. duncan

    duncan Senior Member

    Messages:
    2,085
    Likes:
    4,521
    A problem with Nath's reassurances is they don't seem to take into account the fact that the history of trying to characterize ME/CFS has been marred, repeatedly, by studies that have done precisely what he seems intent on shrugging off, i.e, including non-pwME in the study sample.

    The results almost invariably have been very very bad for us.
     
    JayS likes this.
  13. Comet

    Comet I'm Not Imaginary

    Messages:
    681
    Likes:
    3,665
    :D :D :D
     
    A.B. likes this.
  14. Gingergrrl

    Gingergrrl Senior Member

    Messages:
    10,232
    Likes:
    29,300
    USA
    It's alright and I knew it was not your intent but just wanted to let you know how the words came across to someone 100% wheelchair bound who can only walk a mile in their dreams.
     
    Amaya2014 likes this.
  15. Amaya2014

    Amaya2014 Senior Member

    Messages:
    215
    Likes:
    543
    Columbus, GA
    Gurrl, the struggle is real!:hug:;) Speaking of dreams, I had a most delicious one the other day that I was biking or running...not for sure but it was exhilarating. I wish it could be used as proof that my subconscious hadn't gotten the "all in your mind" memo;):D

    Also, when I was still crashing severely I was offered assistive devices numerous times. I walk a thin line with the fear that I could worsen further and lose mobility. I'm very grateful for the function that I have and I don't take the subject lightly. This illness is devastating at every level.
     
    Last edited: May 2, 2016
    Gingergrrl likes this.
  16. Gingergrrl

    Gingergrrl Senior Member

    Messages:
    10,232
    Likes:
    29,300
    USA
    It's alright Amaya and don't worry about it any further.
     
  17. viggster

    viggster Senior Member

    Messages:
    437
    Likes:
    3,703
    He's not shrugging it off. Patients must meet CCC criteria with PEM. I believe this is the first US federal study with such strict entry criteria.
     
    Bob and Comet like this.
  18. viggster

    viggster Senior Member

    Messages:
    437
    Likes:
    3,703
    Also, I want to say that I understand why folks are suspicious and anxious. I get it. NIH has a bad track record and they've included a few suspect people on this (very large) study team. I think it's fine to analyze the study and look for ways to improve it. At the same time, there seems to be a certain amount of willful ear closing going on. Issues that NIH people have addressed keep being brought up as if they have not been addressed.

    -Nath, a neurovirologist, is the PI. He said final interpretation of data lies with him. He said he's not interested in psychology.
    - Nath explained that he has run stats on sample sizes and that he has flexibility to add more patients if necessary
    - NIH has repeatedly said patients must meet Canadian Consensus Criteria with PEM, and yet a poster a few pages back wrote a long analysis of how this study will fail because NIH is using Fukuda criteria. (Huh???)

    It's almost like half the community watches Fox News and the other half MSNBC. It's hard to have a dialogue when each side feels entitled to its own "facts".
     
    Sasha, Scarecrow, aimossy and 4 others like this.
  19. halcyon

    halcyon Senior Member

    Messages:
    2,330
    Likes:
    5,457
    The large criticism isn't with Nath and data interpretation I don't think. It's with Walitt and patient selection. The study design could be 100% bulletproof, but if you run it on the wrong patients it's game over. If the misdiagnosis rate in the US is anywhere near what it is in the UK (and I don't think we have any data on this) then this is of course a huge concern.
     
    Gingergrrl and duncan like this.
  20. Denise

    Denise Senior Member

    Messages:
    1,095
    Likes:
    3,449
    I would very much like to know how NIH is characterizing PEM and how that characterization plays out in the study. As we know, PEM is not fatigue but I feel as though that is how I have heard NIH speak of PEM. I seem to remember hearing the phrase post-exertional fatigue...
    Unless we are very clear on what we mean when we use terms and unless we are clear on what NIH (and everyone else) means when they use terms - we may be speaking about different things.
     
    voner, Valentijn, Kati and 2 others like this.

See more popular forum discussions.

Share This Page