A New Decade of ME Research: The 11th Invest in ME International ME Conference 2016
Mark Berry presents the first in a series of articles on the 11th Invest in ME International ME Conference in London ...
Discuss the article on the Forums.

What constitutes reliable evidence?

Discussion in 'Other Health News and Research' started by Laelia, Apr 14, 2017.

  1. Laelia

    Laelia Senior Member

    Messages:
    243
    Likes:
    444
    UK
    Dear Jonathan,

    I am not trying to prove that the 'establishment' view of reliable evidence is wrong. I still don't understand what the 'establishment' view of reliable evidence is. So how can I possibly be trying to prove that it is wrong?

    I'm not 'playing around with words'. I am trying to understand their meaning in the context that they are used.

    So have I misunderstood here Jonathan? Is it the case that you and others believe that anecdotal information from patients and doctors can sometimes provide reliable evidence?
     
    Last edited: Apr 26, 2017
  2. Valentijn

    Valentijn Senior Member

    Messages:
    14,281
    Likes:
    45,883
    Anecdote can be a good starting point. But it still rates far below a well-designed trial. If a well-designed trial contradicts the anecdotes, then the anecdotes are likely wrong, not the trial. If neither contradicted nor supported by trials thus far, anecdotes are a very flimsy basis for treatment.
     
  3. arewenearlythereyet

    arewenearlythereyet Senior Member

    Messages:
    1,478
    Likes:
    7,586
    From my perspective @Laelia you have everything you need to understand written in this thread by a number of people who are genuinely trying to help you understand the basics for scientific research and the difference between GPs prescribing stuff. It does come across very strongly that you are twisting words around and disagreeing with every single post or opinion that is made. Perhaps re- read through the thread and ask very specific questions to help you understand if that is genuinely your intention rather than a series of open ended disagreements?. I honestly can't grasp,what's not to understand? This will be my last post here since I feel I've added all I need to and this thread appears to be going around in ever decreasing circles.
     
    svetoslav80 and barbc56 like this.
  4. Jonathan Edwards

    Jonathan Edwards "Gibberish"

    Messages:
    5,255
    Likes:
    32,048
    Thank you.

    Ditto.
     
    barbc56, Barry53 and slysaint like this.
  5. pamojja

    pamojja Senior Member

    Messages:
    810
    Likes:
    870
    Austria
    He could be right, it could have been a temporary improvement already worsening shortly after. However, very unlikely with chronic conditions and continuous remission. There are statistics for the progression of each chronic condition, where improvements with (or despite) anecdotal treatments are very unlikely, like winning the check-pot.

    However, it will add one more anecdotal success to the number already existing for that treatment for people interested to hear. And as a beneficiary of what might appear like winning the check-pot - having overcome a by medical standards non-reversible disease - this question in retrospect does seem a bit moot. While having one's life back means everything (and easy in comparison to admit loosing a scientific question instead). ;)
     
    Laelia likes this.
  6. Barry53

    Barry53 Senior Member

    Messages:
    2,277
    Likes:
    13,716
    UK
    Also ditto.
     
    barbc56 likes this.
  7. Laelia

    Laelia Senior Member

    Messages:
    243
    Likes:
    444
    UK
    Fine.

    If I quote any of your comments in future posts I will do it without using the quote feature so that you will not be sent an alert. I am doing this out of respect for your decision not to contribute any further to this thread.
     
  8. Laelia

    Laelia Senior Member

    Messages:
    243
    Likes:
    444
    UK
    I’m quite sure that I haven't misunderstood here and that Jonathan does consider anecdotes from patients and doctors to be unreliable evidence on their own. (Just to clarify, I'm talking here about evidence that can be used to determine treatment efficacy).

    Hello @Valentijn, thank you for your response :)

    I have a different take on this. I agree with you that anecdotes are often a flimsy basis for treatment, however I don't think this is always the case. I believe that some anecdotes might be able to provide us with quite reliable evidence. For this to be the case they would probably need to meet the following criteria:
    1. The diagnosis is certain

    2. The prognosis for the condition is poor and well established

    3. There are marked improvements in functioning (well beyond any previous improvements if the condition is cyclical)

    4. Improvements can be measured objectively
    I'm sure @pamojja won't mind me using him as an example here, given that he has shared his story on here. Assuming that his story is true and he is not just the owner of some company selling overpriced supplements (and I trust him that this is not the case ;)), I believe that this is an example of an anecdote that provides us with reliable evidence given that it meets all the above criteria.
     
    Last edited: Apr 28, 2017
  9. Valentijn

    Valentijn Senior Member

    Messages:
    14,281
    Likes:
    45,883
    @Laelia - No, those things do not make anecdote reliable. Even with completely objective indications of improvement (relevant lab values, etc), you still don't know if someone would have had the same improvement without the treatment. This is why controlled trials are a huge improvement over non-controlled. They account for random improvements, so the amount of improvement (if any) attributed to the treatment can be determined.

    And anecdotal reports are much more selective than trial reports. Patients don't run around to a dozen internet forums to announce it every time they try something that didn't help. They also often don't account for other possible factors, etc.
     
    BurnA and barbc56 like this.
  10. Laelia

    Laelia Senior Member

    Messages:
    243
    Likes:
    444
    UK
    I'm not disputing this of course!

    This is why I added the second criterion about prognosis. If the prognosis is very poor and it's it's well established that this is the case then I think we can be reasonably confident that they would not have had the same improvement without the treatment.

    Yes, of course I'm not suggesting that a single anecdote provides reliable evidence that the treatment is effective in other patients. I'm simply suggesting that it provides reliable evidence for efficacy in that patient.

    I thought I'd covered all bases here but perhaps I missed something?
     
  11. pamojja

    pamojja Senior Member

    Messages:
    810
    Likes:
    870
    Austria
    These are indeed the criteria for anecdotal evidence. Which is more than enough evidence for warranting further studies. But for it to be considered scientifically validated treatment it has to pass the final pinnacle of randomized controlled trials. Too bad that would cost a million and no one would spend that much on non-patentable compounds without any profits.

    The advantage of it not being validated by RCTs is, that these non-patentable compounds remain more likely to be widely available over the counter. And not marketed as a 'drug' (the opposite of: “These statements have not been evaluated by the Food and Drug Administration. This product is not intended to diagnose, treat, cure or prevent any disease.”) with an invariable price-hike, restricted availability through pharmacies and/or prescription only.

    No I don't mind, and don't own or promote any company. Financially it is indeed a burden, but try to get everything at a reasonable price/quality ratio. For example by buying in bulk, wherever possible.

    As I wrote in my introduction as response to Valentijn's remarks:

    That's a feat - testing synergistic interventions - modern science simply isn't capable of. Though they use all the time multiple drugs at the same time - (for example with CVD: cholesterol lowering, anti-hypertensive and anti-coagulation..) they as well have never established how all these agents evidenced separately in RCTs, would be effective in any of possible combinations. It's also done without 'reliable' evidence all the time.
     
    Laelia likes this.
  12. Knockknock

    Knockknock

    Messages:
    212
    Likes:
    124
    another thing i will like to say, thanks to new tech, epigenetics etc, there is a lot of real evidence of many genetic changes like methyliation path ways, detox and vitamin receptors mutations etc, it may all sense to me that if you supplement , remplace , help this deficiencies it may help our defective mechanism to start working again.
    The funny part is that me/cfs as i mentioned before doesn't have the back up the large studies the long and expensive studies to be validated.
    whenever there is something promising its turn dow by the GOV's, its clear THEY DONT WANT RESEARCH THEY DONT WANT A REAL UNDESTANDING OF THIS ILLNESS!!!!
    over this past year i have read so much, i have seat with my wife and exchange opinion trying to put this puzzle together.

    it all make sense to us..even many things that have being dismissed by the US government and their institutions, NIH. CDC. HHS..

    Guys there is doctors out there speaking not just Dr Me
    we are ginea pick pigs already!!
    i much reather live a miserable life trying than seating back watching our governments and their institution treating us like if we were a bunch of idiots that didn't know what's going on.
     
    Laelia likes this.
  13. wdb

    wdb Senior Member

    Messages:
    1,381
    Likes:
    2,981
    London
    In which case ME/CFS anecdotes can never be reliable as until bio-markers are discovered the diagnosis is never certain, we can't be sure what one of us is suffering with is the same disease as someone else even with the same diagnosis, and the hallmark symptoms (PEM, fatigue,muscle pain,brain fog) are impossible for people to objectively measure in themselves.
     
  14. Laelia

    Laelia Senior Member

    Messages:
    243
    Likes:
    444
    UK
    Couldn't you use that as a reason to say that any research into ME/CFS is unreliable, including even the 'gold standard' double blind controlled trial? I thought we were campaigning for the use of strict criteria met by certain definitions to confirm the diagnosis until a diagnostic biomarker is found?

    I think there have been plenty of ideas from people on here about how to measure these things objectively. I thought that a big part of the criticism of the PACE trial was that they didn't use objective measurements?
     
    pamojja likes this.
  15. RogerBlack

    RogerBlack Senior Member

    Messages:
    897
    Likes:
    2,903
    I question this, for a modest (perhaps zero) investment.
    Cognitive tests that are available on phones seem useful - reaction speed, n-back testing.
    Fatigue - one handed grip strength, single grip, (grip meters are pretty cheap), hearth rate variability seems to also be plausibly measurable.
     
    Laelia likes this.
  16. Laelia

    Laelia Senior Member

    Messages:
    243
    Likes:
    444
    UK
    The PACE trial cost around 5 million pound sterling (and that was for a poorly run study).

    This study looked at the cost of 28 Phase III RCTs (all phase III randomised trials funded by the US National Institute of Neurological Disorders and Stroke before Jan 1, 2000):
    https://www.ncbi.nlm.nih.gov/pubmed/16631910

    They found the mean cost was US$12 million per RCT.

    Does anyone know how much the Fluge and Mella rituximab trials cost?

    I don't know how much the cost of clinical trials vary across the board but it I think it's likely they cost a lot more than one million. Which makes it even less likely that non-patentable compounds will be repeatedly put through large scale clinical trials.
     
    Last edited: Apr 29, 2017
    pamojja likes this.
  17. Laelia

    Laelia Senior Member

    Messages:
    243
    Likes:
    444
    UK
    I think that the weak point in ME/CFS is actually the second criterion. I'm not sure whether the prognosis of ME/CFS is well established. Is anyone able to confirm whether or not this is the case?
     
  18. Kina

    Kina

    Messages:
    10,132
    Likes:
    17,238
    Sofa, UK
    @Laelia

    Are anecdotes reliable evidence?

    Well, in research/science anecdotes can be seen as providing useful information. Researchers use anecdotal evidence to suggest new hypotheses but never use anecdotal evidence as validating evidence.

    I think there is a difference between 'evidence' and 'information'. For example, there is a lot of information on Myhill's website which she backs up with a research study or two. When does the information on her website become evidence? I would say that some of the things found on Dr Myhill's are not backed up by reliable evidence because even though she provides a research paper or two, one can find dozens that do not support what she is saying. Along the same lines, if one is providing erroneous information to start with, how can one back it up with reliable evidence.

    Maybe a better question is 'What constitutes reliable information'.

    When members here on PR post anecdotes on how a supplement works for them, I don't ever see that as evidence of anything, I see it as providing information about the supplement. These anecdotes can be helpful -- some times many members say they are experiencing the same thing, some times nobody experiences the effect and there might be a discussion about that. From these discussions, members often post research that might support what is being discussed or refute what is being discussed.

    Remember that the internet is referred to as the 'information highway', not the 'evidence highway'.

    There is a lot of information out there on the internet. I think it's good to have at least some system judge the reliability of evidence-- anecdotes being the least. The best evidence probably comes from meta-analyses which examine a number of valid studies on a topic and then mathematically combines the results using accepted statistical methodology.

    Randomized controlled clinical trials are carefully planned and they include methodologies that reduce the potential for bias(randomization and blinding). They allow for comparison between intervention and control groups. These studies provide sound evidence of cause and effect if done properly. The problem is many studies out there are not done properly and the results are therefore faulty. Those studies do not provide reliable evidence and some times it takes knowledge of statistics and/or research methodology to figure out if results are good or not. It goes on and on. The PACE study is not reliable evidence because ... been down that road no need to say why.

    There are some scientific guidelines/definitions that guides what reliable evidence is. Members are trying to impart this information on the thread. Not sure why you find it necessary to argue every point made though. This information is something that is accepted throughout the scientific community.
     
    BurnA, svetoslav80, slysaint and 3 others like this.
  19. pamojja

    pamojja Senior Member

    Messages:
    810
    Likes:
    870
    Austria
    Don't know, but maybe she also got across something written by the scientist Ioannidis:

    So Laelia, my million or your 12 millions are rather pennies compared to the $804 million it costs to get a phrarmaceutical 'evidence based'!
     
    Laelia likes this.
  20. Laelia

    Laelia Senior Member

    Messages:
    243
    Likes:
    444
    UK
    Hi @Kina,

    So are you saying that I should just accept what the scientific guidelines say are true without thinking it through and trying to understand it for myself? Maybe in that case I should also just accept that the NICE guidelines for the treatment of ME/CFS are also true? ;)

    Will probably have more thoughts to add later...
     

See more popular forum discussions.

Share This Page