• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of, and finding treatments for, complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia, long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

What constitutes reliable evidence?

Laelia

Senior Member
Messages
243
Location
UK
By unproven in this case I mean not tested in a reliable way, and that can all sorts of things in different situations.

I think then by unproven you don't mean unproven at all! ;)

I think you are playing around with words a bit here, Laelia.

Not at all. If we use words which imply a different meaning to the one which we intend then people are going to get very confused (or perhaps I should say even more confused because I think they are confused already!).
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
There seems to be widespread misunderstanding of what good evidence is. Evidence based medicine is about reducing risk of biased outcomes as much as possible. There are no guarantees. Indeed many controlled case studies are much better than many double blinded RCTs, and many dbRCTs are quite rubbish. RCTs are even worse. Its all about the details.

A really well designed dbRCT has a high chance of producing good results. Not a guarantee, a high chance.

Effect sizes are important. If the effect is blatantly obvious, large and repeated benefit in a large number of case trial subjects with controls, then an RCT can be superfluous. An RCT that was well designed would not tell you much more and would cost a lot more to run.

Now if the effect size is weak, and works only on a small subgroup, then a dbRCT is almost mandatory, one that is big enough to have the statistical power to detect smaller differences.

Reliability is only useful when its understood to be about low risk in the findings being wrong. Studies also fail when they measure the wrong things. I want to know, with any serious drug, what the five year survival rate is. That tells you a lot.

It does no good if a drug saves you from a lethal cardiac event but kills so many that the five year outcome is actually worse. This has happened, with anti-arrhythmia drugs, and was a big impetus for evidence based medicine.

These kinds of findings could be done by simply keeping track of trial participants for a further five years.

Risk in false negatives is also a big problem. Simplistic thinking on these issues is a big problem. I lack enough of a working skill set to evaluate such studies to the extent that is necessary, but I know enough to be aware of the risks. Even regular experimental scientists often lack the skill set needed, and even fewer doctors have such a skills set either.

There are so many problems in modern science that a whole lot of it cannot be considered completely reliable.

Yet if all you have are a few case studies, and no other studies, and drugs have passed at least early safety requirements, then I think its justified to use them in consultation with patients, on the understanding that there are no guarantees, and with full discussion and disclosure of risks. Some with severe disease will want the option to do that.

When I hear "RCTs are the Gold Standard of medicine" I cringe. This is deceptive. It requires a big list of caveats to be accurate, but I am betting most who parrot out this claim are not aware of the caveats.

As many here are aware the PACE trial was an RT with comparison groups, but not double blinded and not technically with a control group. Yet its claimed to be Gold Standard. This is fools gold, not real gold.

Reliability comes with time. Its about lowering risk. Even many dbRCTs for drugs which also pass a meta-analysis still need long term monitoring of patients.

This is still not discussing risk/benefit ratios, which are vitally important. Vaccines for example are definitely dangerous. However aside from bad formulations with severe side effects its typically the case the diseases they help against are even more dangerous than the vaccines. I might not even been sick today if a measles vaccine was around in the early 60s. Many others would have decreased disability and longer life. Yet vaccines can also cause a lot of harm. Its about determining the risk-benefit ratio, not risk or benefit on their own.
 
Last edited:

Jonathan Edwards

"Gibberish"
Messages
5,256
I think then by unproven you don't mean unproven at all! ;)



Not at all. If we use words which imply a different meaning to the one which we intend then people are going to get very confused (or perhaps I should say even more confused because I think they are confused already!).

Dear Laelia,
I am not clear in what way what I have said deviates from ordinary meaning - which changes with each circumstance, as is well recognised by linguists who study 'pragmatics'. As far as I can see you are trying to argue that somehow Dr MPhil's approach is not too bad because others are not perfect either. That makes no sense to me. If Dr Myhill has no reliable evidence her ideas are unproven. That is pretty plain English.
 

pamojja

Senior Member
Messages
2,397
Location
Austria
And in relation to pamojja's comment about statins - that is reliable evidence for what it reports but any sensible person can see that that is not reliable evidence for statins being worth taking. Government guidelines are rubbish and irrelevant here. Reliable evidence is ultimately what an intelligent person with experience of all the pitfalls involved can work out is reliable from common sense. It has nothing particular to do with any specific methodology.

In our days it has everything to do with the specific methodology to get an isolated compound approved as medicine, which costs millions. Therefore will only be used for patentable chemicals not naturally occurring (with their guaranteed toxicity).

Such expensive evidence will never be forthcoming for natural compounds. And therefore natural compounds will never be approved as treatment, like vitamin C or l-lysine (as in Linus Pauling's therapy, despite being around for decades and numerous anecdotal evidence never trialled, but still defamed as 'quack'). Which obviously in my case did much more than all approved chemicals together ever could for my condition.

But the reason I posted the only available evidence for most common prescribed drugs against CVD, the multi-factorial chronic disease most people will die from, is that after almost a century of research and the unbelievable amounts invested against its mortality, despite there isn't anything found. One reason obviously because it's specific profit-driven methodology isn't designed for that..


As a new-comer to this forum I'm really surprised about the prevalent blind faith in this profit-driven methodology, to find a much less likely cure for a much rarer chronic condition, like CFS/ME (despite infinite smaller investment in its research). While on the other hand demanding equal 'evidence' from natural approaches. Which this profit-driven methodology is designed for never to provide. As can be seen with similar multi-factorial diseases, like CVD or cancer.

Why this irrational hope for a cure? Beyond conventional medicine's best marks at 1-3% improvement of 5-year mortality in major chronic diseases? While at the same time expecting evidence in multiples of that from alternative approaches, or being so quick in defamation?

And to be clear. Almost all doctors I told of my informed-consent decision after my risk/benefits evaluation, considered it irresponsible, even accused me of causing my own death and suicide, or mentally disturbed. Because most haven't read up on statins NNT of 83, but acted in the wrong believe of 30% relative risk reduction with statins (2 dying in the treatment arm, versus 3 in the placebo in a 100 patients is indeed 30%, relatively..). In this most common occurring example (who doesn't has a relative on statins believing its the 'medicine') modern medicine is as guilty of quack, as any alternative practitioner wrongly promising a cure.

Therefore yes, common sense. Not 'evidence based' medicine driven by profit. Modern medicine doesn't has any 'cures' for major chronic conditions, nor does alternative. Anecdotal temporary remissions, but no cure for a long time to come. First, do no harm.
 

Laelia

Senior Member
Messages
243
Location
UK
I am not clear in what way what I have said deviates from ordinary meaning

By unproven in this case I mean not tested in a reliable way, and that can all sorts of things in different situations.

What you wrote here suggests to me that testing in a reliable way proves a treatment's efficicacy. For me 'proves' is not the appropriate word.

As far as I can see you are trying to argue that somehow Dr MPhil's approach is not too bad because others are not perfect either.

No I am being misunderstood again! :confused:

This is not what I am trying to argue.

If Dr Myhill has no reliable evidence her ideas are unproven.

I agree with you that Dr Myhill's ideas are unproven. But where I am less certain is whether or not she has any reliable evidence.
 

bertiedog

Senior Member
Messages
1,738
Location
South East England, UK
Does reliable evidence come down to trust in the doctor concerned? If a doctor who specialises in treating a large number of patients who have an illness like ME/CFS finds that certain combinations of drugs and/or supplements appear to benefit a good proportion of these patients then why would one ignore this finding just because there are no studies regarding every single thing they recommend (which obviously costs huge sums of money to run).

To be honest when I watch videos of the many conferences that have taken place over the years since I have been sick I sometimes get the feeling that its a double edge sword. Our illness because of its apparent complexity is e good source of money for researchers. So many years have gone by and yet we still don't even have a consensus as to what is going on with this illness although it does seem there is at least a chance that several researchers might end up soon agreeing about this.

A researcher has an idea of what might be going on and then has to raise large sums of money to test this out which keeps them in work and they don't suffer if it doesn't work out, its the poor patients who do that time and time again. Although I can see that their reputation might well be affected as happened with the XMRV saga.

Personally, it doesn't make any sense to me to ignore the recommendations of a specialist doctor who has vast experience in a specific illness especially when they have at least conducted some published research themselves and who does follow the published research of others in their recommendations to their patients.

Pam
 

wdb

Senior Member
Messages
1,392
Location
London
Yes. All the evidence is listed on her website. I can't comment on the quality of the evidence but it definitely exists. There is mention of a few double blind controlled trials as well, such as this one looking at the effectiveness of magnesium injections:
https://www.ncbi.nlm.nih.gov/pubmed/1672392

That is a surprisingly good study it's on sci-hub is anyone wants to read it. They did a good job of blinding and controlling for expectation and influence and the results look quite strong. Only weakness I see are that it was quite a small sample (15 in treatment and 17 controls) and only lasted 6 weeks so it's unknown if the effect could be maintained. Also the Nottingham Health Profile looks like a bit of a blunt instrument.
 

Laelia

Senior Member
Messages
243
Location
UK
That is a surprisingly good study it's on sci-hub is anyone wants to read it. They did a good job of blinding and controlling for expectation and influence and the results look quite strong. Only weakness I see are that it was quite a small sample (15 in treatment and 17 controls) and only lasted 6 weeks so it's unknown if the effect could be maintained. Also the Nottingham Health Profile looks like a bit of a blunt instrument.

This study was also posted in this thread but there has not much discussion about it so far:

http://forums.phoenixrising.me/inde...d-chronic-fatigue-syndrome.27439/#post-418052
 

Laelia

Senior Member
Messages
243
Location
UK
As far as I can see you are trying to argue that somehow Dr MPhil's approach is not too bad because others are not perfect either.

No I am being misunderstood again! :confused:

This is not what I am trying to argue.

I don't think I am in a position to argue anything at the moment. I have not yet got to grips with these concepts and I find myself getting rather confused. I don't think I am by any means the least intelligent or most brain fogged person on here so if I'm struggling to understand then I'm sure others are too!

Jonathan, am I correct in saying that for you there is only 'reliable' and 'unreliable' evidence with no in between or degrees of reliableness?
 

skipskip30

Senior Member
Messages
237
Having seen Dr Myhill personally she does make it clear that there is a lot of experimentation, trail and error and educated guesses involved in her protocols. She didn't help me personally but I didn't feel misled by her.

There are many things she says which I disagree with (very strongly in some cases) but in a world with very few options for us she has provided some hope for a lot of people.

I realise its very fashionable for the medical profession to hate on her but people should stick to specific proven criticisms rather than "presumably's".
 

wdb

Senior Member
Messages
1,392
Location
London
I realise its very fashionable for the medical profession to hate on her but people should stick to specific proven criticisms rather than "presumably's".

What specific criticisms do you consider unjustified ?

You say that critics should stick to proven statements then do you also think that Myhill should stick to proven statements ?
 

Knockknock

Senior Member
Messages
212
Hi @MEMum. I’m just responding to your comment from the other thread: “I don't think we can expect Dr Myhill to do double-blind placebo controlled etc studies, given that all her patients are paying.However I feel that if significant numbers were getting well enough to return to work and get off benefits, then more of the ME community (PwME and carers) would have heard about it.Any comments I remember seem to have been of small, incremental changes.” [My bold]

There are many recovery stories on here with significant numbers reporting that their improvements were linked to the sorts of treatments Dr Myhill recommends. Of course we don't know whether these people have correctly identified the reasons for their recovery and so it’s difficult to draw any conclusions from this. But the anecdotal evidence is there for sure.

@Jonathan Edwards: “But if Dr Myhill has not done a proper trial she was presumably handing out treatments when she has no idea if they work or not.”

I think Dr Myhill believes (rightly or wrongly) that the treatments she hands out do work. Yes, she does experiment when she is treating patients. I don't see a problem with this, providing it is made clear and the risks explained.

@Jonathan Edwards: “I very much doubt she said to patients 'I have no idea if this works or not but try it because I have a theory it will work that I will never actually know the validity of because I am never going to test it properly'.”

Actually she does admit that she sometimes tries treatments when she doesn't know if they will work or not. For example this is what she says on her website when explaining the indications for use of Enzyme Potentiated Desensitisation (EPD):

“I do sometimes use EPD for the worst possible reason, that is I can't think of anything else to do when all else has been tried. However it is surprising how often this works!”

(Note also here that Dr Myhill believes she is able to determine that this treatment works just from the clinical response).

Many ME/CFS patients want to experiment with different treatments even if there is only poor evidence for their efficacy. For many of us, we fear that the risk of not trying anything is greater than the risk of experimenting. We need practitioners like Dr Myhill who can assist us through this process.

@TiredSam: “The amount of evidence needed to demonstrate that what Dr Myhill is saying is nonsense is zero, so there's enough. If she provides no evidence for her claims then she hasn't given anyone any reason to believe them or to go to the effort of disproving them, there's nothing to disprove.” [My bold]

It’s not the case that she provides no evidence for her claims. I understand that people might want to question the reliability of that evidence but it's incorrect to say that there is none there in the first place.

@Jonathan Edwards: "So the long and short of it seems to be that claims for harm following GET need to be substantiated with convincing controlled study evidence."

Do none of research findings from Ron Davis or any of the scientists provide evidence that GET might be harmful? Is it not possible for an understanding of the biomedical nature of disease to provide reliable evidence for whether a treatment may be beneficial or harmful? (I believe this is the type of evidence that Dr Myhill draws on heavily in making her claims).

@mango: “I used to be very interested in Dr Myhill's advice and articles. I tried a lot of what she suggested, but sadly it didn't work for me.However, my interest in what Dr Myhill has to offer totally disappeared after listening to her talk at the Abundant Energy Summit in 2015.http://forums.phoenixrising.me/index.php?threads/the-abundant-energy-summit.39513/In her talk she made a number of statements that I found hugely off-putting, such as claiming that CFS isn't a diagnosis but a "clinical picture", that carbohydrates and vegetarianism cause CFS, and that people get CFS because they stay up all night studying and drinking buckets of coffee, eating lots of chocolate bars. She also expressed several very judgmental things about the personality traits of CFS patients as a group.She also claimed that the only ones who don't get better with her treatments are the ones who don't try hard enough or not long enough...I was shocked and felt very upset after listening to that talk.”

I am sorry to read this @mango, I’m not surprised you felt that way. Dr Myhill absolutely deserves criticism for saying some of these things.

@Jonathan Edwards: "Unfortunately, audit is quite often hijacked as a 'soft' way of trying to show a treatment works. 'Let's see if the treatment we have been using is making people better'."

"If your lightbulb moment comes from lab science rather than an anecdote, as in my case, there is a place for a small preliminary trial that is not expected to provide reliable evidence that the treatment is effective but can be expected to give some idea whether the whole thing is a waste of time or whether a full trial should be set up."


Is it not possible that the science laboratory could become hijacked in exactly the same way as you describe for an audit?
We all have to respect every one opinion on this threads, they are for everyone to express their opinion, but something I have noticed, there is a subset of people( members) they Always run to challenge, to discredit, to disproved everything, from recovery stories of patients( me/suffers) to treatments that Doctors are using base on their long experiance with their patients, by seen how they respond to treatment.
"" what large studies you guys are asking for?? What trials?? What science?? We all know that have been denied forever to me/cfs!!
Science is being denied blocked missleaded for us!!
They vey little we have its from theories based on personal experiences, doctors experiences, brave doctors and researchers that have treated their patients, that have done their research depite their limitations, lack of resourses and resistance and oposition from the govermments and their heath institutions.
We are not HIV/AIDS, we are not MS... we are ME/CFS.. the forgotten plague!!!
The interesting part is that some of this people that are always criticizing everything and doubting what other members say are not even sick with me/cfs.. even though some do.
Thanks to many of this doctors many patients have recovered their life to some degree, some others have almost completely recovered.
We see this more and more, this is thanks to a better understanding of the disease, people are using many are using antivirals, others supplements, others diets, that are preventing them from further damage to their cells, organs or at lease slowing down their progresion.
Lets be real, the very little we have its working out for many people.
We need more, we avocate for more, but we can not continiusly slam this doctors that with very little are giving us some kind of hope.
 
Last edited:

barbc56

Senior Member
Messages
3,657
@Knockknock

I have to respectfully disagree with you. You may be interpreting debate and critical thinking as negative comments.

While, I will respect others opinions, that doesn't mean I have to agree with each one. Nor does it mean I shouldn't speak up, as long as it is civil and within forum rules.

I strongly believe some of these doctors are sidetracking any real progress and possible treatments for me/cfs.

If there is credible evidence for a treatment, that's what I think. If Myhill has real evidence, if others have really improved from these "maverick doctors " point me to that evidence. Then I might think more highly of them.
 

Laelia

Senior Member
Messages
243
Location
UK
Jonathan, am I correct in saying that for you there is only 'reliable' and 'unreliable' evidence with no in between or degrees of reliableness?

Hello Jonathan,

Another point. I wonder whether you might be mistakenly using the term 'reliable evidence' to refer to only evidence that has been proven to be reliable. This is perhaps where some of the confusion arising from. Surely evidence can sometimes be reliable even though it's not proven to be reliable?

@Jonathan Edwards
 
Last edited:

Laelia

Senior Member
Messages
243
Location
UK
Hello @wdb

What specific criticisms do you consider unjustified ?

I wonder if @skipskip30 had in mind here a couple of the comments made by Jonathan which I responded to earlier in this thread:

@Jonathan Edwards: “But if Dr Myhill has not done a proper trial she was presumably handing out treatments when she has no idea if they work or not.”

I think Dr Myhill believes (rightly or wrongly) that the treatments she hands out do work. Yes, she does experiment when she is treating patients. I don't see a problem with this, providing it is made clear and the risks explained.

@Jonathan Edwards: I very much doubt she said to patients 'I have no idea if this works or not but try it because I have a theory it will work that I will never actually know the validity of because I am never going to test it properly'.”

Actually she does admit that she sometimes tries treatments when she doesn't know if they will work or not. For example this is what she says on her website when explaining the indications for use of Enzyme Potentiated Desensitisation (EPD):

“I do sometimes use EPD for the worst possible reason, that is I can't think of anything else to do when all else has been tried. However it is surprising how often this works!”
[My bold]

You say that critics should stick to proven statements then do you also think that Myhill should stick to proven statements ?

I think you might be referring here to Dr Myhill's statement from the MAIMES campaign:

NICE Guidelines contain no logical, evidence based treatment for PWME. By contrast practitioners working outside conventional NICE Guidelines have established many effective treatments which are safe and efficacious and which get people back to work and off benefits.

I think that Dr Myhill (rightly or wrongly) believes that she has been witness to enough 'proof' to make this statement. So in her eyes she is sticking to proven statements.
 
Last edited:

Laelia

Senior Member
Messages
243
Location
UK
As far as I can see you are trying to argue that somehow Dr MPhil's approach is not too bad because others are not perfect either. That makes no sense to me.

As mentioned already, I'm not trying to argue anything at this stage, I'm just trying to understand.

As far as I can see you are arguing that any science that is unproven is 'bad science'. This also makes no sense to me.