• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of, and finding treatments for, complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia, long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

What constitutes reliable evidence?

Laelia

Senior Member
Messages
243
Location
UK
@user9876: "I had assumed they say this to cover treatments that have been used successfully for many years but because they have been used for so long there are no trials."

(user9876 was responding here to a comment about how NICE say that their guidelines are based partly on evidence from patient and practitioner testimony)

Hi @user9876

Are you sure there are treatments in use that have not been through any trials? I think many on here might find this quite alarming!
 

wdb

Senior Member
Messages
1,392
Location
London
So is this study a reliable or an unreliable piece of evidence?

Well there are levels of reliability it's not one or the other. At the bottom of the spectrum would be anecdotal evidence like 'a few people thought thought they felt a little bit better' this is completely unreliable and impossible to differentiate from what you would expect from placebo response alone, at the top end of reliability would be multiple independent large scale double blinded trials all confirming the same strong result, this would be about as near to proof as you could get.

The magnesium trial I'd personally put towards the reliable end of the spectrum but it's not without some weaknesses.
 

Laelia

Senior Member
Messages
243
Location
UK
Well there are levels of reliability it's not one or the other. At the bottom of the spectrum would be anecdotal evidence like 'a few people thought thought they felt a little bit better' this is completely unreliable and impossible to differentiate from what you would expect from placebo response alone, at the top end of reliability would be multiple independent large scale double blinded trials all confirming the same strong result, this would be about as near to proof as you could get.

The magnesium trial I'd personally put towards the reliable end of the spectrum but it's not without some weaknesses.

Thank you @wdb for a clear and concise answer to my question, I really appreciate you taking the time to respond. I might have more thoughts on this later but I just wanted to say thanks for now... :)
 

Laelia

Senior Member
Messages
243
Location
UK
As far as being a human Guinea pig, this isn't something I would want for myself because I have seen what doctors can do by prescribing inappropriate treatment when they don't know what they are doing. If you are carefree and not risk averse then you could convince yourself that going down this road is something of benefit, but I would always advise to weigh up risks.

I am a very risk averse person too! :nervous: :)

When I said that I thought I might be in favour of the experimental approach I was thinking of things like vitamin B12 injections, magnesium injections, low dose immunotherapy, that sort of thing. Not prescription drugs. I understand that experimenting with drugs carries a far higher risk than these other therapies. I don't believe Dr Myhill prescribes drugs which haven't been through any clinical trials does she?? And I don't think anyone has died from giving themselves a vitamin B12 injection?? :eek: That said, there are always risks whatever the treatment. So I would also advise anyone thinking of going down this road to make sure they are well informed before embarking on any treatments.

Putting aside the ethics of experimenting on humans and all that entails, you could then argue that providing there is a well applied scientific rigour to the design this would still be scientific. If it's just a GP prescribing some stuff and hoping for the best without writing down results in a controlled way then no, this is not science at all.

Obviously one outcome of the treatment if applied scientifically could be that a number of the patients get worse or die due to side effects of whatever treatment is given. This is useful data for the scientific process providing it's designed well and meaningful but not so great for the patient:nervous:.

Isn't this a bit like the auditing that Dr Myhill and the British Society for Ecological Medicine (BSEM) do that I mentioned on the other thread? I don't know if there if there is a well applied scientific rigour to their audits but I think it is a good thing that they are at least collecting some information.
 

user9876

Senior Member
Messages
4,556
@user9876: "I had assumed they say this to cover treatments that have been used successfully for many years but because they have been used for so long there are no trials."

(user9876 was responding here to a comment about how NICE say that their guidelines are based partly on evidence from patient and practitioner testimony)

Hi @user9876

Are you sure there are treatments in use that have not been through any trials? I think many on here might find this quite alarming!

I'm not sure but proper (as in double blinded clinical trials) are relatively new and there are many treatments that have been used for a long time and so I think these haven't been tested by trials but are generally accepted. I think I've read about a few off label treatments that are fairly standard.
 

arewenearlythereyet

Senior Member
Messages
1,478
I am a very risk averse person too! :nervous: :)

When I said that I thought I might be in favour of the experimental approach I was thinking of things like vitamin B12 injections, magnesium injections, low dose immunotherapy, that sort of thing. Not prescription drugs. I understand that experimenting with drugs carries a far higher risk than these other therapies. I don't believe Dr Myhill prescribes drugs which haven't been through any clinical trials does she?? And I don't think anyone has died from giving themselves a vitamin B12 injection?? :eek: That said, there are always risks whatever the treatment. So I would also advise anyone thinking of going down this road to make sure they are well informed before embarking on any treatments.



Isn't this a bit like the auditing that Dr Myhill and the British Society for Ecological Medicine (BSEM) do that I mentioned on the other thread? I don't know if there if there is a well applied scientific rigour to their audits but I think it is a good thing that they are at least collecting some information.

I have given you quite a bit on scientific methodology ...hopefully this shows the difference between a plain old GP prescribing (this is not science) and proper research (science).

Audits are just checks against a standard, not science? So no audits don't validate treatments, they just check that the practice meets a standard. That is not sufficient to say that treatments are valid at all.

With regards to Myhill not prescribing drugs. There is a difference between using an existing drug for a tested purpose and using it for a condition that hasn't been tested. So for example on her pages she recommends carbamazepine for treating people with CFS with neuralgic pain

http://www.drmyhill.co.uk/wiki/Drugs_in_the_treatment_of_CFS_-_always_start_with_tiny_doses

This is problematic in that that there is no study to back up this treatment advice In fact a short search will reveal that carbamazepine taken at the doses she recommends can cause folate deficiency and lead to nerve damage if taken for long enough. Given that she is also recommending treatment for this deficiency, this seems a little .....inconsistent, possibly irresponsible.

Carbamazepine is not something to be prescribing willy nilly since it has many nasty side effects....yet she recommends doseages with no risk statement .....just states it as a matter of fact. This is what's scary about her. She then goes on to promote tempur mattresses on the same page :thumbdown: it's interesting that she does give a click through to tempur beds but no reference to the drug? This to me shows a complete disregard to risk and a level of inconsistency that is difficult to justify as valid medical advice. So she is recommending drugs to patients outside of their intended use and without consideration for the side effects that may be incurred by the patient. :jaw-drop:

the fundamental problem is that it's one thing doing DIY supplement and symptom relief at home, it's another being a doctor prescribing treatments for money. There are lots of people who just want to trust a doctor to tell them what to do. That is why Myhill has a greater responsibility than someone blogging self hacks for faecal transplants etc.
 

barbc56

Senior Member
Messages
3,657
I don't like to post long quotes but I think the following is important enough to put in it's entirety. If we rely on ancedotal reports either from patents or doctors, how do you know what is causing what? There are just too many cofactors to determine this.

Why Bogus Therapies seem to work.

1. The disease may have run its natural course.
Many diseases are self-limiting. For instance, the common cold usually runs a course of approximately five to seven days before it is cured by the body’s own defence mechanisms. Sometimes we take medications on day five when the illness is already improving and then attribute improvement to the medication. In this way antibiotics have “cured” many a common cold.

2. Many diseases are cyclical.
Chronic conditions such as arthritis, multiple sclerosis, allergies and depression have natural “ups and downs”. If treatment is instituted during the beginning of an improvement phase, we then may believe that the treatment has actually effected the cure.

3. The power of the placebo effect.
The belief that we are taking treatment which can cure an illness, causes powerful forces to effect an improvement through mind-body connections. This is called the “placebo effect”.

4. Credit for improvement can be applied incorrectly in dual treatment situations.
Diseases are sometimes treated with a combination of science-based treatment and “alternative” treatment. If improvement then occurs, credit is often inappropriately given to the “alternative” treatment.

5. The original diagnosis or prognosis may have been incorrect.
The diagnosis or prognosis may be incorrect. If improvement occurs after taking a bogus therapy, it can then be given inappropriate credit.

6. Temporary mood improvement can be confused with cure.

Some individuals that practice medicine have forceful and charismatic personalities. They may influence patients to the extent that they experience a psychological upliftment. This feeling of euphoria can be confused with a cure.

7. Psychological needs can distort perceptions.

Persons who invest time and money in bogus therapies, experience psychological distress when they admit to themselves that the treatment has not worked. In order to save face they may experience thought distortions and believe that the treatment has in fact worked. Rather than admit to themselves that the treatment has been a waste, they may convince themselves and others of redeeming value in the treatment.

http://www.health24.com/Medical/Dem...y-bogus-therapies-often-seem-to-work-20120721

Science is built on hierarchies. The bottom of the pyramid shows the least scientific type of evidence. While the RTC is the gold standard you can't stop analyzing there.You still have to question things like methodology, statistics, as well as other factors as reliability and validity. The later can be determined by statistical methods. But you also have to make sure the correct statistics are applied and interpreted correctly.

That's the short version.
.
IMG_0254.PNG
 

barbc56

Senior Member
Messages
3,657
To add to @arewenearlythereyet's list. Apologies if I have repeated things as I wrote this before the other post.

Dr. Myhill has recommended things that are against best medical practice such as babies should be put on their stomachs, the rhythm method for birth control, not having babies go outside for the first six months of their life instead of vaccinations. Using thermal imaging not mammograms as screening for breast cancer as well as other clinically unsubstantiated recommendations.

Tbh, I'm not sure how many of the above practices she was asked to remove from her
 

Laelia

Senior Member
Messages
243
Location
UK
The one below which is about Dr. Myhill. This thread is about reliable evidence, not Myhill

http://forums.phoenixrising.me/inde...care-and-treatment-for-people-with-m-e.49345/

That thread is intended to be about the campaign for a Public Inquiry so I don't think it would be appropriate to take this discussion there.

I think @markielock was right when he made this comment earlier:

Concerning the topic of this thread, I think an apt description for it would be discussing the reliability of evidence using Sarah Myhill as an example case.

I take your point though @barbc56. It would be good if we can try to avoid getting carried away with arguing for or against Dr Myhill's approach and instead stay focused on the question of what constitutes reliable evidence. If we cannot manage to do this then I will set up another thread.
 

Laelia

Senior Member
Messages
243
Location
UK
Some of the more irresponsible advocate an abandonment of scientific process altogether and advocate a more "holistic" approach.

Yes, but there is no reason why an approach cannot be both scientific and holistic.

If you are carefree and not risk averse then you could convince yourself that going down this road is something of benefit

Sadly for many I don't think it's a case of being carefree, it's a case of being desperate. :(

Putting aside the ethics of experimenting on humans and all that entails, you could then argue that providing there is a well applied scientific rigour to the design this would still be scientific.

Are you able to elaborate a bit more on what you mean by this? What would this would entail? Sorry if it seems like we're going over old ground again, I do appreciate you explaining things to me.

the fundamental problem is that it's one thing doing DIY supplement and symptom relief at home, it's another being a doctor prescribing treatments for money. There are lots of people who just want to trust a doctor to tell them what to do. That is why Myhill has a greater responsibility than someone blogging self hacks for faecal transplants etc.

I agree with this, of course. And I think many of your criticisms of Dr Myhill are justified.
 

arewenearlythereyet

Senior Member
Messages
1,478
Yes, but there is no reason why an approach cannot be both scientific and holistic.

Sadly for many I don't think it's a case of being carefree, it's a case of being desperate. :(

Are you able to elaborate a bit more on what you mean by this? What would this would entail? Sorry if it seems like we're going over old ground again, I do appreciate you explaining things to me.

As far as I'm concerned there are some elements of holistic practice that sometimes get passed off as science often to add validity to a treatment. One of the key areas is the science of nutrition and digestion. Holistic practitioners like to hijack this science in the belief that food and herbs etc are ancient remedies so somehow "mystical". This often results in laughable pseudoscience, which arrives through pure ignorance of the available actual science. The best bet is to hire a nutritionist with a recognisable scientific qualification who can give you a diet plan not someone who will combine some foul tasting herbs with a massage.

With regards to some of the other more dubious practices if there is real science to back it up, then I would assess this first before believing anything you hear from the holistic practitioner. CBT is after all included in a lot of holistic courses and we know how well that's been effective for ME. That is quite insulting to a scientist to downgrade them to holistic medicine. Sadly as a biologist that's specialised in food science I get this all the time. Quite annoying that a holistic who's spent £250 on some online "nutrition" course can be compared to someone who's studied at university for 4 years, followed by 5 years post graduate research and then spent 25 years applying their craft in a practical sense in the food industry. So no science and holistic practices are not the same thing. I may be biased of course :lol:

I agree desperation does lead to many people trying potentially futile or dangerous treatments that are unregulated and potentially harmful. The key here is to weigh up the risks vs benefits not ignore the risks and hope for the best. My rule of thumb is if the practitioner is charging you for a treatment, they have a vested interest not to give you the prognosis or risks so it is even more important in these cases to tread with caution. Desperation and money making opportunities for the more corrupt go hand in hand sadly.

Experimenting on humans has ethical issues and nowadays (particularly since some of the atrocities during the second world war) this is tightly regulated by codes of practice. Its a massive area for debate, more that I am qualified or experienced to cover. This could seriously derail this thread so I've posted the following which covers the issues quite well

https://sciencebasedmedicine.org/ethics-in-human-experimentation-in-science-based-medicine/
 

Laelia

Senior Member
Messages
243
Location
UK
Why Bogus Therapies seem to work

I don't think that it's correct to refer to the sorts of therapies we have been talking about are 'bogus'. If we had proof that they did not work then that would be fair enough. But this is not the case.

The power of the placebo effect.

You might find this study interesting:

Is the Placebo Powerless? — An Analysis of Clinical Trials Comparing Placebo with No Treatment
Asbjørn Hróbjartsson, M.D., and Peter C. Gøtzsche, M.D.

N Engl J Med 2001; 344:1594-1602May 24, 2001DOI: 10.1056/NEJM200105243442106

BACKGROUND
Placebo treatments have been reported to help patients with many diseases, but the quality of the evidence supporting this finding has not been rigorously evaluated.

METHODS
We conducted a systematic review of clinical trials in which patients were randomly assigned to either placebo or no treatment. A placebo could be pharmacologic (e.g., a tablet), physical (e.g., a manipulation), or psychological (e.g., a conversation).

RESULTS
We identified 130 trials that met our inclusion criteria. After the exclusion of 16 trials without relevant data on outcomes, there were 32 with binary outcomes (involving 3795 patients, with a median of 51 patients per trial) and 82 with continuous outcomes (involving 4730 patients, with a median of 27 patients per trial). As compared with no treatment, placebo had no significant effect on binary outcomes, regardless of whether these outcomes were subjective or objective. For the trials with continuous outcomes, placebo had a beneficial effect, but the effect decreased with increasing sample size, indicating a possible bias related to the effects of small trials. The pooled standardized mean difference was significant for the trials with subjective outcomes but not for those with objective outcomes. In 27 trials involving the treatment of pain, placebo had a beneficial effect, as indicated by a reduction in the intensity of pain of 6.5 mm on a 100-mm visual-analogue scale.

CONCLUSIONS
We found little evidence in general that placebos had powerful clinical effects. Although placebos had no significant effects on objective or binary outcomes, they had possible small benefits in studies with continuous subjective outcomes and for the treatment of pain. Outside the setting of clinical trials, there is no justification for the use of placebos.

http://www.nejm.org/doi/full/10.1056/NEJM200105243442106
 

barbc56

Senior Member
Messages
3,657
Hopefully, this be my last post as people will either use alternative medicine or not. It's a personal choice.
That thread is intended to be about the campaign for a Public Inquiry so I don't think it would be appropriate to take this discussion there
I guess I'm confused about the title. The title of this thread is is called What is Reliable Evidence. The other thread is about Sarah Myhill’s campaign. Why not name this thread What reliable evidence does Dr. Myhill practice?
I don't think that it's correct to refer to the sorts of therapies we have been talking about are 'bogus'. If we had proof that they did not work then that would be fair enough. But this is not the case
I did not write the article nor give it the title. The title doesn't bother me. The content, however, is excellent! I think it's kind of picky to be quibbling about the title. It really has nothing to do with this discussion of what constitutes reliable evidence. Many types of alternative medicine have actually been shown not to work. In the United States there is a institution called the National Institute of Complimentary and Alternative Medicine.
You might find this study interesting:
While your study says that the placebo effect is not as plentiful and I don't know whether this is true or false, it does admit that it does indeed exist. I find that problematic.

I hope I am not repeating myself from the other thread here.

This study had different treatments for patients with Asthma. Some were given inhalers without Albuterol, some acupuncture plus an arm of the study where patients with asthma did get albuterol which is often used for asthma.

Each group reported feeling better. However when physical tests were given, only the group who albuterol actually had improved lung function. So if someone has an asthma attack are you going to give then acupuncture or their albuterol inhaler? I think the answer is obvious.

There is only scientifically proven, evidence-based medicine supported by solid data or unproven medicine, for which scientific evidence is lacking. JAMA
https://sciencebasedmedicine.org/about-science-based-medicine/

.
 

Laelia

Senior Member
Messages
243
Location
UK
Hello @barbc56 :)

Why not name this thread What reliable evidence does Dr. Myhill practice

Because that is not its intended purpose.

I did not write the article nor give it the title. The title doesn't bother me. The content, however, is excellent! I think it's kind of picky to be quibbling about the title.

I thought I was making quite an important point actually, but that's fine if you disagree with me. I might have been guilty of being pernickety on one occasion earlier in this thread, for which I apologised. I don't think this was the case here though.

Many types of alternative medicine have actually been shown not to work.

I'm genuinely interested here. Please can you tell me which ones?
 

barbc56

Senior Member
Messages
3,657
There are too many to list here and it would be too time consuming p. . If you are interested in this, I suggest you start with the Science Based Medicine blog.. They have a search box as well as different categories to look under. .There're also recommended books.

I'm sure there are other sources besides the above.

https://sciencebasedmedicine.org/about-science-based-medicine

I'll let others take over from here.
 

Laelia

Senior Member
Messages
243
Location
UK
If you are interested in this, I suggest you start with the Science Based Medicine blog.. They have a search box as well as different categories to look under.

I had a quick look on the website, it just appears to be just lots of articles, so not easy to find this information.

There're also recommended books.

I'm not that interested to bother reading an entire book about it.. :)

There are too many to list here and it would be too time consuming

Are any of them the types of therapies people are experimenting with widely on here? If so, it would be really helpful if you could just name one or two?
 

Laelia

Senior Member
Messages
243
Location
UK
Are any of them the types of therapies people are experimenting with widely on here? If so, it would be really helpful if you could just name one or two?

Or if you don't know which therapies people are experimenting with widely here then you can just give me any example of an alternative medicine which has been shown not to work. We can use it as an example of how evidence based medicine works to disprove a theory.

@barbc56