• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

What constitutes reliable evidence?

RogerBlack

Senior Member
Messages
902
The authoritative British Medical Journal has conducted an investigation into 2,500 of the most prescribed regular medical treatments and medicines. The results have been published in the Clinical Evidence Handbook. And these outcomes, done by the regular medical world themselves, are very worrying. Medicine has been using drugs for many years, of which only 12% could have a positive effect.

My understanding is that this is not quite correct.
12% of treatments are in the "very good evidence of efficacy" category.
This is not the same as 12% of people treated.
22% were 'likely to be beneficial', and about 50% were 'unknown'.

The unknown includes medicines and treatments for which there is mixed research outcomes.

However, in many cases, trials are a problem, because people respond to different medicines differently, and what actually works is to try people on different treatments till one works for them.

There are few trials that do 'Taking these five commonly available treatments, try patients on each in sequence and see what works for them'.
It would be really nice if actual patient results could be folded into evidence, to provide the gold standard.
But generally they aren't.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
There are few trials that do 'Taking these five commonly available treatments, try patients on each in sequence and see what works for them'.
I regard this as a huge problem. It has its reasons though, such as those treatments may belong to different and competing drug companies, or doing that is against the isolate and test principle. However I do think that we should be testing such sequences as a matter of clinical utility and evidence. It may be that here is a good opportunity to get away from RCTs and start looking at other cheaper study methodologies, given that drug companies are unlikely to put any funding into this.
 

Jonathan Edwards

"Gibberish"
Messages
5,256
So, are you Dawkins and your colleagues liars or have you been misleaded as well by the methode of science for twenty five years? I know the great Einstein has been wrong also about many things. Is this how science works?

Lying about what. All that we are saying is that if a treatment is reliably shown to work it is no longer considered 'alternative'. This has nothing whatever to do with whether or not treatments used by doctors have all been reliably shown to work. There is no such thing as 'medicine' that uses drugs. People use drugs and some of those have reliable evidence for effectiveness and some do not. So there is no point in making a distinction between 'orthodox' and 'alternative' medicine. All that matters is whether the treatment has been shown to work.
 

Jonathan Edwards

"Gibberish"
Messages
5,256
My understanding is that this is not quite correct.
12% of treatments are in the "very good evidence of efficacy" category.
This is not the same as 12% of people treated.

Yes. I have not looked at this list but it is probably pretty inevitable that it will look like that. Let us say there is a drug X, such as methotrexate, for which there is very good evidence for efficacy in rheumatoid arthritis. Then there is almost certainly rather doubtful evidence for the efficacy of X in seventy three other less common conditions (I can think of a dozen straight away). There are twenty times as many less common conditions than there are common ones and by and large reliable evidence is hard to get for drug effects in less common conditions. But there are many fewer patients with the less common conditions, obviously. At least eighty percent of the patients in my clinics had one of the half dozen common conditions I dealt with. So 12% might be about the best you could hope for and might well mean that most episodes of treatment are based on the best evidence.
 

Jonathan Edwards

"Gibberish"
Messages
5,256
I regard this as a huge problem. It has its reasons though, such as those treatments may belong to different and competing drug companies, or doing that is against the isolate and test principle. However I do think that we should be testing such sequences as a matter of clinical utility and evidence. It may be that here is a good opportunity to get away from RCTs and start looking at other cheaper study methodologies, given that drug companies are unlikely to put any funding into this.

Except that you do not really need trials for 'Taking these five commonly available treatments, try patients on each in sequence and see what works for them'. That is what a rheumatologist does in the clinic every week of the year. And there is no 'result' that needs documenting in the way that it does in an efficacy trial. But unless you know that the drug has an effect at all, from a controlled trial, you are completely at sea trying drugs out in sequence because people may just get better by chance while taking one or other. Once you know that each of five drugs is effective in a proportion, above placebo levels, then it is just logical to try them in sequence and we all do. But you don't learn anything generalisable. You just discover which person benefits from which drug. You might then look for response biomarkers but often there are no reliable ones.
 

Jonathan Edwards

"Gibberish"
Messages
5,256
It would be really nice if actual patient results could be folded into evidence, to provide the gold standard.
But generally they aren't.

Does that actually make sense, Roger? Actual patient results are the basis of all controlled trials. Individual responses are not generalisable, so provide no knowledge, unless they can be used to identify biomarkers, but identifying biomarkers takes you back to the controlled trial methodology.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
But unless you know that the drug has an effect at all, from a controlled trial, you are completely at sea trying drugs out in sequence because people may just get better by chance while taking one or other.
Of course. I am presuming that every single method used in such a trial has had multiple RCTs precede it.

Expert evidence is good but its not reliable. However there must be a huge number of case studies potentially available from clinical practice records.

One of the reasons I am concerned with this is less to do with doctors but more to do with medical bureaucracy, including private insurance. There are lots of potential treatments not being paid for. Doing the kind of study I am suggesting means we can get a grip on costing versus efficacy data, and fight bureaucratic cost managers aiming to deny treatment. It gives us harder evidence than just expert opinion.
 

Laelia

Senior Member
Messages
243
Location
UK
I don't think that it's correct to refer to the sorts of therapies we have been talking about are 'bogus'. If we had proof that they did not work then that would be fair enough. But this is not the case.

Many types of alternative medicine have actually been shown not to work.

Are any of them the types of therapies people are experimenting with widely on here?

Hi @barbc56

I thought you (and others) might be interested to hear that in lecture 8 of the Science Based Medicine lecture series Dr Hall also uses the word 'bogus' when referring to a number of detoxification methods. Some of these detox methods are recommended by Dr Myhill and are used widely by people on here so this is particularly interesting.

From the lecture:

Except in cases of poisoning, the body doesn't need any help, the liver and kidneys do a fine job by themselves. Toxins are constantly being injested and they're also and they're also being constantly created in the body by metabolic processes. The liver breaks down toxins and prepares them for excretion in bile or urine then they're eliminate.

Bogus Detoxification
  • Colon cleansing
  • Infrared saunas
  • Foot pads and footbaths
  • Gerson therapy
  • Oil pulling
  • Scientology's Purification Rundown
  • Chelation*
  • Maple syrup/cayenne pepper/lemon juice
  • Removal of dental fillings
None of these will do you any good and some of them are likely to harm you.

*Except for chelation of heavy metal poisoning which Dr Hall recognises as science based.

The last sentence seems like quite a bold statement to me "none of these will do you any good". Is Dr Hall really able to prove this? She didn't provide any evidence in the lecture to back up her claim. Is it the case that Dr Hall really has no evidence or did she just forget to mention it?
 
Last edited:

Laelia

Senior Member
Messages
243
Location
UK
Yesterday watched the first video only. And it did remind my to always check the science again, the data and methodology itself, to see if science is really settled in one point.

Hello pamojja,

After watching the first video do you feel any differently about collecting all the information from anecdotal recovery stories on here?
 

RogerBlack

Senior Member
Messages
902
Does that actually make sense, Roger? Actual patient results are the basis of all controlled trials. Individual responses are not generalisable, so provide no knowledge, unless they can be used to identify biomarkers, but identifying biomarkers takes you back to the controlled trial methodology.

I should have elaborated that this could only really help in some cases.

Individual responses are not generalisable in the absence of biomarkers to select the drug that will have effect.

However, for conditions with comparatively large effect sizes and measurable responses measures like:
What percentage of patients end up well controlled disease after all available treatments have been tried;
Effectiveness in the general population with co-morbidities;
Comparative efficacy and better choice between drugs. Is the best order ABCD, or ADCB or ... to minimise number of drugs tried;
Does this differ in the elderly, or female, or ...

This is only 'simple' if there are a spectrum of drugs to be chosen between, with no clear and obvious winner, and there is a large measurable outcome, and the treatment order is sufficiently randomised that you can disambiguate 'got better on their own' as an outcome.

For many drugs, perhaps especially with CFS, effects seem to be small at best, or help a small portion of the patient population, that a 'big data' type approach above is going to be problematically confounded, perhaps almost hopelessly, to the point of observational dietary studies of single foods.

Done wrong, it could add another impersonal and rushed-through obligation on clinicians, and not actually help much.

The data scientist in me says that everyone should get a fitbit, or similar, and analysable medical records, and we should try to measure actual, not surrogate responses as well as possible.
The realist (pessimist?) suspects it's not that simple.

Though reading above, the point that if effect sizes are large, you just try them anyway is not unreasonable.
 
Last edited:

pamojja

Senior Member
Messages
2,384
Location
Austria
Hello pamojja,

After watching the first video do you feel any differently about collecting all the information from anecdotal recovery stories on here?

As long as the scientific evidence of all major chronic multi-factorial diseases show validated available allopathic treatments have only a 1-3% 5-year mortality reduction, I'm not satisfied and look for anecdotal evidence for other treatments. Which most often are not patentable and therefore will never be trialled - and as you know helped me to have already remission with such a non-reversible chronic disease.

As soon I hear of a treatment for a specific chronic disease concerning me with more than that meager outcome, I'm all ears. But as it is - like with my version of CAD already in remission on account of pure anecdotal evidence (Linus Pauling's therapy), where even a century of research and all funds available did find nothing (beyond 1-3% 5-year reduction of mortality) - I simply don't have the time to wait..
 
Messages
1,478
Hi @barbc56

I thought you (and others) might be interested to hear that in lecture 8 of the Science Based Medicine lecture series Dr Hall also uses the word 'bogus' when referring to a number of detoxification methods. Some of these detox methods are recommended by Dr Myhill and are used widely by people on here so this is particularly interesting.

From the lecture:



The last sentence seems like quite a bold statement to me "none of these will do you any good". Is Dr Hall really able to prove this? She didn't provide any evidence in the lecture to back up her claim. Is it the case that Dr Hall really has no evidence or did she just forget to mention it?

She doesn't need to provide evidence @Laelia her evidence is that the human body does the job well and we understand it. There are lots of evidence for this. Your question is a bit like saying prove to me the earth isn't flat? It's up to the alternative practice people to provide evidence that their often invasive procedures they recommend do more good than harm. Perhaps turn the question around? Where is the proof that these practioners aren't doing harm or just conning patients out of money?

In fact let me lay down a challenge :

Provide me with 5-15 evidence based papers published in reputable journals on each of the above procedures that show what the evidence is that they work and what the risks are for trying them out?
 

Laelia

Senior Member
Messages
243
Location
UK
As long as the scientific evidence of all major chronic multi-factorial diseases show validated available allopathic treatments have only a 1-3% 5-year mortality reduction, I'm not satisfied and look for anecdotal evidence for other treatments.

Yes that's what I thought you would say. I think you're doing the right thing too :)

This is a comment you made on your own thread:

"The same disheartening ignoring is going on whenever a cancer patient has beat his disease, just filed as 'spontaneous remission', without even inquiring what successful patients could have done differently."

After spending some time on these forums you might now understand why this happens. Anecdotal information from patients and doctors is always considered 'unreliable' by those in medical research and this is why it is ignored.
 

Laelia

Senior Member
Messages
243
Location
UK
She doesn't need to provide evidence @Laelia her evidence is that the human body does the job well and we understand it. There are lots of evidence for this.

One could use exactly the same argument as a reason not to provide any evidence to prove that a treatment works (which I don't think you would find acceptable).

Your question is a bit like saying prove to me the earth isn't flat?

Nonsense!

In fact let me lay down a challenge :

Provide me with 5-15 evidence based papers published in reputable journals on each of the above procedures that show what the evidence is that they work and what the risks are for trying them out?

Why should I have to be the one to do this?! I'm not the one making the claims that they work!!
 
Last edited:

pamojja

Senior Member
Messages
2,384
Location
Austria
After spending some time on these forums you might now understand why this happens. Anecdotal information from patients and doctors is always considered 'unreliable' by those in medical research and this is why it is ignored.

I kindly disagree. Only when it's not patentable and therefore able to be brought to the market only at a huge loss (shareholders wouldn't agree too).

If it is patentable, then it is of course not ignored. In some case even tried to get the inventor prosecuted and the patent blatantly stolen, as it happened to Burzynski (in that case even without any evidence that it works, just the promise of profits):

http://www.burzynskimovie.com/typography/chapter-9-of-10-sourced-transcript/#.WP9PtTekL8g
 

Laelia

Senior Member
Messages
243
Location
UK
I kindly disagree. Only when it's not patentable and therefore able to be brought to the market only at a huge loss (shareholders wouldn't agree too).

If it is patentable, then it is of course not ignored. In some case even tried to get the inventor prosecuted and the patent blatantly stolen, as it happened to Burzynski (in that case even without any evidence that it works, just the promise of profits):

http://www.burzynskimovie.com/typography/chapter-9-of-10-sourced-transcript/#.WP9PtTekL8g

You're right of couse! Thank you for pointing out my mistake. :)
 

Laelia

Senior Member
Messages
243
Location
UK
Anecdotal information from patients and doctors is always considered 'unreliable' by those in medical research and this is why it is ignored.

I think that is a mistake by the way (just in case I didn't make that clear!)
 

Laelia

Senior Member
Messages
243
Location
UK
This is another statement made by Dr Hall in the first lecture when she was explaining the reasons why we come to believe that ineffective treatments work:

Your symptoms might have gone away for other reasons that had nothing to do with the treatment. You have no way of knowing if your symptoms might have gone away if you hadn't used any treatment at all.

@pamojja

What would your response be if someone said the above to you?
 

Jonathan Edwards

"Gibberish"
Messages
5,256
I think that is a mistake by the way (just in case I didn't make that clear!)

Dear Laelia,
You are trying very hard to prove that the 'establishment' view of reliable evidence is wrong, but in doing so you are playing around with word meanings in a way that makes no sense.

Consider a perfectly designed randomised double blind controlled trial. The first patient to get treatment returns saying they are 70% better and the blood work is 70% better. The researcher goes to the pub and says to his friend - 'things are looking interesting, our first patient got 70% better'. His friend says 'yes but that is just one patient, just an anecdote told to me by a doctor who has been told by a patient'. So the researcher says 'gee, your right, I'll chuck the records in the bin and try again tomorrow'. So he ends up chucking all the data in the bin because it is anecdotal and therefore 'unreliable'.

The point is that the opposite of reliable evidence for efficacy of a treatment is the absence of reliable evidence for a treatment, it is not 'unreliable evidence'. Fluge and Mella did not consider the fact that one of their lymphoma patients got better for ME 'unreliable evidence'. They thought it might be very important. But they, very rightly indeed, considered that on its own it did not constitute reliable evidence for efficacy. They still do not think they have reliable evidence for efficacy having done two major trials.