What constitutes reliable evidence?

[barbc56]

Quite so. Having your car break down because its last service was done very incompetently, does not invalidate the need for good quality servicing.

Which reminds me of this quote:

Just because there are problems with aircraft design, that doesn’t mean magic carpets really fly,
Ben Goldacre

The following is for those who are interested in reading more about this subject.

I would highly recommend reading this first as it tells how Britt Hermes, a naturalpathic doctor who started to question the basis of her training. It's absolutely fascinating.
https://sciencebasedmedicine.org/nd-confession-part-1-clinical-training-inside-and-out/

She has now started her own blog Naturalpathic Diaries; Confessions of a Naturalpathic Doctor.
https://www.naturopathicdiaries.com/

Now, I'm really out of here.

 

[TiredSam]

I would highly recommend reading this first as it tells how Britt Hermes who was a naturalpathic doctor who started to question the basis of her training. It's absolutely fascinating.
https://sciencebasedmedicine.org/nd-confession-part-1-clinical-training-inside-and-out/

That's really interesting, I've only just started reading it and I'm already seeing a lot of parallels to the BPS brigade:

Naturopathic education exists in a bubble without critical oversight
The issue of this deceit boils down to the education and clinical training of naturopaths. The naturopathic profession perpetuates a series of false assertions to justify its advancement, which unfold in a closed-loop system that eschews external criticism. Two examples of this dynamic show that naturopaths are the sole regulators of naturopathic medicine.

A proximal consequence of this insulated community is that naturopathic education remains mysterious to outside observers, and is falsely presented to the public as being on-par with MDs or DOs and capable of training primary care physicians. The ultimate consequence is that pseudoscience becomes legalized as medicine through political maneuvering.

Sound familiar?

 

[Barry53]

The key finding from the study is that placebos have no effect on any objective measurements, only subjective ones.

... as Jonathan Edwards highlighted in his recent paper. It is why you need to have objective measures where at all possible, or at the very least not combine subjective measures with placebo-etc-prone methodology.

 

[Barry53]

Maybe we should start a new thread on that.

NOOOOOooooooooooooo !!! ...

 

[Laelia]

If you read carefully my posts and listened carefully to Dr Hall you would see that these statements are not misleading because they do not suggests what you suggest they would suggest. In fact Dr Hall and I have explicitly indicated that we do not suggest what you are suggesting we suggest.

I see little point in distracting the debate from this simple fact by raising straw men about what people seem to be suggesting when if you read carefully they have said they are not!!

Dear Jonathan,

I was not suggesting that it was your or Dr Hall’s intention to mislead, I'm very sorry you thought that.

Part of the reason why I pointed these things out was because your earlier comment had left me genuinely confused, which would have been obvious to anyone who was following the conversation!

I think you are conflating reliability of a piece of evidence with reliability of the body of available evidence for a treatment's efficacy.

I don't think so. Presumably there is a spectrum of reliability when considering the body of evidence too.

None of this discussion is ever going to raise current treatments provided by private practitioners for ME/CFS to the status of 'has reliable evidence for efficacy', not even rituximab.

No and that is not the purpose of this discussion. I hope we can use this thread in a positive way to educate ourselves and others as well as to encourage critical thinking.

I am sorry again for the misunderstanding.

 

[Barry53]

I think when talking about reliable (trustworthy) evidence, you need to be very careful about whether all the right tests are being applied for providing said evidence. Not just the right tests, but all the right tests ... omissions can be disastrously misleading.

I was once told a story (no idea of its truth but it illustrated the point) of sea trials being done on a new ship-board missile system. One of the missiles misfired, so the ship turned around to return to base ... and the missile exploded in its launch tube. Apparently the missile had (as I presume they all do these days) a new safety mechanism (flawed in this case!) whereby after launch, if the missile went out of control and headed back towards the ship, it self-destructed. Unfortunately in this case the system failed to realise the missile had not left the ship in the first place.

Although this sea trial did rather spectacularly prove the system was not safe, preceding trials should have already avoided it happening. The preceding trials would have inevitably indicated "reliable evidence" to safely move on to sea trials, but at least one crucial aspect of testing was overlooked. What seemed like reliable evidence from earlier trials was in fact deeply unreliable, because the test specification had a huge hole in it.

It is usually quite easy to specify a raft of tests that need doing to "prove" something, but invariably much harder to then identify what crucial tests are still missing. Quality peer reviewing typically plays its part here, because spotting what is not there often benefits from lateral thinking and collaborative discussions.

 

[Barry53]

That's really interesting, I've only just started reading it and I'm already seeing a lot of parallels to the BPS brigade:

Sound familiar?

We are back to religion again - closed belief systems.

 

[Barry53]

Which reminds me of this quote:


The following is for those who are interested in reading more about this subject.

I would highly recommend reading this first as it tells how Britt Hermes, a naturalpathic doctor who started to question the basis of her training. It's absolutely fascinating.
https://sciencebasedmedicine.org/nd-confession-part-1-clinical-training-inside-and-out/

She has now started her own blog Naturalpathic Diaries; Confessions of a Naturalpathic Doctor.
https://www.naturopathicdiaries.com/

Now, I'm really out of here.

Just had a brief look for the moment, but this struck a chord ...

I am saddened to report that not only was I misled, but so were hundreds of legislators, thousands of students, and tens of thousands of patients.

I think a time will come when a great many high integrity professionals will realise how they also have been duped by PACE etc.

 

[CFS_for_19_years]

I would highly recommend reading this first as it tells how Britt Hermes, a naturalpathic doctor who started to question the basis of her training. It's absolutely fascinating.
https://sciencebasedmedicine.org/nd-confession-part-1-clinical-training-inside-and-out/

She has now started her own blog Naturalpathic Diaries; Confessions of a Naturalpathic Doctor.
https://www.naturopathicdiaries.com/

Now, I'm really out of here.

Thanks for posting these. She talks about her training at Bastyr in Seattle, where I live. I've gone there a few times and been underwhelmed. I'm looking forward to reading everything she has to say about it, if only to be on guard in the future.

 

[Laelia]

... as Jonathan Edwards highlighted in his recent paper. It is why you need to have objective measures where at all possible, or at the very least not combine subjective measures with placebo-etc-prone methodology.

I must read this paper of Jonathan's. I heard it was rather brilliant.

 

[Barry53]

I must read this paper of Jonathan's. I heard it was rather brilliant.

I would suggest it should be required reading for this thread of yours

 

[alex3619]

Thank you Alex. Aren't case studies based largely on anecdotal evidence? Does that mean that you think that anecdotes can sometimes provide reliable evidence?

No. A single case is anecdotal. Series of cases or series plus controls, and quite a few other designs, are considered higher levels of evidence.

This is one way to look at it. There are others. Studies need to be examined on technical grounds, but also whether or not they make any sense (which seems to be lacking in some analyses based only on technical grounds). As you can see, RCTs are no longer at the top. Its also important to realize RCTs only apply to clinical interventions. Other aims are not addressed by RCTs.

Its all about minimizing bias, which may give you an estimated risk of error.

However I do not regard scientific analysis and evidence based analysis as synonymous. Too much pseudoscience is in evidence based reviews, and it should be summarily rejected. EBM is also as much about making managerial decisions as scientific ones. So its not accurate to call the entire thing scientific. Its also not accurate to presume that a purely scientific analysis can address all the needs that EBM is for.

There are a huge number of problems in EBM. That would be a different topic however.

 

[alex3619]

Sound familiar?

Ummm, I think its been familiar since the early part of last century. Recall Freud? Once he and his followers got a grip in the USA the game was afoot.

 

[Laelia]

I would suggest it should be required reading for this thread of yours

Yes it should be

Perhaps we need a reading list to accompany this thread. Does anyone have any more suggestions for recommended reading?

 

[arewenearlythereyet]

Yes it should be

Perhaps we need a reading list to accompany this thread. Does anyone have any more suggestions for recommended reading?

I like this. It's part of the Christmas lectures. Debunks myths in food.

http://richannel.org/christmas-lectures/2005/2005-john-krebs

I like the bit about oranges and vitamin c starting around 9 minutes in

 

[Laelia]

I like the bit about oranges and vitamin c starting around 9 minutes in

Which video are you referring to? (your link takes us to all the videos, not any one in particular)

 

[arewenearlythereyet]

Which video are you referring to? (your link takes us to all the videos, not any one in particular)

Sorry the one I was watching was "when food goes wrong". It's worth watching the others though since it helps put context to much of the nonsense around the food industry, additives, nutrition and allergies ....much of these subject fields have been hijacked by so called naturopaths.

 

[Gijs]

This is not a 'quote'. It is what I and most of my colleagues have been saying to my students for twenty five years at least. Dawkins has just paraphrased what everyone else has been saying (what he tends to do). Dr Hall's version is the usual one.

The authoritative British Medical Journal has conducted an investigation into 2,500 of the most prescribed regular medical treatments and medicines. The results have been published in the Clinical Evidence Handbook. And these outcomes, done by the regular medical world themselves, are very worrying. Medicine has been using drugs for many years, of which only 12% could have a positive effect. The rest is quackery or even life-threatening. So, are you Dawkins and your colleagues liars or have you been misleaded as well by the methode of science for twenty five years? I know the great Einstein has been wrong also about many things. Is this how science works?

 

[slysaint]

 

[arewenearlythereyet]

I like the quote in response to "science doesn't know everything"
"well science knows it doesn't know everything....otherwise it would stop!"