• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

BBC Radio 4 - Ben Goldacre's - Bad Evidence

Messages
15,786
A narrow outcome measure such as how much money is saved on disability payments has to be balanced against costs and benefits in other areas. Simply isolating that one figure and optimizing it might cost society and government MORE not LESS in the big picture. Is this being tracked?

But governments and even many for-profit corporations often do focus on the "little picture" even when it's obvious the big picture will suffer. To a large extent, I think it's due to the way these organizations operate - bonuses/appointments/re-elections/etc are structured to reward specific short term goals without accounting for damage done to the wider or longer-term implications.

It's wrong and it's stupid, but as long as the system is set up in such a manner, it doesn't seem particularly fixable, especially since it's the people in charge and benefiting from the stupid policies that have the most power to change those policies.

Are medical costs being taken into account? A broke disabled person might not even be able to get to a doctor or hospital, or be otherwise unable to look after themselves adequately. Hence they might have more medical issues. On the one hand they might die, oh look, thats a big saving to government. On the other hand their health issues might worsen and take up more and more medical resources in an already overstretched system. Is that a saving?

If Simon Wessely had his way, people who have CFS or "Medically Unexplained Symptoms" of any sort and do not comply with CBT, etc to fix them, should not be allowed anything except infrequent routine doctor visits:

In patients with a long history of severely impaired functioning, or who have proven consistently resistant to treatment, management is essentially supportive with infrequent but regular contact. The aim with this approach is to at least reduce further deterioration and limit unnecessary or repeated investigations and treatments.
Reid S, Wessely S. Chronic fatigue syndrome. Conn’s Current Therapy (ed Rakel, Bope). Saunders, 2002, 110-113
The expert consensus is that, once an organic cause for symptoms has been excluded, further examination and investigation should only be initiated if a new symptom develops.
Page L, Wessely S. Medically unexplained symptoms: exacerbating factors in the doctor-patient encounter. J Royal Soc Medicine 2003: 96: 223-227

So in the case of ME/CFS, "resistant" patients would be put into a corner and ignored, along with FM, IBS, repetitive strain injury, post-viral fatigue syndrome, tension headache, tempomandibular joint dysfunction, atypical facial pain, globus syndrome, non-cardiac chest pain, chronic pain, PMS, chronic pelvic pain, hyperventilation syndrome, and multiple chemical sensitivity.

That could save quite a bit of money, especially when doctors are encouraged to avoid "excessive" investigations when diagnosing these conditions.

If you think about it, it's quite brilliant - attribute disease to patient beliefs, devise a treatment that isn't a cure, blame patient beliefs if the treatment doesn't work, withhold medical treatment, withhold permanent disability status, and generally obstruct biological research into these illnesses. And then you can extend these methods to use on "medically unexplained" aspects of known biological diseases which are currently incurable and have some of the same symptoms, and seriously consider the same withholding of doctor access and disability status - after all who knows how functional they'd be able to be if they'd just try harder!

Oh yeah. It's also evil.

BPS is looking more and more like a means to shaft the public when it comes to health care.
 

Firestormm

Senior Member
Messages
5,055
Location
Cornwall England
Am off to listen to the broadcast now: http://www.bbc.co.uk/programmes/b01phhb9


Alex, I don't suppose (apart from cost) there is much to prevent a e.g. disabled charity or pressure group from completing a RCT in order to challenge Government perception. These things need not always originate with the Government. And an RCT (post-hoc) has to be better than a survey as a means of challenging Government policy for example...
 

user9876

Senior Member
Messages
4,556
A narrow outcome measure such as how much money is saved on disability payments has to be balanced against costs and benefits in other areas. Simply isolating that one figure and optimizing it might cost society and government MORE not LESS in the big picture. Is this being tracked?

The chosen outcome measures can affect the value and result of any trial or more generally decision. Years ago I remember being told about the planning for the Victoria Line (London underground) where there initial models and simulations they had didn't make it viable so they changed their metrics and included costs saved due to congestion reduction on roads. This widening of the outcomes made the decision viable.

The message is that whatever trials etc you run often it is the choice of metrics to judge them that will determine sucess. And where there is experiance with techniques being tested a researcher should be able to choose outcomes that meet their goals (unless what they are doing is so bad). Where they get this wrong they end up changing protocols and associated analysis for no good reason - for example changing the definition of harm in PACE.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
Hi Firestormm, I think you are correct. It could be done. It could also be done retrospectively even with ME.

I think if we could get info on ME patients in the long term, some treated one way, some treated another, we could compare the outcome retrospectively in a pilot study. That would give it some credibility and assist with getting funds and support for a prospective study.

As for non-ME issues, it can still be done. However given the wide reach of government changes in a short time, testing government policy is very hard for non-government. Its not like they would easily cooperate - though I wonder if it were presented as a chance to cheaply give government new and better policies if there might be some support.

Bye, Alex
 

Firestormm

Senior Member
Messages
5,055
Location
Cornwall England
Well thus far it is an interesting broadcast and no, RCT's are not being presented as the be-all-and-end-all but as another important tool that should be in the box. Indeed, the UK Government under Gus O'Donnell have allocated 20 million over 10 years to this group to see what impact RCT's might have on established policies. Will maybe try and summarise some main points for those unable to listen to the broadcast - but not today. Need a break and some time in bed unfortunately.

Thanks for the debate :)
 

Marco

Grrrrrrr!
Messages
2,386
Location
Near Cognac, France
Well thus far it is an interesting broadcast and no, RCT's are not being presented as the be-all-and-end-all but as another important tool that should be in the box. Indeed, the UK Government under Gus O'Donnell have allocated 20 million over 10 years to this group to see what impact RCT's might have on established policies. Will maybe try and summarise some main points for those unable to listen to the broadcast - but not today. Need a break and some time in bed unfortunately.

Thanks for the debate :)

All fine and dandy as long as its not a case of "now we've got a 'gold standard' methodology we can put the old brains into neutral".
 

Firestormm

Senior Member
Messages
5,055
Location
Cornwall England
All fine and dandy as long as its not a case of "now we've got a 'gold standard' methodology we can put the old brains into neutral".

What was interesting also was hearing how decisions are made without much recourse to evidence. Thinking about that and applying it to my own career in private business it is perhaps sobering to think how much money can be spent without adequate thought to how effective the decision might be.

I do think that with government policy not enough is done in terms of monitoring, and that RCT's might help to dislodge some notions of 'well we've always done it like that and it seems to be fine'. I mean how often does that happen in life?
 

user9876

Senior Member
Messages
4,556
What was interesting also was hearing how decisions are made without much recourse to evidence. Thinking about that and applying it to my own career in private business it is perhaps sobering to think how much money can be spent without adequate thought to how effective the decision might be.

I do think that with government policy not enough is done in terms of monitoring, and that RCT's might help to dislodge some notions of 'well we've always done it like that and it seems to be fine'. I mean how often does that happen in life?

Much decision making seems to be based on expert opinion and sometimes which expert has the strongest voice. Seems to me that getting away from this is a good thing. Having worked as a civil servant and in a large company I think decision making appears problematic in both settings.

Rather than RCTs I prefer the idea of using mathematical models and simulation to explore a system and possible behaviours. The modelling process forces you to describe the system and various assumptions that are being made. The creation of a formal model is important here in that it is hard to gloss over the difficult to describe pieces - although you can wrap up some effects as stocastic variables. Having written a model you can then try to validate it by collecting data. For example doing a human factors study as to how a process works in practice rather than how it is specified. Using the right modelling language the model can be run and hence the model can be explored under different conditions or with different changes. This can help in exploring the decision surface for example looking for sharp cliffs suggesting the effects of a decision may be very subseptable to changes in the environment.

We should of course not forget that a model is a tool to help thinking and not an accurate rendition of a system. I guess a RCT would claim to be an accurate rendition of a system but one where it is hard to manipulate many different factors. Hence results from an RCT could be used to help validate a model but equally a model may suggest factors to control for within a trial. I come from an engineering background which I suspect is where my preference for modelling comes from.
 

Firestormm

Senior Member
Messages
5,055
Location
Cornwall England
I think as I said at the very start - one thing that stands against a preference to thoroughly check everything out - is time (the other being money). Of course in politics I suspect ideology holds greater sway but at the end of the day a decision must be made and one can only do so much. But something is better than nothing and if that something includes an appropriate RCT it would seem preferable. Still have to decide yea or nay and move forward - can't procrastinate forever. Tempted to say that that's what scientists are for :)
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
I think as I said at the very start - one thing that stands against a preference to thoroughly check everything out - is time (the other being money). Of course in politics I suspect ideology holds greater sway but at the end of the day a decision must be made and one can only do so much. But something is better than nothing and if that something includes an appropriate RCT it would seem preferable. Still have to decide yea or nay and move forward - can't procrastinate forever. Tempted to say that that's what scientists are for :)

This is the rationalists dilemma. To be fully rational you have to find out as much as you can and then carefully deliberate. In nearly all situations this is not feasible, and quite often is impossible. At some point you just have to act, and deal with the uncertainty. That is why act and respond cycles are important - do, learn, adapt, repeat. The irony is that to be fully rational you have to be irrational far too often. Where rationalism has the edge is in the learning and adaptation, but you still have to do stuff in the first place.
 

barbc56

Senior Member
Messages
3,657
I came across an online book on evidence based medicine - I've not read it but thought people might be interested hence i'm posting a link

http://eprints.kmu.ac.ir/2225/1/Evidence-Based Medicine in its Place.pdf

Thanks for this reference. I plan to read it. I hope it goes into the evidence based vs. scientific based medicine.

Evidence Based Medicine can be a very narrow view of how science works. Science based medicine as I have said earlier in other posts goes a step further.

Alternative medicine sometimes uses EBM to prove implausible theories. Followers of EBM say, "Just look at the evidence" but fails to go beyond that

The classic absurd unintended consequence of EBM is that a really bad idea can easily be made to seem as though it is perpetually on the verge of legitimacy, no different than a much better idea that simply hasn’t yet been validated by research.
For instance, a quack selling sharp-stick-in-the-eye therapy could use pitch perfect EBM-speak to say technically correct but patently ridiculous things like this: “There is insufficient evidence to support the use of sharp sticks in the eye to treat blindness, so we should keep studying it!” Prominent EBM publications like The Cochrane Collaborationand The Natural Standard contain wonderful real-world examples: they routinely publish remarkably uncritical statements about therapies that have never yet had any evidence supporting them and — importantly — almost certainly never will.
http://saveyourself.ca/articles/ebm-vs-sbm.php