• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Testing Treatments: better research for better health care (free book - available in 10 languages)

Dolphin

Senior Member
Messages
17,567
Following a recommendation by Ben Goldacre in his book Bad Science, I just read the book:

Testing Treatments: better research for better health care
Imogen Evans, Hazel Thornton, Iain Chalmers, and Paul Glasziou

Here's a description:

How do we know whether a particular treatment really works? How reliable is the evidence? And how do we ensure that research into medical treatments best meets the needs of patients? These are just a few of the questions addressed in a lively and informative way in Testing Treatments. Brimming with vivid examples, Testing Treatments will inspire both patients and professionals.

Building on the success of the first edition, Testing Treatments has now been extensively revised and updated. The second edition includes a thought-provoking account of screening, explaining how early diagnosis is not always better, and a new chapter exploring how over-regulation of research can work against the best interests of patients. Another new chapter shows how robust evidence from research can shape the practice of healthcare in ways that allow treatment decisions to be reached jointly by patients and clinicians.

Testing Treatments urges everyone to get involved in improving current research and future treatment, and outlines practical steps that patients and doctors can take together.

It can be read in 10 languages on this site:
www.testingtreatments.org

The 10 languages are:
 

Dolphin

Senior Member
Messages
17,567
I've read more medical research than most people (probably over 500 papers) so it wasn't as informative to me as I would have liked.

However, it was interesting enough to read different examples of how medical practice has changed at different stages.

Also, there were some points (e.g. about patient preferences and the importance of patient involvement) that I feel could be made more in the ME/CFS field.

Many things encouraged me to remain involved as a patient who comments on research.

I'll post some extracts over the next day or two that I underlined. However, other people may find other bits of interest.
 
Last edited:

Dolphin

Senior Member
Messages
17,567
These extracts are from the first edition in English from http://www.jameslindlibrary.org/pdf/testing-treatments.pdf but the second edition is also now available http://www.testingtreatments.org/wp-content/uploads/2012/09/TT_2ndEd_English_17oct2011.pdf

Sometimes I just underlined a short bit but have added some surrounding text to help give context.

We dedicate this book to William Silverman (1917–2004), who encouraged us repeatedly to challenge authority.

FOREWORD

Medicine shouldn’t be about authority

Sometimes evidence can be distorted through absent-mindedness, or
the purest of motives, for all that motive matters. Doctors, patients,
professors, nurses, occupational therapists, and managers can all become
wedded to the idea that one true treatment, in which they have invested
so much personal energy, is golden
.

Equally, the way that evidence is communicated can be distorted,
and misleading. Sometimes this can be in the presentation of facts and
figures, telling only part of the story, glossing over flaws, and “cherry
picking” the scientific evidence which shows one treatment in a particular
light.

FOREWORD TO THE FIRST PRINTING

In practice, good medicine routinely requires
good guesswork. But too often in the past many medical professionals have
blurred the distinction between guessing and good evidence. Sometimes they
even proclaim certainty when there is really considerable doubt
.

If we are passive consumers of medicine we will never drive up standards. If we prefer simplistic answers we will get pseudoscience. If we do not promote the rigorous testing of treatments we will get pointless and sometimes dangerous treatment along with the stuff that really works.
 

Dolphin

Senior Member
Messages
17,567
Intro

We have been colleagues for many years, sharing not only professional
camaraderie but also a deep-seated conviction that medical treatments, whether
new or old, should be based on sound evidence
.

she [author who is a patient] ought to be an active participant in the quest for progress concerning her treatment, not a more or less passive recipient of care.

Since then he has strenuously promoted the view that
decisions in healthcare should be informed by unbiased evidence from relevant
research, particularly the results of systematic reviews of controlled trials.

We hope that you, the reader, will emerge from this
book sharing some of our passion for the subject and go on to ask awkward
questions about treatments, identify gaps in medical knowledge
, and get involved
in research to find answers for the benefit of everybody.
 

Dolphin

Senior Member
Messages
17,567
Chapter 3:

Overall, the greater the element of subjectivity in assessing outcomes, the greater is the need for blinding to make tests of treatments fair.

Even when bias might creep in – for example in assigning a cause of death – this can be done by people who do not know which treatments individual patients received.

Commercial interests are not alone in leading to biased selection from the available evidence for inclusion in reviews. We all have prejudices that can do this – researchers, health professionals, and patients alike.
 

Dolphin

Senior Member
Messages
17,567
Chapter 4-6 (I lost track):

In this chapter, we want to consider uncertainties of this kind – that is, when there is inadequate information about the effects of alternative treatments, and there are no strong patient preferences. [Comment: patient preferences don't get talked about much in the ME/CFS literature e.g. re: CBT and GET]

(Gives an example of patient preferences)
We know that the choice of treatments for BPH depends on preferences and trade-offs: individual men differ in their personal assessment of the risks and benefits. Those who choose surgery have the best chance of relieving their symptoms but face the risk of complications such as incontinence, retrograde (backwards) ejaculation, and impotence. Drug treatment is less successful at relieving symptoms but avoids the risks associated with surgery.

Then they must discuss the options with their patients and be as clear as possible about the patients’ preferences.

PATIENTS’ REACTIONS TO UNCERTAINTY
In 2002, a 58-year-old woman from New Jersey, USA, commented:
‘Sadly, in my experience anyway, I have found it impossible to have a rational
conversation with a physician, where my concerns are respected on the topic of
mammograms, as the New York Times article says a patient should have.
Doctors get belligerent and almost hostile if I say I have reservations about
getting a yearly mammogram. The upshot is that I don’t feel I have a good
relationship with a physician, and that is not good. A good scientist is not afraid
to express uncertainty on a topic or to discuss a topic openly
. I’m afraid the
practising physicians who I have come across do not have that scientific
mindset.

Diane Palacios in correspondence with Dr D A Berry of the MD Anderson
Cancer Center, University of Texas, 2002 (reproduced with permission).

Ms Palacios also wrote a letter directly to theNew York Times, concluding I can live with uncertainty. I do not wish to live with dishonesty’.
Palacios D. Re: Senators hear from experts, then support mammography
(news article March 1). New York Times, 2002 Mar 4, pA20.

Writing in 2005, a British junior doctor who had lately become a patient herself
noted:
‘Having worked in the NHS I had realistic expectations with regard to waiting, the
uncertainties of diagnosis, and the time pressures on doctors. What I valued
above all was being diagnosed and treated by individuals I could trust – by
doctors who laid out the facts and uncertainties as known

The problem vexed a British paediatrician over 30 years ago when he
pithily observed that he needed permission to give a treatment to half his
patients (that is, to find out about its effects by giving half the patients the
new treatment and half the existing treatment in a controlled comparison),
but not if he wanted to give the treatment to all of them as a
standard prescription.58

[A doctor]
I’m more likely to be able to help if I can get to know more about you and your priorities and preferences.

Consumer commentaries on trials in the register could cover, for
example, the importance of the questions being addressed, whether these had
already been answered satisfactorily by previous research, whether the design
of the study was scientifically and ethically robust
, whether the primary
outcomes chosen mattered to patients
, and whether arrangements were in
place for communicating the results of the research to those who had
participated in it.
 

Dolphin

Senior Member
Messages
17,567
Chapter 5-6 (I lost track):

In earlier chapters we emphasised why research must be designed properly and address questions that matter to patients and the public.

Another problem concerns selective reporting of
results within published trials – that is, some of the results are excluded
because they do not support the trialists’ or funders’ interpretation of the
effects of the treatments being tested. This is indefensible. But why is
biased under-reporting of research so important?

As the evidence mounted, it became clear by the
mid-1970s that antibiotics did indeed reduce the risk of death after surgery.
Even so, trials continued to be approved by ethics committees and done by
researchers throughout the 1980s to address the very same question. As a
result, half the patients in these later studies were denied a treatment that
had been shown to reduce their risk of dying after operations. How could
this have happened? The most likely explanation is that trialists embarking on the later studies did not review the accumulated evidence systematically, or present the results of new research in the context of
an up-to-date review of all the relevant evidence
. Clearly, research ethics
committees had not required researchers to have done so before approving
the new studies
. In other words, neither researchers nor ethics committees
had put the interests of patients first.82

Researchers in Bristol decided to pose a fundamental question: ‘To what extent are questions of importance to patients with osteoarthritis of the knee and the clinicians looking after them reflected in the research on this condition?

How many other fields of therapeutic research would, if evaluated in this way, reveal similar mismatches between the questions about treatment effects that matter to patients and clinicians, and those that researchers are addressing?

Belatedly, some researchers have begun to realise the importance of working with patients, carers, and the public to find out about their experiences of illness and healthcare.

Thus, in the trial, sensitive exploration of patients’ views in preliminary studies, rather than the imposition of trialists’ design ideas, led to an acceptable and feasible way to compare the treatment options. An acceptable study seems likely to recruit in a shorter time, thereby identifying the good and bad effects of treatments more quickly for patients and those treating them.

Some of the other reward systems within universities also lead to
research being done for the wrong reasons. As a former editor of the New
England Journal of Medicine noted: ‘large-scale multi-institutional clinical
trials provide less opportunity for authorship than individual or small group research’.
105
Authorship of research reports is highly prized within
universities as a condition of personal academic advancement and a measure of institutional success and continued funding. As a result, researchers
and the institutions in which they work regard multicentre, collaborative
research projects – which are often published in the name of a group – as a
threat to individual recognition and kudos. So, studies continue to be done
by individual researchers and small teams and are often not big enough to
yield reliable results that are important to patients
.

This distorted research agenda raises worrying questions, not only
about the research that does get done and reported, but also about the
research that does not get done at all. The ‘opportunity cost’ of these
tendencies is substantial – many questions about the effects of treatments
intended to improve health are not being addressed because they do not
interest either the commercial sector or universities.

Yet this study faced some opposition from industry and some university researchers. Why? They were engaged in commercial trials assessing the effects of expensive new drugs – so-called neuroprotective agents – on outcome measures of questionable importance to patients, and they did not wish to face competition for participants.

This example illustrates the crucial importance of addressing questions
which are of no interest to industry or universities: failure to do so can
result in patients being harmed. There is currently inadequate funding
in the UK for clinical trials done independently of industry.

Another reason for tackling these unanswered questions is to help
ensure that the precious resources available for healthcare are not being
wasted.
 

Dolphin

Senior Member
Messages
17,567
Chapter 7 to end:
--

The Bristol research also showed how facilitated discussion among groups of patients and
health professionals could reveal what their priorities were, and which
uncertainties about the effects of treatments mattered most to them and
should be addressed.

But do the voices of patients and the public really count in the conduct
of healthcare research? Happily, the formerly closed world of medicine is
opening its doors to admit fresh ideas and former ‘outsiders’. Paternalism is
steadily diminishing
. There is increasing support for involving patients as
partners in the research process. And there is accumulating evidence from
questionnaire surveys;116 systematic reviews of research reports;117 and reports of individual trials 118 that involvement of patients and the public
can contribute to improving tests of treatments. Patients have experience
that can enhance deliberations and provide insights. Their first-hand
knowledge can shed valuable light on the way in which people react to
illness and how this affects choice of treatments
.

Roles are evolving 119 to accommodate various ways of enabling
patients and the public to work with health professionals to improve the
testing of healthcare interventions. This is happening across the spectrum
of research activities:
- formulation of questions to be addressed
- design of projects, including selection of outcomes
- project management
- development of patient information leaflets
- analysis and interpretation of results, and
- dissemination and implementation of findings.

Today, there are moves away from clinicians imposing treatments and
research on patients
.

Patients regularly complain about lack of information. Although some
prefer not to have detailed information about their illness and treatment
options and to leave things entirely to their professional advisers, many are
keen to learn more. They want to know and understand how the results of
treatment tests have a bearing on them personally. For them, openness
and transparency are essential
. They need the assurance that everyone
knows what tests of treatments are going on; that results are published,
whether ‘positive’ or ‘negative’
; that systematic reviews of all the relevant
evidence are undertaken and kept up to date; and that adverse effects are
not covered up
. Clearly, patients need to be sure that researchers know
what has already been done before embarking on new research: as we
have outlined in earlier chapters, patients have suffered when researchers
have not bothered to find out what has gone before.

A medical researcher who underwent a few blood tests after an episode of
back pain was told he had some results that were not clear cut but were
somewhat worrying. Over the next few years the diagnosis of a type of blood
cancer (myeloma) became clear. He reflected on what had happened, including
the unavailability of evidence:

‘What are the lessons? Firstly, that as a patient I felt even more strongly about
what I’ve been fighting for throughout my career. Research results should be
easily accessible to people who need to make decisions about their own health.
The delay in the analysis of the four randomised controlled trials struck me as a
case in point. Why was I forced to make my decision knowing that information
was somewhere but not available
? Was the delay because the results were less
exciting than expected? Or because in the evolving field of myeloma research
there are now new exciting theories (or drugs) to look at? How far can we
tolerate the butterfly behaviour of researchers, moving on to the next flower well
before the previous one has been fully exploited
? Unfortunately this is possible in
a world where clinical research has become dominated by commercial interests.
When you are a patient you wonder how (we) researchers can keep forgetting
the principle that the priority should be collaboration for better hypotheses, not
competition
.’

People with HIV/AIDS in the USA in the late 1980s were another
exceptionally well-informed and well-organised group. They were politically geared to defend their interests against the establishment, paving the
way for patients to participate in the design of trials. This involvement
ultimately led to a choice of treatment options offered to patients in the
trials and flexible designs to encourage participation
.
 

Dolphin

Senior Member
Messages
17,567
HELPING TO INFLUENCE FUTURE RESEARCH
‘It is essential that cancer research takes into account the needs and interests
of the people it is trying to help. Cancer specialists are usually aware of the
gaps in their understanding of the diagnosis and treatment of cancer, but
patients, their families and friends may see other aspects of their care that
need further research
.’


These AIDS activists made trialists sit up: what some researchers had
viewed as havoc caused by organised patient groups was in fact a
legitimate challenge to the trialists’ interpretation of uncertainty. Until
then, the researchers’ approach had overlooked the patients’ preferred
outcomes
.

RESEARCHERS FOR PATIENT PARTICIPATION
‘We encourage patients and their organizations to participate in the planning of
clinical trials. Such participation is likely to ensure greater agreement with the
objectives and design of the trial and to make people with AIDS more aware of
the opportunities to enter trials.’
Byar DP, Schoenfeld DA, Green SB,et al. Design considerations for AIDS trials.
New England Journal of Medicine1990;323:1343-8.

[MS]
Worse still, because interferons had now become
the accepted standard treatment for the disease, independent studies of
alternative ways of helping patients had suddenly become prohibitively
expensive. Why? Because clinicians and patients claimed that it had
become unethical to compare any new treatment against placebos: new
treatments had to be compared with the costly interferons.

Nevertheless, in this book we illustrate how the existing ‘drivers’ for research – commercial and academic – have not done enough to identify patients’ priorities.

In particular, we should demand that all clinical trials are registered publicly at the outset, and that all the results are published in full on completion.

Agree to participate in a clinical trial only on condition (i) that the study protocol
has been registered publicly on www.controlled-trials.com; (ii) that the protocol
refers to the systematic reviews of existing evidence showing that the trial is
justified; and (iii) that you receive a written assurance that the full study results
will be published, and sent to all participants who indicate that they wish to
receive them
.

Challenge treatments offered to you or your family on the basis of beliefs and
dogmas, but unsubstantiated by reliable evidence
.
 

Sean

Senior Member
Messages
7,378
'I can live with uncertainty. I do not wish to live with dishonesty’.

Well said.

I can understand doctors feeling pressured to have a definite answer and effective treatment. A pressure, ironically, in large part due to the great success of modern science based medicine.

But the serious problem of not being sufficiently open about the limits to their knowledge, particularly in the clinic, still needs to be addressed.
 

Little Bluestem

All Good Things Must Come to an End
Messages
4,930
I’m afraid the practising physicians who I have come across do not have that scientific mindset.
That would be because practising physicians are generally not scientists. I think this confusion is the result of calling people who do not have a PhD 'doctor'.