• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

GETSET - Graded Exercise Therapy guided Self-hElp Treatment (GETSET)

joshualevy

Senior Member
Messages
158
Its interesting that this appears to have been allowed after the end of the open label trial. If an ethics committee has approved this we should be calling for their resignation.

First, you should read the study's abstract. The study was partly blinded, as described here:
Primary outcomes were rated by participants, who were necessarily unmasked to assignment; the statistician was masked to treatment arm assignment for the analysis.​
So the patients were not blinded, but the data analysis was. This is what would be expected for a trial testing a physical treatment that can't be blinded.

Second, you should read about what an ethics committee does. The purpose of an ethics committee is to protect the patients in the study. The committee's job is to make sure the study is safe to participate in. As long a filling out a extra questionnaire is safe for the study participants, they should approve it. They are not a "quality control" committee. That's what peer review is for.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
The study was partly blinded, as described here:
Primary outcomes were rated by participants, who were necessarily unmasked to assignment; the statistician was masked to treatment arm assignment for the analysis. So the patients were not blinded, but the data analysis was. This is what would be expected for a trial testing a physical treatment that can't be blinded.
As you say, it was open label. With post-hoc methodology including primary outcomes.

The purpose of an ethics committee is to protect the patients in the study. The committee's job is to make sure the study is safe to participate in. As long a filling out a extra questionnaire is safe for the study participants, they should approve it. They are not a "quality control" committee. That's what peer review is for.
I think user meant a steering committee. Peer review isn't a quality control process. It's an academic process.
 

user9876

Senior Member
Messages
4,556
First, you should read the study's abstract. The study was partly blinded, as described here:
Primary outcomes were rated by participants, who were necessarily unmasked to assignment; the statistician was masked to treatment arm assignment for the analysis.​
So the patients were not blinded, but the data analysis was. This is what would be expected for a trial testing a physical treatment that can't be blinded.

The point is that those carrying out the study are aware of what is going on - the people doing treatments know so unless there is a very strong Chinese wall between those carrying out therapies and those running the trial they will have a good idea about its success. Also statisticians will have an idea of the significance of differences between two groups as well as the effect sizes even if they do not know which group has changed (there are only two) so they will know if there is not a significant gap although could get a shock about the direction. Think through who who knows what and when.


Second, you should read about what an ethics committee does. The purpose of an ethics committee is to protect the patients in the study. The committee's job is to make sure the study is safe to participate in. As long a filling out a extra questionnaire is safe for the study participants, they should approve it. They are not a "quality control" committee. That's what peer review is for.

As I understand it ethics committees are responsible for the overall ethics. A long time ago I read through some of the EMA regulations that talked about the research ethics committees needing to review protocol changes. With PACE the MRC said they were happy with outcome switching because they were reviewed including by the MREC.

http://forums.phoenixrising.me/inde...e-from-the-mrc-to-hooper-pace-complaint.8867/
From Frances C Rawle PhD, Head of Corporate Governance and Policy, MRC

4. I acknowledge that there have been changes to the protocol for the PACE trial since it started. It is not uncommon for minor protocol modifications to be made while a trial is in progress; all such modifications must be approved by the Trial Steering Committee (TSC), the Data Monitoring and Ethics Committee, and the MREC that approved the original protocol, and they were in this case. In addition, the MRC Board was aware of the changes when it agreed to the extension to the original funding period for the trial, so clearly the Board did not consider the changes undermined the trial

The point is they should be checking the ethics of the whole trial which includes all the conduct of the trial and how it is reported. Now maybe they were presented with exceptional reasons for changes to be made to the protocols after the end of the trial. But the PACE group failed to give good reasons and in some cases reasons they gave were simply wrong. This is some of the same group. The role is not about quality control but about having strong governance in place and they seem to be failing in this by not asking questions. If we want to wipe out arbitary outcome switching then we need ethics committees that will enforce rules and basic ethics of good reporting. If they are fail they should go and be replaced by people who will.
 

user9876

Senior Member
Messages
4,556
As you say, it was open label. With post-hoc methodology including primary outcomes.


I think user meant a steering committee. Peer review isn't a quality control process. It's an academic process.
No I did mean the "London Bridge Research Ethics Committee" who approved the trial and should be approving or denying approval to major protocol changes. It is unethical to do outcome switching and it leads to misreporting of a trial. There can be technical reasons as to why it is necessary but they should be properly outlined. In an area where they have run many trials this should not be the case. As far as I can see the ethics committees are the basic governance structure for medical trials and yet they just seem to rubber stamp changes.
 

halcyon

Senior Member
Messages
2,482
I think we can expect a lot of bad publicity, wild claims from them as they release these papers. I suspect that the SMC will bring out the usual hate speech against patients with ME so that White and friends can avoid the difficult questions and gloss over the issues with PACE.
I'm sure it's just a total coincidence that they published this during ME awareness month while we have been getting increased positive press coverage.
 

Justin30

Senior Member
Messages
1,065
I'm sure it's just a total coincidence that they published this during ME awareness month while we have been getting increased positive press coverage.

Why do these people not want us to get better...like back as functional productive members of society?

It seems as though in some sick and twisted way they want us to stay sick and live within the torture that is ME.

Some days I am ashamed to be be part of the same human race as people like this....

If I could I would.......I would not choose this life nor would I choose it for anyone else....
 

joshualevy

Senior Member
Messages
158
It is unethical to do outcome switching and it leads to misreporting of a trial.

At the very worse they added a new primary outcome. "Outcome switching" would be either removing an old outcome or not reporting on it, and reporting on a new outcome in it's place. That would be the switch. And they clearly did not do that. The worst thing that can be said is that they added a new outcome and reported on it.

Most scientists would call what they did "reporting more data than they planned to report".

As far as I can see the ethics committees are the basic governance structure for medical trials and yet they just seem to rubber stamp changes.

It doesn't work that way in the US, and I doubt it does in Europe or the UK, either. Their are two limitations on changing a study: one is patient safety (as enforced by the ethics committee and the US FDA) and the other is fear that it will fail peer-review or that it will not be published. That's it. Once you have your data, how you analyse it is not a safety issue, so is covered by the "fear of peer review failure or fear of publication rejection". In the US, there is no pre-approval of any of that. Not by an ethics committee or any one else.
 

joshualevy

Senior Member
Messages
158
I think user meant a steering committee.

That would make even less sense, since most studies don't have steering committees at all. Usually there is just a PI (the principal or primary investigator) and he or she just makes decisions, often by consulting with the lead statistician or the other researchers involved, but not with a formal committee of any kind. It is only the largest, and most complex studies that have a formal steering committee. I've covered dozens of studies (not in ME/CFS), and only a handful had steering committees, and all of those that did had more than 1000 patients involved. PACE certainly had a steering committee, but that was unusual (especially for a 400+ person study). I supposed it's possible that it's a UK thing, but I've covered some trials in the UK, and they did not have steering committees, either.
 

A.B.

Senior Member
Messages
3,780
The study was partly blinded, as described here:
Primary outcomes were rated by participants, who were necessarily unmasked to assignment; the statistician was masked to treatment arm assignment for the analysis.​
So the patients were not blinded, but the data analysis was. This is what would be expected for a trial testing a physical treatment that can't be blinded.

Lack of blinding generates misleading results. The rest of medicine has long accepted this.

It would be possible to minimize this problem with a study design that doesn't rely entirely on subjectively rated health.

Cardiopulmonary exercise testing could be used as measure of the actual fitness capacity. With this test it's possible to tell how much effort a person is making.

Actigraphy would probably generate more reliable data than self rated health as well.

Of course we can already guess why White et. al. have abandoned objective measures in GETSET. Because they show no or marginal improvement, as seen in PACE or other studies.
 

user9876

Senior Member
Messages
4,556
At the very worse they added a new primary outcome. "Outcome switching" would be either removing an old outcome or not reporting on it, and reporting on a new outcome in it's place. That would be the switch. And they clearly did not do that. The worst thing that can be said is that they added a new outcome and reported on it.

Most scientists would call what they did "reporting more data than they planned to report".

I take your point that they have not removed the original primary outcome. But this is not about reporting more data than planned it is about how they describe the success of the trial. They choose to only use the SF36 scale as a primary outcome and then added an additional primary outcome and they will be talking about the overall success of the trial in terms of meeting this new set of primary outcomes. It changes the way the trial is reported and results are seen.

I've never liked the idea of pre-guessing the primary outcome because in more mature trials it allows those who pick the outcomes to use their experience to pick what the ones they consider will give the best results. They should give detailed reasons as to their choices but I've rarely seen this. I would prefer for a range of different end-points to be collected and for success to be declared as they are all changing or where their is good reason to understand why certain measures are not changing. But they preferred methodology for clinical trials is to design the protocol with primary end points at the start and use these to judge the trial. To then change the set of primary outcomes after the end of the trial just says 'dodgy' especially when it is an open label trial. I have not seen the full paper (I don't think it is published yet) so they may give some very good reasons for this but in the past they have not so I am judging them on past behaviour.

It doesn't work that way in the US, and I doubt it does in Europe or the UK, either. Their are two limitations on changing a study: one is patient safety (as enforced by the ethics committee and the US FDA) and the other is fear that it will fail peer-review or that it will not be published. That's it. Once you have your data, how you analyse it is not a safety issue, so is covered by the "fear of peer review failure or fear of publication rejection". In the US, there is no pre-approval of any of that. Not by an ethics committee or any one else.

So after doing a quick search for changing primary end points I found a European Medical Agency document that talks about getting approval for trials
http://ec.europa.eu/health/files/pharmacos/docs/doc2005/10_05/ca_14-2005_en.pdf

To quote a paragraph

.
4.2.3.2 Protocol
When the sponsor intends to make a substantial amendment to the protocol that would make a significant impact on the criteria in 4.2.3.1 he should notify the concerned CA and relevant ethics committee. For instance reducing the number of clinic visits might impact on the safety or physical or mental integrity of the subjects. Introducing a new monitoring procedure or a change in the principal investigator might significantly affect the conduct or management of the trial respectively. The use of a new measurement for the primary endpoint could alter the scientific value of the trial. Altering the procedure for reconstitution and administration of an IMP could affect the safe use of an IMP in the trial. These types of changes would be considered substantial amendments.

The point they make is that the protocols are there for multiple things including the scientific value as well as safety.

I'm somewhat alarmed that "fear of peer review failure or fear of publication rejection" could be used as a reason for changing primary end-points in a trial. This places a primary motivation of getting published over providing good information. Of course if there is a deviation from the predefined primary end points for a trial without being very clear and with very good reasons then peer reviewers should reject the paper. However, most will never read the protocol and most reviews don't spend that much time reviewing papers.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Edzard Ernst wrote a good piece on A vs A+B trials

No negatives please, we are alternative!
Since several years, researchers in this field have adopted a study-design which is virtually sure to generate nothing but positive results. It is being employed widely by enthusiasts of placebo-therapies, and it is easy to understand why: it allows them to conduct seemingly rigorous trials which can impress decision-makers and invariably suggests even the most useless treatment to work wonders.
That's a helpful blog...

This is the research paper associated with it, which could be a handy reference to keep a note of...

Ernst E, Lee MS. A trial design that generates only "positive" results. J Postgrad Med. 2008;54:214-6.
http://www.jpgmonline.com/text.asp?2008/54/3/214/41806
http://www.ncbi.nlm.nih.gov/pubmed/18626172

Ernst et al. said:
We conclude that the 'A + B versus B' design is prone to false positive results and discuss the design features that might prevent or exacerbate this problem.
Ernst et al. said:
In conclusion, our systematic review of RCTs of acupuncture for pain control with the 'A + B versus B' suggests that this trial methodology is likely to produce false positive results. Such studies may therefore not be adequate scientific tests of the effectiveness of therapeutic interventions.

It also mentions objective outcome measures...
Ernst et al. said:
Also, studies that employ objectively measurable endpoints might, in some cases, offer a protection against such false positive findings.

More discussion...
Ernst et al. said:
It would, of course, be an over-interpretation of our results to state categorically that RCTs with the 'A + B versus B' designs can, in principle, only generate positive results. We have shown this to be likely only for acupuncture as a treatment for pain. In other therapeutic areas, negative trials with that design may exist. Crucially, however, these RCTs do not fulfill the three additional criteria outlined in the introduction of this article. Interventions that are less prone to generating false positive findings when tested in a 'A + B versus B' study include those that not only do not improve but worsen the condition in question and treatments which do not generate sizable placebo-effects. Also, studies that employ objectively measurable endpoints might, in some cases, offer a protection against such false positive findings.
 
Last edited:

Bob

Senior Member
Messages
16,455
Location
England (south coast)
For anyone like me, who hadn't worked this out, it seems that the full GETSET paper hasn't been published yet, but just the abstract as part of a list of conference abstracts: The 4th annual scientific conference of the European Association for Psychosomatic Medicine (EAPM) in Luleå, Sweden, 16-18 June 2016. The conference is titled: "Transforming health through evidence and empathy." (If you believe it!) http://eapm2016.com/
 
Last edited:

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Not a therapy recommendation.

For anyone interested, the chronic fatigue Service at the NHS East London Foundation Trust (ELFT), based at St Leonard’s Hospital in Hackney, has a "GETSET" (i.e. self-guided GET) leaflet from Barts available for patients on its website:
https://www.elft.nhs.uk/uploads/files/1/Services/Chronic Fatique/GETSET Leaflet.pdf

It is to be found at the bottom of this webpage:
https://www.elft.nhs.uk/service/308/Chronic-Fatique-Service/

Maybe they're getting ready to roll out GETSET as a proven treatment strategy, but the leaflet acknowledges (page 2) that it's not yet a proven treatment.

Note that it's an NHS leaflet, and the ELFT is an NHS service.
 
Last edited:

Comet

I'm Not Imaginary
Messages
693
GETSET Leaflet:
Will GET
do me harm?
You may be worried that any
increase in exercise or physical
activity could make your condition
worse. Be reassured - research
has shown that a guided, gradual
exercise programme can help
people who suffer from CFS/ME
without causing ill effects.

However, a self-help guide
such as this has not been
officially tested so it is
important that you check
fi rst with your GP or hospital
specialist that a GET schedule
is suitable for you. You should
also continue to consult them
regularly while undertaking
your GET programme

Ummm...
 
Messages
13,774
Once you have built up the
amount of time you exercise to
30 minutes per day, five days
per week, you can start looking
at getting fitter by increasing
the intensity of your exercise.
For example, if your exercise
is walking you could slightly
increase the speed you walk,
introduce walking up a small hill
or carrying a load.

...

It will probably take you several
months to work your way slowly
through the last two steps of
a sensible GET programme:
increasing the duration and
intensity of your added exercise.

Probably? So, for the majority of patients? Not sure what evidence supports this.

It is often incorrectly assumed
that an increase in symptoms
equals harm. It doesn’t.

I'm not sure if it's up to them to decide this.