• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Clinical and cost-effectiveness of the Lightning Process for chronic fatigue syndrome

IreneF

Senior Member
Messages
1,552
Location
San Francisco
It looks to me like they might be minimising the number of differences with CBT, maybe to try to say that CBT might work similarly well with adjustments.

Given that LP involves hypnosis/similar, it doesn't seem that similar to CBT.
CBT and LP both have their origins in a theory called "general semantics" developed by Alfred Korbyzski in the 1920s.
https://en.m.wikipedia.org/wiki/General_semantics
 
Messages
13,774
How seperate were the pilot SMILE trial and the non-pilot SMILE trial? How seperate are they meant to be?

Sorry if I'm asking stupid questions, but I don't know what's normal with this sort of thing.

Was there seperate ethics approval for i) the pilot/feasability study ii) the non-pilot trial?

Was funding gained for the pilot, and then new funding for the main trial?

I hope @Dx Revision Watch will be able to fill me in on this stuff.

My memory was that there was only really announcements on funding and ethics approval for the pilot. Was I just distracted by PACE after that? eg: I remember this getting attention, but I don't remember new of further funding after that: http://www.meassociation.org.uk/201...o-the-lightning-process-and-children-with-me/

The non-pilot SMILE paper says:

This trial was funded by the Linbury Trust (grant number LIN2038) and the Ashden Trust (grant numbers ASH1062, 1063, 1064). EMC was funded by an NIHR Clinician Scientist fellowship followed by an NIHR Senior Research Fellowship (SRF-2013-06-013) during the trial.

The only mention I could see of an approved grant from the Linbury trust website was in 2010 - presumably for the pilot:

University of Bristol - £44,000 (Over three years) Towards the scientific study of the "Lightening Process" at Bristol University.

http://www.ashdentrust.org.uk/files/Ashden Trust Annual report - 5 April 2010.pdf

This letter dated Nov 2009 does seem to fit with this being for a feasability study: https://meagenda.files.wordpress.com/2010/07/funding-linbury-trust-04-11-09.pdf

It looks like this was then paid out over a number of years.

In the non-pilot paper they say:

Between September 2010 and April 2013, we recruited participants after clinical assessment by the Bath/Bristol paediatric CFS/ME service, a large regional and national NHS specialist service.

In the feasability/pilot study paper they say:

A total of 56 children were recruited from 156 eligible children (1 October 2010 to 16 June 2012).

And then:

Full trial first randomization: 19 September 2012.

In the non-pilot paper they say:

Sensitivity analyses of the primary outcome adjusted for variables for which there was baseline imbalance; excluded those recruited up to 31 January 2011 preceding the protocol amendment; and used multiple imputation of missing data (see online supplementary appendix 1 for details).

What's going on? This seems rediculous to me, but I have no idea how this sort of thing is meant to work. It looks like this supposedly non-pilot trial includes data from loads of participants who were part of the pilot study that was used to justify changing the primary outcome away from school attendance at six-months (this outcome showing no signifcant difference between groups in the non-pilot paper, and not being reported in the paper on the pilot study) to more subjective self-report outcomes. That can't be okay can it? It looks a bit like, instead of trying to get funding for a proper follow-up to the pilot, they just botched it and chucked a few extra participants in at the end. Surely they wouldn't try to get away with something else while they're already getting so much attention over PACE?

The only other mention of a protocol change in the non-pilot paper was this:

We used multiple imputation to correct for potential bias due to missing data and conducted sensitivity analyses restricted to participants recruited after the protocol changed to collect primary outcome data by telephone, which improved follow-up rates suggesting that results were robust.

Does anyone with some knowledge of how pilot studies are meant to work want to comment on this? @Jonathan Edwards ?
 

Jonathan Edwards

"Gibberish"
Messages
5,256
Does anyone with some knowledge of how pilot studies are meant to work want to comment on this? @Jonathan Edwards ?

It does look very fishy, at least as you have presented it. And in this business when things start to look fishy they usually seem to end up looking decidedly cod-like.

A pilot study is a separate study that would require a separate ethical approval and a separate protocol. The same funding can be used but funding applied for for a pilot study is unlikely to be adequate to cover a formal non-pilot study. So the question arises as to whether funding was not granted for a formal study but it went ahead all the same, maybe using some of the pilot material. If so then it cannot be treated as a formal trial and certainly not a controlled trial.

I have myself been in a slightly similar situation. I did a pilot study and published it. I then did further pilot work, with a view to a formal study. I published the further pilot work, including a re-publication of the first pilot study as part of the total. However, I published this as a retrospective review of clinical experience and made no statistical analysis because I knew that no valid statistical analysis could be made. It was merely an exercise in reporting observations made for the record. I then undertook the formal study with entirely new ethical approval and new funding.

If this study really is a botched extension of a pilot because no further money was available it raises the possibility of very bad research practice indeed. At least the PACE trial went through the motions of getting approval for changes within a pre-defined structure. Looks like a job for Morse again.*

(*Morse was a British detective series.)
 

user9876

Senior Member
Messages
4,556
If this study really is a botched extension of a pilot because no further money was available it raises the possibility of very bad research practice indeed. At least the PACE trial went through the motions of getting approval for changes within a pre-defined structure. Looks like a job for Morse again.*

The pilot study didn't report results I think it just included interviews with participants around 'acceptability'. When it was published it did read like a list of reasons to update the protocol - I remember it sounding more like an 'official' version of outcome switching.

@Esther12 I think patients from the pilot were included in the results as they talk about getting outcome results over the phone from Feb 2011. From the dates you give it looks like this was when they changed the protocol, finished the pilot and then started again sometime later for the full trial randomization.
 

Londinium

Senior Member
Messages
178
On the question of use of school attendance as a guide to improvement, I realise that aspects of the problem have already been discussed but I think a name for the problem has not yet been mentioned.

Is not the issue under discussion "presenteeism", which will be familiar to those who have suffered from mild ME and been hanging on to their jobs by their fingertips? The fact that someone is sitting behind their desk is no indicator of their state of health.

Presenteeism aside, I think there's just a fundamental flaw with using school attendance as a supposed objective outcome that is worse than using employment outcomes for adults. After all, had the FINE long-term follow up found significantly better employment outcomes then I think it would have provided a decent argument for at least exploring CBT/GET further (patient selection issues aside). The difference compared to SMILE is that, in PACE/FINE, it was the patient who decides whether they are well enough to go to work. But for SMILE, we are talking about children and therefore the decision as to whether to go to school on a given day is, at best, a joint decision. School attendance cannot be considered a truly objective outcome because it is possible that a family that has received a 'treatment' materials that consists of telling them to ignore symptoms and avoid 'negative thinking' might well dispatch their child to school when a family in the control group wouldn't have.

School attendance ceases to be an objective outcome if the parents are aware of which treatment their child receives and therefore could be influenced as to whether they think their child is cured. It's as if the PACE trial had also sent information to the bosses of patients in the CBT/GET arms telling them that their employee was now cured and should get their arse back into work.
 
Messages
13,774
School attendance, combined with attainment, could be better? But there are a lot of potential problems there too.

Nothing is perfect, but I still think that school records on attendance would be a better outcome measure than the SF36-PF or Chalder Fatigue Scale.

All of these things would be less useful if the intervention itself was targeted at improving the particular outcome, rather than the patient's health more broadly - eg it's easy to imagine how a programme designed to improve school attendance could do so, even if this did nothing the improve people's health, and actually served to make their lives worse.
 
Messages
13,774
A pilot study is a separate study that would require a separate ethical approval and a separate protocol.
...

If this study really is a botched extension of a pilot because no further money was available it raises the possibility of very bad research practice indeed. At least the PACE trial went through the motions of getting approval for changes within a pre-defined structure. Looks like a job for Morse again.*

Thanks.

[IGNORE - this is about MAGENTA]: The letter about ethics approval from November 2016 talks only about a feasability study:

http://www.bristol.ac.uk/media-library/sites/ccah/cfsme/study-docs/15 SW 0124 Confirmation of favourable opinion 24.11.16 (1).pdf


I think that there are already a couple of detectives on this: @Dx Revision Watch @JohntheJack - do either of you have any idea about how the feasability study and the full study relate to one another, and what approval was granted when for what?

In the trial protocol (for the full trial, not the feasability study) they say:

Ethical review
A favourable ethical opinion was given on 8 September
2010 (reference 10/H0206/32) by South West 2 Local Re-
search Ethics Committee. Two favourable opinions have
been provided on 31 May 2011 and 6 September 2012 for
amendments to study documents and protocol.

http://www.bristol.ac.uk/media-library/sites/ccah/migrated/documents/protocol1.pdf

But surely that relates to the feasability study? Could the 'amendment' of 6th September 2012 allow them to move on to a full trial? That's just before they said they were due to start randomization for the full trial.

To me, using a feasability study to change outcomes from one with a null result to one with a positive result seems a bit dodgy already (although I can see how justifications could be made for it if it was done very openly), but if the full study included data that was from the feasability study used to justify switching outcomes that's getting pretty odd.

The pilot study didn't report results I think it just included interviews with participants around 'acceptability'. When it was published it did read like a list of reasons to update the protocol - I remember it sounding more like an 'official' version of outcome switching.

@Esther12 I think patients from the pilot were included in the results as they talk about getting outcome results over the phone from Feb 2011. From the dates you give it looks like this was when they changed the protocol, finished the pilot and then started again sometime later for the full trial randomization.

Yes - that's what it read like to me to. As if being so in your face about it made it less of a problem.

Surely it's dodgy that they say in their feasability study: "Full trial first randomization: 19 September 2012"

...then in their full trial paper: "Between September 2010 and April 2013, we recruited participants after clinical assessment by the Bath/Bristol paediatric CFS/ME service, a large regional and national NHS specialist service."

Surely the fact that the Lightning Process itself is so transparantly quacky means that they'd try to do everything else as by the book and above board as possible? Why would they take risks with something so likely to blow up in their face.
 
Last edited:

TiredSam

The wise nematode hibernates
Messages
2,677
Location
Germany
School attendance ceases to be an objective outcome if the parents are aware of which treatment their child receives and therefore could be influenced as to whether they think their child is cured.
If a parent asks a child "How are you feeling today, do you think you can manage school?" And the child has recently been brainwashed into ignoring or not admitting their symptoms to themselves or anyone else under threat of being labelled at fault for not trying hard enough, what's the child more likely to reply? Not wanting to let their parents, therapist or self down, they may well say "yep, just let me step out of this circle" and try to do school instead of doing M.E, as they've been instructed by all around them. It's either that or disappoint everyone by not trying hard enough.

I really don't think it would be going too far for Esther Crawley to be arrested and charged with child abuse and to be put on some kind of registry. Her manipulation of sick children and their parents to further her own career is sickening.
 

Wonko

Senior Member
Messages
1,467
Location
The other side.
I'd say it depends on the aims of the study/protocol/etc.

If the aim is simply to increase school attendance in children then increased attendance is a valid, and objective, outcome measure.

If the aim is to achieve something meaningful, such as increase grades, or, heaven forbid, actually cure people, then it's not.

If you take the view/accept that the people commissioning, performing, and reporting on this study are number driven morons with no understanding of the actual problem(s), then it all makes sense.

It's still utter tosh, it's still mumbo jumbo ritualised brainwashing that should result in criminal charges, but you can sort of see.........
 

user9876

Senior Member
Messages
4,556
Yes - that's what it read like to me to. As if being so in your face about it made it less of a problem.

Surely it's dodgy that they say in their feasability study: "Full trial first randomization: 19 September 2012"

...then in their full trial paper: "Between September 2010 and April 2013, we recruited participants after clinical assessment by the Bath/Bristol paediatric CFS/ME service, a large regional and national NHS specialist service."

Surely the fact that the Lightning Process itself is so transparantly quacky means that they'd try to do everything else as by the book and above board as possible? Why would they take risks with something so likely to blow up in their face.

Something seems strange. The Feasibility study paper
https://trialsjournal.biomedcentral.com/articles/10.1186/1745-6215-14-415
Trial registration
Feasibility study first randomization: 29 September 2010.

Trial registration: Current Controlled Trials ISRCTN81456207 (31 July 2012).

Full trial first randomization: 19 September 2012.

I take that as saying the feasibility study was not registered since it started in 2010 and the registration is in 2012.

The trial registraion
http://www.isrctn.com/ISRCTN81456207

Has
Primary outcome measures
1. Chalder Fatigue Scale at 6 months
2. SF 36 physical function short form at 6 months
....

Overall trial start date
01/08/2012

Overall trial end date
31/03/2013

Reason abandoned


Eligibility
Participant inclusion criteria
Children will be included if they have CFS/ME and are between 12 and 18 years old

Participant type
Patient

Age group
Adult

And also
Ethics approval
South West 2 Local Research Ethics Committee, 8th September 2010, ref: 10/H0206/32. Amendement approved on 31/05/2011

So it looks very unclear. The trial registration is badly filled in (including saying its adults in one place). Its also confusing but it looks like
1) Ethics committee approved something sept 2010
2) Randomisation for feasibility study trial started sept 2010
3) Ethic committee approved amended protocol May 2011
4) Trial was registered July 2012 (but using original ethics approval)
5) Trial recruitment started Sept 2012
6) Trial should have finished March 2013 - but given the 1 year monitoring this is impossible so I assume that is end of randomisation. The paper says April 2013 was the last randomization.

I've not looked at the ethics approval yet
 

user9876

Senior Member
Messages
4,556
The ethical approval form
http://www.bristol.ac.uk/media-library/sites/ccah/migrated/documents/recfrmrfs.pdf

has it down as
Study administering questionnaires/interviews for quantitative analysis, or using mixed quantitative/qualitative
methodology

Title said:
Assessing the feasibility and acceptability of comparing the Lightning Process with specialist medical care for
Chronic Fatigue Syndrome or Myalgic Encephalopathy (CFS/ME) pilot Randomized Controlled Trial.
At this point in 2010 it wasn't a registered trial
International Standard Randomised Controlled Trial Number (ISRCTN): n/a
ClinicalTrials.gov Identifier (NCT number): n/a
European Clinical Trials Database (EudraCT) number: n/a
Project website: n/a

The aim here in the application form is clearly to test whether a trial is feasible

A10.
What is the principal research question/objective?
To assess the feasibility and acceptability of conducting a randomised controlled trial to investigate the effectiveness and cost effectiveness of specialist medical care compared with specialist medical care plus the Lightning Process.
 
Messages
13,774
Just pulling out the bits from the protocol that refer to the feasibility study:

This trial continues from the SMILE feasibility study
which showed that recruitment, randomisation and data
collection on health resource use was feasible and accept-
able [13].

In-depth interviews

Interviews with parents and children/young people were
conducted at three time points for the feasibility study:
after initial assessment but prior to randomisation, after
randomisation but prior to the intervention and after
the intervention. Analyses of data will continue from the
feasibility study, supplemented with further in-depth in-
terviews undertaken with some parents and children after
the intervention to further assess their experiences of the
trial and intervention.

Qualitative data analysis
Analysis will be an ongoing and iterative process com-
mencing soon after data collection. It will build on data
analysed from the feasibility study and will inform fur-
ther sampling and data collection.

My impression is that the feasibility study and the full study ended up being the same thing.

Yeah - or at least, they just added some extra participants. And surely that makes the way they changed their primary outcome extra dodgy?
 

user9876

Senior Member
Messages
4,556
I note that in the latest paper they don't reference an ethics committee approval number just
  • Ethics approval South West 2 Local Research Ethics Committee.

But the trial registration references the ethics approval for the feasibility study with the application mentioned here
http://forums.phoenixrising.me/inde...nic-fatigue-syndrome.54507/page-6#post-911369
seeming clear that this is a feasibility study not a full trial.

There appear to have been two amendments
A favourable ethical opinion was given on 8 September 2010 (reference 10/H0206/32) by South West 2 Local Research Ethics Committee. Two favourable opinions have been provided on 31 May 2011 and 6 September 2012 for amendments to study documents and protocol.

But I can't find information on these. It would seem to me unreasonable to turn an ethics approval for a feasibility into a full trial and if the ethics committee did that it brings into question whether a full trial was given full consideration and for the full committee.

I'm starting to think that Bristol University may not have had ethics permission for the trial.
 
Messages
13,774
But I can't find information on these. It would seem to me unreasonable to turn an ethics approval for a feasibility into a full trial and if the ethics committee did that it brings into question whether a full trial was given full consideration and for the full committee.

I'm starting to think that Bristol University may not have had ethics permission for the trial.

It will be interesting to see what those two ammendments were for, and what exactly they say.
 

JohntheJack

Senior Member
Messages
198
Location
Swansea, UK
Thanks.

The letter about ethics approval from November 2016 talks only about a feasability study:

http://www.bristol.ac.uk/media-library/sites/ccah/cfsme/study-docs/15 SW 0124 Confirmation of favourable opinion 24.11.16 (1).pdf

I think that there are already a couple of detectives on this: @Dx Revision Watch @JohntheJack - do either of you have any idea about how the feasability study and the full study relate to one another, and what approval was granted when for what?

...

I wrestled with this question myself and then like you decided that the feasibility study became the first part of the full study, ie the participants and results of the feasibility study were added to the new participants to make the complete trial.

I noticed the acceptance rate is about the same (30%). I wonder whether that may be why they used the feasibility study in that way: despite doing the trial 'because of demand from patients and parents', they were just not getting enough to even claim significance. There may have been time and financial constraints, so easier for them just to fold the feasibility study into the full study.
 

Jonathan Edwards

"Gibberish"
Messages
5,256
I am amazed. We have Morse, Poirot, Holmes and Marple on the case and they are still 'wrestling with the question' and it's my bedtime! Ah well, maybe it takes little longer. Maybe we need one of those FOI jobs.

Only teasing. It does look seriously odd to me but I don't want to jump to conclusions and this sort of ferreting around dates is not something I am very good at personally.

It might be possible to make enquiries to the editor of Archives of Diseases of Childhood if we are clear what the questions should be.