• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Trial By Error: No Ethical Review of Crawley School Absence Study

lilpink

Senior Member
Messages
988
Location
UK

Cheshire

Senior Member
Messages
1,129
Follow up:

Trial By Error: Seeking More Details on Crawley School Absence Study

This morning I sent the following freedom of information request to Bristol University. My friend and colleague Steven Lubet, a professor at Northwestern Pritzker School of Law, joined me in making this request. Professor Lubet is an expert on legal ethics, among many other fields, and in July he guest-blogged here about the purported “ethics” lecture given at Oxford by Professor Michael Sharpe.

http://www.virology.ws/2017/08/30/10013/
 
Messages
2,125
Does someone remember what actually happened with the SMILE trial (Lightning Process) ethical review?
I remember that there was a problem because this "treatment" was tested on children while it had never been tested on adults, which is normaly impossible. But I don't recall on what ethical grounds it was finaly accepted.

just about everything about SMILE can be found here:
https://meagenda.wordpress.com/category/dr-esther-crawley/

It makes for some interesting reading, particularly her correspondence with the REC committee.........you can almost hear her arrogant exasperation when they point out faults in the various editions of the protocol, and yes she did say that CFS/ME in children is different and that's why it didn't need to be on adults first...........

"Our protocol makes it clear that CFS/ME in children is not sufficiently close to CFS/ME in adults to mean that research in adults could be extrapolated to children."

and yet:
"
Some of the activities do not relate to young people (especially for the younger participants) e.g. Q4 pushing a vacuum, bowling, playing golf, lifting or Q5 carrying groceries, Q13 time spent on work, Q22 normal work. This needs to make items more young person focused or add a column to say “never done this”.

Thank you for your thoughts on this. We initially tried to use the CHQ on children
(developed for children but not tested head to head with the SF36) with CFS/ME but
they told us the activities were not relevant and it did not adequately reflect their
disability. We therefore changed to the SF36 on the request of the children in our service
and have now used this for 6 years in over 1000 children with CFS/ME. We rarely have missing data on this reflected in our publications.
We understand that changing the SF36 has in the past provoked the threat of legal action and we
remain concerned about changing a validated inventory that we have already used extensively on this cohort both clinically and for our previous research.
There are risks to interpreting findings and comparing with other published work if we change the inventory.

During this study we will examine young people’s views and understanding on completing the SF36 (and other inventories) and will be able to find out if there are issues over the interpretation of the questions.
If there are, we intend to use this to apply for further funding to develop a CFS/ME measure of severity.
We would respectfully request that we are allowed to continue to use this well
recognised and validated inventory in this group of patients."
 
Last edited:

Cheshire

Senior Member
Messages
1,129
To check if the "protocol makes it clear that CFS/ME in children is not sufficiently close to CFS/ME in adults to mean that research in adults could be extrapolated to children.", I had a look at the the SMILE protocol:

CFS/ME is different in children and adults with different risk factors [33-35], course and outcome [36]. It is therefore not possible to complete a study in adults and extrapolate the results to children.
http://www.bristol.ac.uk/media-library/sites/ccah/migrated/documents/protocol1.pdf

Not very explicit and detailed, so I searched the studies cited to sustain that claim:

33. Hempel S, Chambers D, Bagnall AM, Forbes C: Risk factors for chronic fatigue syndrome/myalgic encephalomyelitis: a systematic scoping review of multiple predictor studies.
Psychol Med 2008, 38(7):915–926.
http://www.ncbi.nlm.nih.gov.ololo.sci-hub.cc/pubmed/17892624


34. Harvey SB, Wadsworth M, Wessely S, Hotopf M: Etiology of chronic fatigue syndrome: testing popular hypotheses using a national birth cohort study.
Psychosom Med 2008, 70(4):488 –495.
http://www.ncbi.nlm.nih.gov.ololo.sci-hub.cc/pubmed/18378866


35. Viner R, Hotopf M: Childhood predictors of self reported chronic fatigue syndrome/myalgic encephalomyelitis in adults: national birth cohort study.
BMJ 2004, 329(7472):941.
http://sci-hub.cc/10.1136/bmj.38258.507928.55


36. Joyce J, Hotopf M, Wessely S: The prognosis of chronic fatigue and chronic fatigue syndrome: a systematic review.
QJM 1997, 90 (3):223 –23



  • The first study explored different cohorts of children and adults but never talks about differences between the two groups.

  • The second and third ones are using a national birth cohort to explore childhood factors that could be risk factors for developing CFS. They never compare children and adult cohorts to see if there are differences in their illness.

And BTW they reach totally opposite conclusions, a real caricature of what CFS research field is:​

Individuals who exercise frequently are more likely to report a diagnosis of CFS in later life. This may be due to the direct effects of this behavior or associated personality factors. Continuing to be active despite increasing fatigue may be a crucial step in the development of CFS”​

“We identified no association between maternal or child psychological distress, academic ability, parental illness, atopy, or birth order and increasing risk of lifetime CFS/ME. Sedentary behaviour increased the risk”​


  • Only in the last study about prognosis could I find a difference into the two groups (I couldn’t access the paper, only the abstract):

Of 26 studies identified, four studied fatigue in children, and found that 54-94% of children recovered over the periods of follow-up. Another five studies operationally defined chronic fatigue syndrome in adults and found that < 10% of subjects return to pre-morbid levels of functioning, and the majority remain significantly impaired.​


I must say that I'm puzzled. Did I miss something in the studies, did I mixed up the references?
If I didn't make a mistake, this is no at all convincing...

I'm too drained to check another time.
 

lilpink

Senior Member
Messages
988
Location
UK
To get back to the main thrust of this post - Crawley and the apparent misuse the REC number - I have a bit more from my 'contact' to add to this story:


One of the published studies that used the REC reference number was by Esther Crawley and Peter White - https://academic.oup.com/qjmed/article-lookup/doi/10.1093/qjmed/hct061 - adults this time, not children. So how can that REC number possibly fit? And yes, a complaint was made to the journal – QJM- about this paper too….and the journal’s response (after 3+ months to investigate) - “no corrections to this article are required at this time”. No more, just that. They gave no explanations regarding this REC number and didn’t address the concerns raised in the complaint about the apparent lack of informed consent of at least some patients who would have contributed to the National Outcomes Database. Badly done, I’d say.


This is an extract of the email sent in reply to the Editor of QJM:

“I am both surprised and disappointed that, following your long and 'substantive' investigation, you have not provided any explanations or evidence to me to allay my concerns over the matters that I raised about the above paper.

It is astonishing that you have failed to offer me any explanation regarding the apparent misuse of the REC number.

To use common parlance - as it stands, this stinks.”


NB We are being offered this information on a plate by someone (who doesn’t have ME and who doesn’t want the limelight) who has gone to great lengths to help our cause for no personal gain. I think it’s a pity that PR members don't seem to be taking more of an interest in this. I think we have a lot to lose by ignoring contributions from less prominent sources.

Thought it might be worth re-posting this from earlier in the thread. It might resonate more now that people have had a chance to read Tuller's blog.
Edit: ignore the last paragraph as it's no longer relevant .
 
Messages
13,774
Thought it might be worth re-posting this from earlier in the thread. It might resonate more now that people have had a chance to read Tuller's blog.
Edit: ignore the last paragraph as it's no longer relevant .

Thanks. I think that I did miss this post.

NB We are being offered this information on a plate by someone (who doesn’t have ME and who doesn’t want the limelight) who has gone to great lengths to help our cause for no personal gain. I think it’s a pity that PR members don't seem to be taking more of an interest in this. I think we have a lot to lose by ignoring contributions from less prominent sources.

There is a problem that so much info is being constantly posted, that it can take a while to notice and absorb it. Thanksfully, having lots of people looking at different things does allow stuff to be picked up repeatedly until it gains prominence.
 
Messages
2,125
Without going into detail I've emailed the HRA with a general enquiry giving them the REC number and also listing the papers as listed by @lilpink. I also cc d the Bristol cable.
I got replies from the Bristol Cable; unfortunately a lot of people are on hols at the moment but they definitely seemed interested.
Today I got a reply from the media department at HRA (because I cc'd BC they treated it as a media enquiry);
pretty much as expected:
"
I’m afraid I’m not able to comment on the specific REC decision. The REC provides independent advice, with a focus on protecting the rights, dignity and well-being of those taking part in health research and to facilitate high quality research.
I am able to point you in the direction of some of the guidance the HRA has produced, which might provide helpful context. The HRA publishes detailed guidance on when a study is considered research, details of which can be found here:"

followed by the relevant links on their website.

I tried:(
 
Last edited:

Jo Best

Senior Member
Messages
1,032
I must say that I'm puzzled. Did I miss something in the studies, did I mixed up the references?
If I didn't make a mistake, this is no at all convincing...

I'm too drained to check another time.
I don't think you made a mistake.

A letter of complaint about this to the GMC in 2011 concluded:

1.5 None of the research referenced in the SMILE Protocol, provides evidence for Dr. Crawley's claim that CFS/ME is different in adults and children.

2. The World Health Organisation does not differentiate between paediatric/adult ME/CFS. ME is listed by WHO at ICD-10 G.93.3 under Diseases of the Nervous System – Post-Viral Fatigue Syndrome: Benign Myalgic Encephalomyelitis. CFS is listed in the alphabetical index with a reference back to G.93.3.

3. Neither the Chief Medical Officer Report (Department of Health 2002) nor the NICE Guidelines (2007) for CFS/ME make a distinction between CFS/ME in adults and children except that children may be diagnosed earlier (at 3 months instead of 6).

The GMC replied:

The Case Examiners have reviewed all the documents provided by the complainants and Dr Crawley and cannot find any evidence from peer-reviewed randomised controlled trails that demonstrates similarities/differences of CFS/ME in adults and children or that supports the efficacy or otherwise of LP. They note that SWREC considered Dr Crawley’s rationale to be legitimate.

So the GMC agreed that no evidence could be found of differences or similarities and passed the buck to the SWREC for considering Esther Crawley's rationale to be legitimate, in spite of the fact that the thrust of the complaint to the GMC was that Esther Crawley had appeared to have misled the ethics committee, as ethics committees are entitled to trust the information provided to them by the research applicant.
 

Large Donner

Senior Member
Messages
866
"Our protocol makes it clear that CFS/ME in children is not sufficiently close to CFS/ME in adults to mean that research in adults could be extrapolated to children."

Why are you calling it the same thing then. I guess if you choose any criteria which suits you at any given time then you can never match two groups together if you dont want to.

Maybe adults dont get childhood "CFS" because adults dont miss 3 days of high school each term.
 

lilpink

Senior Member
Messages
988
Location
UK
The following observations have been sent to me by the individual who has been investigating these areas of concern.

*****

The following is a summary of the points that come out of this case and other cases that use the same REC reference number. (Some new material here).


· Crawley AND HER COLLEAGUES (who may too be culpable for any proven ethical misconduct) used the same REC number – 07/Q2006/48 - for many published studies (list provided in thread). This REC number appears to be entirely unmatched/inappropriate for some of these studies.


· The most notable of these was the school absence study http://bmjopen.bmj.com/content/1/2/e000252 where it was used to negate the need for ethics approval/review for a study which many would consider to be definitely ‘research’ rather than ‘service evaluation’. It is great that David Tuller has taken this up http://www.virology.ws/2017/08/28/trial-by-error-no-ethical-review-of-crawley-school-absence-study/ and is investigating further along with Steven Lubet - http://www.virology.ws/2017/08/30/10013/


· BMJ Open, published this study despite being warned about ethical concerns by the peer reviewer Matthew Hotopf. BMJ Open has been a leading light in promoting transparency and openness in the publication of medical research yet in this case appears to have failed to properly scrutinize ethical approval statements and to have failed to take heed of its own peer reviewers. When concerns were raised with BMJ Open about this paper they were not satisfactorily addressed.


BMJ Open is part of the BMJ group of journals. BMJ journals are regarded as high-quality around the world. As part of an organisation (BMJ.com) that seems to set much store by questioning answers and answering questions - https://twitter.com/bmj_latest?ref_src=twsrc^google|twcamp^serp|twgr^author - BMJ Open have, in my opinion, failed miserably in answering the important questions raised about this paper, and appear not to have questioned or scrutinized the ‘answers’ provided by Bristol University in the course of the ‘investigation’ into this case.


· BMJ Open is a member of COPE – Committee on Publication Ethics. COPE advises journals and journal editors on ethical matters and on how to deal with cases of misconduct. Member journals/publications should abide by COPE’s retraction guidelines - PDF]Retraction Guidelines - Committee on Publication Ethics – and consider issuing an ‘Expression of Concern’ when investigations will take some time or when there is evidence that suggests (rather than proving) misconduct. It took BMJ Open more than 4 months to reply to this complaint. In that time, they did not issue an ‘Expression of concern’ about this paper, even though the person who submitted the complaint repeatedly reminded them of these COPE guidelines.


· Another complaint was submitted to QJM over this White and Crawley paper - https://academic.oup.com/qjmed/article-lookup/doi/10.1093/qjmed/hct061 - that uses the same REC number. The REC number does not seem to match in this case because this study is with adults only - the REC number given refers to a longitudinal paediatric study.


Also raised in this complaint was the issue of whether patients attending ME/CFS clinics across the UK were misinformed when asked to complete questionnaires for the National Outcomes Database and whether they completed the questionnaires without informed consent regarding what they were for or how they would be used in the future. There is some patient documentary and circumstantial evidence to support this hypothesis. Also, Action for ME accounts seem to show that from as early as 2008/2009 there was the intention to use the NOD for future research purposes to support ‘large scale biomedical research such as Genome Wide Association studies’ and the NOD project was one of 2 projects reviewed by Professor Holgate of the MRC ‘to assure the scientific quality of the proposals’. It was considered that the NOD had ‘considerable potential for acting as a catalyst for much larger research programmes underwritten by the major funding bodies’. [Last 3 quotes from Action for M.E. Report and Accounts for the year ended 31st March 2009 – obtained via FOI request to the Charity Commission.]


(The NOD was at least part-funded by the ‘Cathery Fund’, a fund set up from money raised by a celebrity-style fund-raiser to support biomedical research into ME/CFS. This fund is recorded as an ‘Unrestricted Fund’ in Action for ME Accounts.)


However, it would appear that some patients, at least, were not informed of this intended use of the NOD at the time they were asked to complete the NOD questionnaires, i.e. after this intention was documented by the Action for ME charity. Crawley, White et al claimed that this study was a service evaluation and as such did not require REC review/approval.


But, according to the HRA - http://www.hra.nhs.uk/resources/res...-arrangements-for-research-ethics-committees/ :


“Subject to any overriding legal requirements, REC review is not required for the following types of research:

Research limited to secondary use of information previously collected in the course of normal care (without an intention to use it for research at the time of collection), provided that the patients or service users are not identifiable to the research team in carrying out the research”


(IF this transcription of Dr Crawley’s talk on the Future of ME/CFS research- https://meagenda.wordpress.com/2010...presentation-the-future-of-research-in-cfsme/ [go to slide 16 ] is accurate, then it would also lend support to the idea that Crawley and colleagues intended to use the NOD for future research purposes.)


QJM dismissed the issues raised in this complaint following a ‘substantive investigation of the issues’ raised, finding that ‘no corrections to this article are required at this time’. QJM is also a COPE member; it took them more than 3 months to reply with the outcome of their investigation and no ‘Expression of Concern’ was applied to the paper during that time.


· A third, (and broadly similar complaint to the last one), was made to the Biomed Central Health Service Research journal about this Crawley and Collin (et al) paper - https://bmchealthservres.biomedcentral.com/articles/10.1186/1472-6963-11-217 . This is clearly an adult study so the quoted REC number that refers to the paediatric study doesn’t appear to match here either. This study also utilises the NOD which may well have been collected without the informed consent of the patients that took part.


To date this journal has not replied with the outcome of the investigation it has said it is conducting; more than 6 months have now elapsed. BMC Health Services Research is a member of COPE, so COPE’s Retraction Guidelines should apply here also, but no ‘Expression of Concern’ has so far been raised.

______________________



It seems that many people and organizations may now have important questions to answer about all this – Professor Crawley and her co-authors, the Royal National Hospital for Rheumatic Diseases (Bath)’s R&D manager/department, the University of Bristol, 3 health journals (BMJ Open, QJM and Biomed Central Health Services Research), COPE (who were contacted via a general enquiry and were asked to remind the member journals of their obligations), and the HRA and South West – Central Bristol REC (who were both informed of the multiple use of this REC number but apparently have not taken the matter further).


______________________________
 

lilpink

Senior Member
Messages
988
Location
UK

So the HRA isn’t responsible for the RECs?

The person who made the complaint to the Bristol REC about the multiple use of the REC number by Crawley et al was told to communicate instead with the Corporate Secretary of the HRA, so it would seem very strange for the HRA to now tell slysaint that they are “not able to comment on the specific REC decision”, if that’s what they wrote. What is the function of the HRA if not to oversee the work of the RECs?

From: http://www.hra.nhs.uk/about-the-hra/

“We make sure that health and social care research involving them [members of the public*] is ethically reviewed and approved, that people are provided with the information they need to help them decide whether they wish to take part, and that their opportunity to do so is maximised by simplifying the processes by which high quality research is assessed. In doing this, we will help to build both public confidence and participation in health research, and so improve the nation’s health.” [*added here for meaning]


OR DO THEY? So much for helping to build public confidence!


And from http://www.hra.nhs.uk/about-the-hra/our-committees/


“We are also responsible for Research Ethics Committees (RECs),”
 
Messages
2,125
if that’s what they wrote
"
Thanks for your enquiry.

It was passed to me, as it looked to be a media enquiry (given the copied in Bristol Cable).

I’m afraid I’m not able to comment on the specific REC decision. The REC provides independent advice, with a focus on protecting the rights, dignity and well-being of those taking part in health research and to facilitate high quality research.

I am able to point you in the direction of some of the guidance the HRA has produced, which might provide helpful context. The HRA publishes detailed guidance on when a study is considered research, details of which can be found here:

http://www.hra.nhs.uk/resources/before-you-apply/is-it-research/

Our guidance includes a clear definition of service evaluation, which you can find here:

http://www.hra-decisiontools.org.uk/research/glossary.html#S

I hope this helps.

If you do have any future media enquiries, do feel free to come straight to the comms team, and we’ll help you where we can. The general email address is hra.comms@nhs.net, or you’re welcome to contact me directly.

Dave
"
my bolding.

eta: I have replied
"
I am not looking for comment on the specific original REC decision, I just find it strange that the same number has been used multiple times over a number of years for numerous research papers........"
plus a bit more.
 
Last edited:

lilpink

Senior Member
Messages
988
Location
UK
do feel free to come straight to the comms team

Quote presumably from HRA's comms team representative - "do feel free to come straight to the comms team"


I expect Stephen Robinson, HRA's Corporate Secretary, would be pleased to help you instead, if you wanted to email the HRA again using the General enquiries: contact.hra@nhs.net address instead with subject heading 'FAO Stephen Robinson, Corporate Secretary' - only a suggestion
3.gif