• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of, and finding treatments for, complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia, long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Trial By Error: No Ethical Review of Crawley School Absence Study

lilpink

lilpink

Senior Member
Messages
988
Location
UK
Thanks to the mods for enabling me to edit this post in order to clarify what is going on in this thread. This thread is the product of two merged threads, the first of which started life with the post below, the second of which was started when David Tuller, using the information associated with said post, created this blog (and the newly changed title of this thread) here : http://www.virology.ws/2017/08/28/trial-by-error-no-ethical-review-of-crawley-school-absence-study/

I hope that now makes sense. I expect as your scroll you will see some duplicate posts where the threads have merged, so please bear with them.

Subsequent to the first Tuller blog on this subject we now have a second blog about a FOIA request to try to access more specific detail which is self evident here: http://www.virology.ws/2017/08/30/10013/

*********************************************************

Original posting starts here:


I have been given permission by a 3rd party to post this information here on PR Forum. I think this is important and I'd query whether or not it's acceptable to brush these concerns under the rug as BMJ Open appear to have done? Please bear with me if responses to comments are delayed, this is due entirely to health concerns on my part... I will be unable to respond as quickly as I would like to do.

*********


Astonishingly poor response by BMJ Open to ethical concerns raised about Crawley school absence study -http://bmjopen.bmj.com/content/1/2/e000252.info

Crawley EM, Emond AM, Sterne JAC

Unidentified Chronic Fatigue Syndrome/myalgic encephalomyelitis (CFS/ME) is a major cause of school absence: surveillance outcomes from school-based clinics

BMJ Open 2011;1:e000252. doi: 10.1136/bmjopen-2011-000252


In this paper, under ‘Ethics approval’ the authors state that:

“The clinical service in this study was provided as an outreach from the Bath specialist CFS/ME service. The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of data from children and young people seen by the CFS/ME specialist service were part of service evaluation and as such did not require ethical review by the NHS Research Ethics Committee or approval from the NHS R&D office (REC reference number 07/Q2006/48).”


The REC reference number (07/Q2006/48) which they quote for this study does not appear to relate to this study at all but to a local longitudinal pediatric study “What happens to children with CFS/ME? The study of a longitudinal cohort of children who access a paediatric CFS/ME service” which the REC determined was a service evaluation study in May 2007…….four years previously.


The full details of this REC application and decision were accessed via a FOI request to the HRA and there appears to be nothing in the REC’s decision that relates to this school-based study. The REC reference number (07/Q2006/48) has been used to negate the need for ethics approval in at least 8 other published papers by Crawley and her colleagues. Some of these studies are in BMJ journals, some use the National Outcome Database or ‘NOD’, a huge national database of NHS ME/CFS patients which was set up and run by a University of Bristol team, apparently without any specific research ethics approval. This particular school absence study does not say that it uses NOD data.


A problem here is that this study was supposed to be a ‘service evaluation’ study, yet it seems to have been presented by BMJ Open as a research-type study but without research ethics approval/REC review. NB, It not only has a ‘Hypothesis’ in the Article Summary but a ‘Research Question’ also. It is also presented by the BMJ Open under the Heading ‘Paediatrics’ – subheading – ‘Research’. It seems ridiculous to claim that it is a ‘service evaluation’ study and not therefore requiring of research ethical approval. (For more information search wales nhs defining research for a pdf with a useful guide on how to differentiate research from audit and service evaluation.)


Why is this important? Service evaluation studies are not required to go through the research ethics approval process / REC review, and informed patient consent is not mandatory for service evaluation studies.


The Peer Reviewer, Matthew Hotopf also appeared somewhat surprised or concerned regarding the ethics approval for this study as evidenced in the Review History - http://bmjopen.bmj.com/content/1/2/e000252.reviewer-comments .

Look under ‘REPORTING & ETHICS’ – here Hotopf notes his surprise that the REC passed this study as ‘service evaluation’. It’s worth reading this closely. He then asks the authors for a more detailed description of the recruitment and consent process, in particular – what information was given to parents, whether they could opt out, and what consent procedures and information were given to those who went on to have a clinic visit. The authors’ response appears to answer none of those questions adequately yet the BMJ Open then proceeded to publish the study. The onus must surely be on reputable scientific journals such as the BMJ to check that studies to be published have the requisite ethical approval/s and, if in any doubt, to double check this, especially when warning bells have been sounded by their own peer review process.


And what was the BMJ Open’s response to the complaint they received about this paper? It took them from the 30th January to the 13th June (both this year) – i.e more than 4 months - to reply to these ethical concerns. During that time, there was no ‘Expression of Concern’ placed on the paper even though COPE Retraction guidelines say that if an investigation is taking an unreasonable length of time to complete, or if there is inconclusive evidence of wrongdoing, then an ‘Expression of Concern’ should be considered. NB It shouldn’t have taken BMJ Open long to investigate this because the REC information was supplied to them.


When the BMJ Open finally replied they started their explanation by referencing a University of Bristol statement on the NOD from this link - http://www.bristol.ac.uk/red/research-governance/researchintegrity.html (half way down the page – middle link)

Rather strange, to say the least, when this particular complaint was about a study that doesn’t appear to mention the NOD!



The BMJ Open then admitted that “the article published in BMJ Open is not strictly a service evaluation” but went on to say that, in agreement with Bristol University’s statement on the NOD [which in turn references the HRA’s ‘Governance Arrangements for Research Ethics Committees (GAfREC)- http://www.hra.nhs.uk/resources/res...-arrangements-for-research-ethics-committees/ ] further ethical approval would not have been required for the study because it constituted a secondary use of the information that had been collected in the course of normal care/treatment for the patients. However, there is a caveat in HRA’s guidelines/rules that indicates that in such cases ‘service evaluation’ use applies only “provided that the patients or service users are not identifiable to the research team in carrying out the research”. Now according to the BMJ Open response these criteria were met. But it doesn’t take much effort to find information from the published paper that suggests they weren’t. You will see in the Footnotes of this paperunder the heading ‘Contributors’ that ‘EMC conceived the idea for this study, conducted the school clinics, analysed the data with support from JACS and wrote the first draft of the paper.’ So, it appears that this study was a new idea, conceived by Dr Crawley……and the children/patients involved WERE identifiable to the research team since Esther Crawley conducted the school clinics and then analyzed the data. As described in the paper ‘children identified in school clinics as having fatigue were invited to attend the Bath specialist CFS/ME service’, so not only were the children identified by Esther Crawley but they were then invited to attend the Bath CFS/ME service which was presumably run by…. guess who.

This has been pointed out to BMJ Open but, disappointingly, they have not responded.

BMJ Open did not respond to the concerns raised about the REC reference number either.


So, to summarize:

· This was a study involving vulnerable children

· The REC reference number provided by the authors for this study appears to be for a different study, according to information relating to the REC number provided under FOIA request from the REC. BMJ Open failed to address this concern in their response to the complaint

· In the peer review process, the authors appear to have side-stepped answering questions put by the peer reviewer regarding research ethics and patient consent

· BMJ Open published anyway

· BMJ Open have admitted that it wasn’t strictly a service evaluation

· The study does not appear to meet the ‘service evaluation’ criteria that BMJ Open claims it does, but BMJ Open have failed to respond when challenged with this

· Serious questions remain over whether this was a ‘service evaluation’ or a research study masquerading as a ‘service evaluation’.
 
Last edited:
Skippa

Skippa

Anti-BS
Messages
841
"Hahahaaaaa blagging scientific studies and influencing policy is a doddle. These fools are either in my circle of influence or are inconsequential muppets who accept whatever we throw at them. You just set up a study about one thing, pitch it as another, and when asked for references and any supporting proofs just throw in study #s that are slightly related and already green-lighted. Any further intrusions can simply be ignored, and noone bats an eyelid. We can do anything we want. These suckers lap it all up. Muhahahhahaaa..."

- These people behind closed doors. Probably.

This kind of stuff used to make for great television drama series'. We all used to watch conspiracy shows, things that go all the way to the top, "yes sir Mr President". Oh how we loved the entertainment, we said "imagine if this kind of stuff happened in real life".

Heh.
 
U

user9876

Senior Member
Messages
4,556
I think it just sounds like incompetence to me. I suspect Crawley was doing the clinics and thought that she could push her views as a research paper and it was too late for ethics approval but seemed like it could go under a different heading.

Its not like a UK ethics committee would stop any project. I suspect if someone proposed a project to look at bleeding times post decapitation then the ethics committees would approve it as long as the right forms are filled in. They certainly don't do an in-depth look at ethics of a proposal.
 
U

user9876

Senior Member
Messages
4,556
The attendance officer in each school identified all children in years 7e11 (ages 11e16 years) who had missed $20% of school over a 6-week term. Children were excluded if they only had a single episode of absence (eg, a 2-week illness), a medical illness that could reasonably explain the absence, a known hospital admission or were known to have been on holiday or to be truanting. Families of the remaining children were sent a letter from the school that invited them to meet with a paediatrician from the Bath specialist CFS/ME team (EMC) and a member of school staff to discuss why their child was missing school.
Click to expand...

Given the involvement of the "attendance officer" (i.e. social services) then I very much suspect there was no informed consent here but implied threats over action because children are missing school and hence forced attendance of a clinic (although they only got 76.7% to actually attend). They did have 8 kids who were truanting but then if you are likely to get into trouble then perhaps as a kid you do say yes I'm too tired for school. Or perhaps some kids are too tired due to late night video gaming!

Of the 23 children diagnosed as having CFS/ME, four were given advice about managing CFS/ME but did not attend follow-up appointments or return follow-up inventories. Of the remaining 19 children, 12 (63.2%) were attending full-time school by 6 months, of whom six had made a full recovery and were attending full-time school with minimal advice after 6 weeks. One housebound child improved and was attending school part time every day by 3 months. For the remaining six children, school attendance did not alter significantly at 6 months.
Click to expand...

So 6 ME patients got better within 6 weeks so this really brings into question the quality of the diagnosis.
 
U

user9876

Senior Member
Messages
4,556
user9876 said:
I think it just sounds like incompetence to me. I suspect Crawley was doing the clinics and thought that she could push her views as a research paper and it was too late for ethics approval but seemed like it could go under a different heading.

Its not like a UK ethics committee would stop any project. I suspect if someone proposed a project to look at bleeding times post decapitation then the ethics committees would approve it as long as the right forms are filled in. They certainly don't do an in-depth look at ethics of a proposal.
Click to expand...

I've changed my mind this is essentially an experimental service and thus I feel even to run the clinics they should have got approval. Its not a standard part of their service but a test to see if extending the service would bring them more work.


They claim funding for the project so it would be interesting what the funding proposal said.

Funding This work was supported by the Linbury Trust Grant number LIN 1750. The funders did not take part in the collection, analysis or interpretation of the data and have not been involved in writing this article for publication. The researchers are independent from the funders.
Click to expand...
 
Jonathan Edwards

Jonathan Edwards

"Gibberish"
Messages
5,256
user9876 said:
Its not like a UK ethics committee would stop any project. I suspect if someone proposed a project to look at bleeding times post decapitation then the ethics committees would approve it as long as the right forms are filled in. They certainly don't do an in-depth look at ethics of a proposal.
Click to expand...

When I was on the UCL ethics committee I spent many hours wading through projects for a committee meeting and we quite often sent projects back as not fit to be approved. When more recently I applied for approval i had very close grilling on what might have seemed minor issues but were in truth important.

Our ethics committee took a particular close look at any studies involving children. What was often passed off as routine care quite often was not. If there is a chance of these patients being identifiable either by themselves or others there are major issues.
 
U

user9876

Senior Member
Messages
4,556
Jonathan Edwards said:
When I was on the UCL ethics committee I spent many hours wading through projects for a committee meeting and we quite often sent projects back as not fit to be approved. When more recently I applied for approval i had very close grilling on what might have seemed minor issues but were in truth important.

Our ethics committee took a particular close look at any studies involving children. What was often passed off as routine care quite often was not. If there is a chance of these patients being identifiable either by themselves or others there are major issues.
Click to expand...

I'm sure you gave them a careful review and I hope my impressions are wrong.

I get a very different impression for ME studies but maybe that is because they are not drug therapy but just psychotherapy and there is a belief that they can't do any harm. It seems like potential harm wasn't considered in the approval of the SMILE and FITNET-NHS trials where in the latter case after people raise issues Crawley quotes the Cochrane review as saying GET is safe - but they didn't look further to see the caveats that there is insufficient data.

I did think a fun research project would be to get various academics to submit spoof trials to ethics committees and see which (or who) got through. I think someone did something similar with papers which were written by a paper generator. I also remember that someone submitted the call for papers to a conference and got accepted.
 
lilpink

lilpink

Senior Member
Messages
988
Location
UK
It might be interesting to copy Hotopf's Peer Review concerns here:

"The paragraph at the bottom of p7 describes the position with the research ethics committee, who apparently thought that this was not research. If ethics approval was sought and not obtained because the REC made that decision, the authors cannot do much about it. However the wording in the paragraph is a bit ambiguous. My view is that it is understandable that the REC might see use of routine data from the existing clinical service as not being research (which is what the paragraph seems to say), but it is surprising that they did not see the surveillance component as research. Children who are unknown to services were being contacted using information from their schools and it seems to me that there are significant issues of confidentiality and data protection which, in my experience as a researcher and one time REC member, I am surprised the REC did not think amounted to research. Assuming the REC was fully aware of these issues, and still made a decision that the work was not research, then it would be unfair to oppose publication on these grounds, but the authors should make a fuller explanation, and in the interest of openess might wish to make their application to the REC and subsequent correspondence available with the publication. Given these comments, I think that the authors should describe the process of recruitment and consent in more detail than they do. What information was provided to parents? Were parents able to opt out if they did not want data on their child handed to the researcher? What were the consent procedures and information given to children and parents identified as eligible for the clinic visit?"

From here (as linked in the initial posting): http://bmjopen.bmj.com/content/bmjopen/1/2/e000252.reviewer-comments.pdf
 
U

user9876

Senior Member
Messages
4,556
lilpink said:
It might be interesting to copy Hotopf's Peer Review concerns here:

"The paragraph at the bottom of p7 describes the position with the research ethics committee, who apparently thought that this was not research. If ethics approval was sought and not obtained because the REC made that decision, the authors cannot do much about it. However the wording in the paragraph is a bit ambiguous. My view is that it is understandable that the REC might see use of routine data from the existing clinical service as not being research (which is what the paragraph seems to say), but it is surprising that they did not see the surveillance component as research. Children who are unknown to services were being contacted using information from their schools and it seems to me that there are significant issues of confidentiality and data protection which, in my experience as a researcher and one time REC member, I am surprised the REC did not think amounted to research. Assuming the REC was fully aware of these issues, and still made a decision that the work was not research, then it would be unfair to oppose publication on these grounds, but the authors should make a fuller explanation, and in the interest of openess might wish to make their application to the REC and subsequent correspondence available with the publication. Given these comments, I think that the authors should describe the process of recruitment and consent in more detail than they do. What information was provided to parents? Were parents able to opt out if they did not want data on their child handed to the researcher? What were the consent procedures and information given to children and parents identified as eligible for the clinic visit?"

From here (as linked in the initial posting): http://bmjopen.bmj.com/content/bmjopen/1/2/e000252.reviewer-comments.pdf
Click to expand...

The response is interesting basically they change the language to cover up. But I thought this bit was written to mislead:

We agree that the reasons that ethical committee approval was not required for this clinical evaluation study requires clarification. Our paper reports results from a clinical outreach project from the CFS/ME specialist service working in partnership with local authority attendance officers to improve school attendance. As described in the paper, the specialist service has been advised that ethical approval for routine collection and analysis of service data is not required. The author EC is a community paediatrician and children were seen in school-based clinics. The schools invited children and their parents to attend and a member of the school staff (usually the attendance officer) was present. Information about the children was held by the schools until the child was referred to the specialist CFS/ME service. The project has been of great interest to the Department of Education who included tt last year as an exemplar in their training for attendance officers in the UK. Based on the results of this outreach project, school nurses in the local authority now routinely assess children missing school for fatigue. We checked with the co-ordinator for the local REC that recording outcomes on school based clinics run by school nurses is part of service evaluation (and therefore does not require a submission to Ethics) and they have agreed that it is.
Although we regard “surveillance” as part of routine clinical care for community paediatricians, we acknowledge that this term can have other meanings and recognise that this makes the paper read as
a research study rather than reporting on a clinical project.
Click to expand...

What this is basically saying is they ran an experimental screening service with schools with Crawley as a doctor and then started to roll something out with school nurses. But checked with the ethics committee that they could use data from clinics run by school nurses. This was clearly an experiment in service development.

Also of interest here is that they seem to suggest that the attendance officer or other member of staff was present in any 'clinic'.

They note that the schools retained data yet in their paper they quote other reasons for absence so clearly they had some access to data.
 
You must log in or register to reply here.