Thanks to the mods for enabling me to edit this post in order to clarify what is going on in this thread. This thread is the product of two merged threads, the first of which started life with the post below, the second of which was started when David Tuller, using the information associated with said post, created this blog (and the newly changed title of this thread) here : http://www.virology.ws/2017/08/28/trial-by-error-no-ethical-review-of-crawley-school-absence-study/
I hope that now makes sense. I expect as your scroll you will see some duplicate posts where the threads have merged, so please bear with them.
Subsequent to the first Tuller blog on this subject we now have a second blog about a FOIA request to try to access more specific detail which is self evident here: http://www.virology.ws/2017/08/30/10013/
*********************************************************
Original posting starts here:
I have been given permission by a 3rd party to post this information here on PR Forum. I think this is important and I'd query whether or not it's acceptable to brush these concerns under the rug as BMJ Open appear to have done? Please bear with me if responses to comments are delayed, this is due entirely to health concerns on my part... I will be unable to respond as quickly as I would like to do.
*********
Astonishingly poor response by BMJ Open to ethical concerns raised about Crawley school absence study -http://bmjopen.bmj.com/content/1/2/e000252.info
Crawley EM, Emond AM, Sterne JAC
Unidentified Chronic Fatigue Syndrome/myalgic encephalomyelitis (CFS/ME) is a major cause of school absence: surveillance outcomes from school-based clinics
BMJ Open 2011;1:e000252. doi: 10.1136/bmjopen-2011-000252
In this paper, under ‘Ethics approval’ the authors state that:
“The clinical service in this study was provided as an outreach from the Bath specialist CFS/ME service. The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of data from children and young people seen by the CFS/ME specialist service were part of service evaluation and as such did not require ethical review by the NHS Research Ethics Committee or approval from the NHS R&D office (REC reference number 07/Q2006/48).”
The REC reference number (07/Q2006/48) which they quote for this study does not appear to relate to this study at all but to a local longitudinal pediatric study “What happens to children with CFS/ME? The study of a longitudinal cohort of children who access a paediatric CFS/ME service” which the REC determined was a service evaluation study in May 2007…….four years previously.
The full details of this REC application and decision were accessed via a FOI request to the HRA and there appears to be nothing in the REC’s decision that relates to this school-based study. The REC reference number (07/Q2006/48) has been used to negate the need for ethics approval in at least 8 other published papers by Crawley and her colleagues. Some of these studies are in BMJ journals, some use the National Outcome Database or ‘NOD’, a huge national database of NHS ME/CFS patients which was set up and run by a University of Bristol team, apparently without any specific research ethics approval. This particular school absence study does not say that it uses NOD data.
A problem here is that this study was supposed to be a ‘service evaluation’ study, yet it seems to have been presented by BMJ Open as a research-type study but without research ethics approval/REC review. NB, It not only has a ‘Hypothesis’ in the Article Summary but a ‘Research Question’ also. It is also presented by the BMJ Open under the Heading ‘Paediatrics’ – subheading – ‘Research’. It seems ridiculous to claim that it is a ‘service evaluation’ study and not therefore requiring of research ethical approval. (For more information search wales nhs defining research for a pdf with a useful guide on how to differentiate research from audit and service evaluation.)
Why is this important? Service evaluation studies are not required to go through the research ethics approval process / REC review, and informed patient consent is not mandatory for service evaluation studies.
The Peer Reviewer, Matthew Hotopf also appeared somewhat surprised or concerned regarding the ethics approval for this study as evidenced in the Review History - http://bmjopen.bmj.com/content/1/2/e000252.reviewer-comments .
Look under ‘REPORTING & ETHICS’ – here Hotopf notes his surprise that the REC passed this study as ‘service evaluation’. It’s worth reading this closely. He then asks the authors for a more detailed description of the recruitment and consent process, in particular – what information was given to parents, whether they could opt out, and what consent procedures and information were given to those who went on to have a clinic visit. The authors’ response appears to answer none of those questions adequately yet the BMJ Open then proceeded to publish the study. The onus must surely be on reputable scientific journals such as the BMJ to check that studies to be published have the requisite ethical approval/s and, if in any doubt, to double check this, especially when warning bells have been sounded by their own peer review process.
And what was the BMJ Open’s response to the complaint they received about this paper? It took them from the 30th January to the 13th June (both this year) – i.e more than 4 months - to reply to these ethical concerns. During that time, there was no ‘Expression of Concern’ placed on the paper even though COPE Retraction guidelines say that if an investigation is taking an unreasonable length of time to complete, or if there is inconclusive evidence of wrongdoing, then an ‘Expression of Concern’ should be considered. NB It shouldn’t have taken BMJ Open long to investigate this because the REC information was supplied to them.
When the BMJ Open finally replied they started their explanation by referencing a University of Bristol statement on the NOD from this link - http://www.bristol.ac.uk/red/research-governance/researchintegrity.html (half way down the page – middle link)
Rather strange, to say the least, when this particular complaint was about a study that doesn’t appear to mention the NOD!
The BMJ Open then admitted that “the article published in BMJ Open is not strictly a service evaluation” but went on to say that, in agreement with Bristol University’s statement on the NOD [which in turn references the HRA’s ‘Governance Arrangements for Research Ethics Committees (GAfREC)- http://www.hra.nhs.uk/resources/res...-arrangements-for-research-ethics-committees/ ] further ethical approval would not have been required for the study because it constituted a secondary use of the information that had been collected in the course of normal care/treatment for the patients. However, there is a caveat in HRA’s guidelines/rules that indicates that in such cases ‘service evaluation’ use applies only “provided that the patients or service users are not identifiable to the research team in carrying out the research”. Now according to the BMJ Open response these criteria were met. But it doesn’t take much effort to find information from the published paper that suggests they weren’t. You will see in the Footnotes of this paperunder the heading ‘Contributors’ that ‘EMC conceived the idea for this study, conducted the school clinics, analysed the data with support from JACS and wrote the first draft of the paper.’ So, it appears that this study was a new idea, conceived by Dr Crawley……and the children/patients involved WERE identifiable to the research team since Esther Crawley conducted the school clinics and then analyzed the data. As described in the paper ‘children identified in school clinics as having fatigue were invited to attend the Bath specialist CFS/ME service’, so not only were the children identified by Esther Crawley but they were then invited to attend the Bath CFS/ME service which was presumably run by…. guess who.
This has been pointed out to BMJ Open but, disappointingly, they have not responded.
BMJ Open did not respond to the concerns raised about the REC reference number either.
So, to summarize:
· This was a study involving vulnerable children
· The REC reference number provided by the authors for this study appears to be for a different study, according to information relating to the REC number provided under FOIA request from the REC. BMJ Open failed to address this concern in their response to the complaint
· In the peer review process, the authors appear to have side-stepped answering questions put by the peer reviewer regarding research ethics and patient consent
· BMJ Open published anyway
· BMJ Open have admitted that it wasn’t strictly a service evaluation
· The study does not appear to meet the ‘service evaluation’ criteria that BMJ Open claims it does, but BMJ Open have failed to respond when challenged with this
· Serious questions remain over whether this was a ‘service evaluation’ or a research study masquerading as a ‘service evaluation’.
I hope that now makes sense. I expect as your scroll you will see some duplicate posts where the threads have merged, so please bear with them.
Subsequent to the first Tuller blog on this subject we now have a second blog about a FOIA request to try to access more specific detail which is self evident here: http://www.virology.ws/2017/08/30/10013/
*********************************************************
Original posting starts here:
I have been given permission by a 3rd party to post this information here on PR Forum. I think this is important and I'd query whether or not it's acceptable to brush these concerns under the rug as BMJ Open appear to have done? Please bear with me if responses to comments are delayed, this is due entirely to health concerns on my part... I will be unable to respond as quickly as I would like to do.
*********
Astonishingly poor response by BMJ Open to ethical concerns raised about Crawley school absence study -http://bmjopen.bmj.com/content/1/2/e000252.info
Crawley EM, Emond AM, Sterne JAC
Unidentified Chronic Fatigue Syndrome/myalgic encephalomyelitis (CFS/ME) is a major cause of school absence: surveillance outcomes from school-based clinics
BMJ Open 2011;1:e000252. doi: 10.1136/bmjopen-2011-000252
In this paper, under ‘Ethics approval’ the authors state that:
“The clinical service in this study was provided as an outreach from the Bath specialist CFS/ME service. The North Somerset & South Bristol Research Ethics Committee decided that the collection and analysis of data from children and young people seen by the CFS/ME specialist service were part of service evaluation and as such did not require ethical review by the NHS Research Ethics Committee or approval from the NHS R&D office (REC reference number 07/Q2006/48).”
The REC reference number (07/Q2006/48) which they quote for this study does not appear to relate to this study at all but to a local longitudinal pediatric study “What happens to children with CFS/ME? The study of a longitudinal cohort of children who access a paediatric CFS/ME service” which the REC determined was a service evaluation study in May 2007…….four years previously.
The full details of this REC application and decision were accessed via a FOI request to the HRA and there appears to be nothing in the REC’s decision that relates to this school-based study. The REC reference number (07/Q2006/48) has been used to negate the need for ethics approval in at least 8 other published papers by Crawley and her colleagues. Some of these studies are in BMJ journals, some use the National Outcome Database or ‘NOD’, a huge national database of NHS ME/CFS patients which was set up and run by a University of Bristol team, apparently without any specific research ethics approval. This particular school absence study does not say that it uses NOD data.
A problem here is that this study was supposed to be a ‘service evaluation’ study, yet it seems to have been presented by BMJ Open as a research-type study but without research ethics approval/REC review. NB, It not only has a ‘Hypothesis’ in the Article Summary but a ‘Research Question’ also. It is also presented by the BMJ Open under the Heading ‘Paediatrics’ – subheading – ‘Research’. It seems ridiculous to claim that it is a ‘service evaluation’ study and not therefore requiring of research ethical approval. (For more information search wales nhs defining research for a pdf with a useful guide on how to differentiate research from audit and service evaluation.)
Why is this important? Service evaluation studies are not required to go through the research ethics approval process / REC review, and informed patient consent is not mandatory for service evaluation studies.
The Peer Reviewer, Matthew Hotopf also appeared somewhat surprised or concerned regarding the ethics approval for this study as evidenced in the Review History - http://bmjopen.bmj.com/content/1/2/e000252.reviewer-comments .
Look under ‘REPORTING & ETHICS’ – here Hotopf notes his surprise that the REC passed this study as ‘service evaluation’. It’s worth reading this closely. He then asks the authors for a more detailed description of the recruitment and consent process, in particular – what information was given to parents, whether they could opt out, and what consent procedures and information were given to those who went on to have a clinic visit. The authors’ response appears to answer none of those questions adequately yet the BMJ Open then proceeded to publish the study. The onus must surely be on reputable scientific journals such as the BMJ to check that studies to be published have the requisite ethical approval/s and, if in any doubt, to double check this, especially when warning bells have been sounded by their own peer review process.
And what was the BMJ Open’s response to the complaint they received about this paper? It took them from the 30th January to the 13th June (both this year) – i.e more than 4 months - to reply to these ethical concerns. During that time, there was no ‘Expression of Concern’ placed on the paper even though COPE Retraction guidelines say that if an investigation is taking an unreasonable length of time to complete, or if there is inconclusive evidence of wrongdoing, then an ‘Expression of Concern’ should be considered. NB It shouldn’t have taken BMJ Open long to investigate this because the REC information was supplied to them.
When the BMJ Open finally replied they started their explanation by referencing a University of Bristol statement on the NOD from this link - http://www.bristol.ac.uk/red/research-governance/researchintegrity.html (half way down the page – middle link)
Rather strange, to say the least, when this particular complaint was about a study that doesn’t appear to mention the NOD!
The BMJ Open then admitted that “the article published in BMJ Open is not strictly a service evaluation” but went on to say that, in agreement with Bristol University’s statement on the NOD [which in turn references the HRA’s ‘Governance Arrangements for Research Ethics Committees (GAfREC)- http://www.hra.nhs.uk/resources/res...-arrangements-for-research-ethics-committees/ ] further ethical approval would not have been required for the study because it constituted a secondary use of the information that had been collected in the course of normal care/treatment for the patients. However, there is a caveat in HRA’s guidelines/rules that indicates that in such cases ‘service evaluation’ use applies only “provided that the patients or service users are not identifiable to the research team in carrying out the research”. Now according to the BMJ Open response these criteria were met. But it doesn’t take much effort to find information from the published paper that suggests they weren’t. You will see in the Footnotes of this paperunder the heading ‘Contributors’ that ‘EMC conceived the idea for this study, conducted the school clinics, analysed the data with support from JACS and wrote the first draft of the paper.’ So, it appears that this study was a new idea, conceived by Dr Crawley……and the children/patients involved WERE identifiable to the research team since Esther Crawley conducted the school clinics and then analyzed the data. As described in the paper ‘children identified in school clinics as having fatigue were invited to attend the Bath specialist CFS/ME service’, so not only were the children identified by Esther Crawley but they were then invited to attend the Bath CFS/ME service which was presumably run by…. guess who.
This has been pointed out to BMJ Open but, disappointingly, they have not responded.
BMJ Open did not respond to the concerns raised about the REC reference number either.
So, to summarize:
· This was a study involving vulnerable children
· The REC reference number provided by the authors for this study appears to be for a different study, according to information relating to the REC number provided under FOIA request from the REC. BMJ Open failed to address this concern in their response to the complaint
· In the peer review process, the authors appear to have side-stepped answering questions put by the peer reviewer regarding research ethics and patient consent
· BMJ Open published anyway
· BMJ Open have admitted that it wasn’t strictly a service evaluation
· The study does not appear to meet the ‘service evaluation’ criteria that BMJ Open claims it does, but BMJ Open have failed to respond when challenged with this
· Serious questions remain over whether this was a ‘service evaluation’ or a research study masquerading as a ‘service evaluation’.
Last edited: