Excellent article that explains why your local MECFS doctor probably doesn't publish in Journals

[Murph]

http://slatestarcodex.com/2017/08/29/my-irb-nightmare/

This is a long piece but a good one, outlining one young psychiatrist's attempt to run a study, and the insane bureaucracy that ensued.

Some key paragraphs from near the end of the piece:

I’m capable of doing research, I think I have something to contribute to Science. I still think the bipolar screening test is inappropriate for inpatient diagnosis, and I still think that patients are being harmed by people’s reliance on it. I still think somebody should look into it and publish the results.
I’m just saying it’s not going to be me. I am done with research. People keep asking me “You seem really into science, why don’t you become a researcher?” Well…


I feel like a study that realistically could have been done by one person in a couple of hours got dragged out into hundreds of hours of paperwork hell for an entire team of miserable doctors. I think its scientific integrity was screwed up by stupid requirements like the one about breaking blinding, and the patients involved were put through unnecessary trouble by being forced to sign endless consent forms screaming to them about nonexistent risks.


I feel like I was dragged almost to the point of needing to be in a psychiatric hospital myself, while my colleagues who just used the bipolar screening test – without making the mistake of trying to check if it works – continue to do so without anybody questioning them or giving them the slightest bit of aggravation.


I feel like some scientists do amazingly crappy studies that couldn’t possibly prove anything, but get away with it because they have a well-funded team of clerks and secretaries who handle the paperwork for them. And that I, who was trying to do everything right, got ground down with so many pointless security-theater-style regulations that I’m never going to be able to do the research I would need to show they’re wrong.

 

[Woolie]

I saw this article, @Murph. I looked at it from the opposite point of view. The writer has no qualifications in research, and I think this is essential if you are to do good research. Psychiatric specialist training is about absorbing information and learning current best diagnostic and treatment practice. Its not sufficient for conducting research studies. But many doctors ignore this - its an aspect of the overconfidence thing you so often see in doctors.

This writer should contact his local University, find out if there are any trained research personnel there with an interest in the topic. They could advise on study design and help with the paperwork.

 

[Murph]

I saw this article, @Murph. I looked at it from the opposite point of view. The writer has no qualifications in research, and I think this is essential if you are to do good research. Psychiatric specialist training is about absorbing information and learning current best diagnostic and treatment practice. Its not sufficient for conducting research studies. But many doctors ignore this - its an aspect of the overconfidence thing you so often see in doctors.

This writer should contact his local University, find out if there are any trained research personnel there with an interest in the topic. They could advise on study design and help with the paperwork.

Reasonable points. he is a junior doc and obviously inexperienced. I'm not sure if you can say with confidence that his senior partner in the research also lacks research skills/ qualifications.

Overall though I'm inclined to say that bureaucracy tends byzantine. This may especially be true in a small hospital where they're not used to doing a lot of research. Other institutions may well be better.

But it seems to be a quality/quantity trade off. Either you discourage doctors from looking into things systematically and leave that to a few high powered institutions, or you encourage a larger volume of smaller less impactful studies with a higher chance of that research being wrong.. I think I prefer the latter, especially for under-researched conditions.

 

[JohnCB]

Some key paragraphs from near the end of the piece:

I haven't followed the link to the full article. I read the paragraphs you quoted.

My immediate reaction was "good". I got the immediate sense that this was a rant about PC gone mad. He is complaining about regulations put in place to protect patients after the harm done to them taking the same attitude in the past. I wouldn't want to be a participant in a study run by him. I am glad that he has been blocked.

You cannot do good science with "a study that realistically could have been done by one person in a couple of hours". It is probably a century since any good science could be done like that. I feel he is saying he knows the answer already but people won't believe him because it hasn't been published. A bit of an Anne Elk theory. I think we all have a lot of ideas like that.

Good science needs a lot of hard work and close attention to detail. If you can cope with that, then you should be able to cope with getting the paperwork right. In the end it is all paperwork anyhow - the objective is to publish a paper. What he has at present is probably appropriate for a posting on Mental Elf, with a little help from Sir.

As he says there are a lot of crappy studies out there already. There is no need to add to that pile. No, I read a typical ignorant moan from an yet another arrogant psychiatrist overrating his own capabilities.

Thank you for quoting enough to save me from reading the rest.

 

[Murph]

I haven't followed the link to the full article. I read the paragraphs you quoted.

My immediate reaction was "good". I got the immediate sense that this was a rant about PC gone mad. He is complaining about regulations put in place to protect patients after the harm done to them taking the same attitude in the past. I wouldn't want to be a participant in a study run by him. I am glad that he has been blocked.

You cannot do good science with "a study that realistically could have been done by one person in a couple of hours". It is probably a century since any good science could be done like that. I feel he is saying he knows the answer already but people won't believe him because it hasn't been published. A bit of an Anne Elk theory. I think we all have a lot of ideas like that.

Good science needs a lot of hard work and close attention to detail. If you can cope with that, then you should be able to cope with getting the paperwork right. In the end it is all paperwork anyhow - the objective is to publish a paper. What he has at present is probably appropriate for a posting on Mental Elf, with a little help from Sir.

As he says there are a lot of crappy studies out there already. There is no need to add to that pile. No, I read a typical ignorant moan from an yet another arrogant psychiatrist overrating his own capabilities.

Thank you for quoting enough to save me from reading the rest.

ahhh, the old 'have not read but strongly disagree'. An internet classic! <3

EDIT: Perhaps people imagine this study is about putting psychoactive drugs in patients. In fact it's about comparing two steps in the normal process of admitting a patient (an initial screening test and a later comprehensive test) and seeing how often they agree, as a way to test the effectiveness of the (widely used but never properly validated) initial screening test.

Basically nothing would happen to patients that wouldn't happen anyway. The research was in comparing two pieces of paper for each patient to see if they said the same thing.

The researchers worked on it for two years before giving up in the face of the logistical difficulties imposed by the various layers of review.

 

[frozenborderline]

This article was great, @Murph , and explains some of why i think there is way too little of doctors thinking like researchers--since they don't even really get opportunities to do research that may overlap greatly with clinical practice.

It also makes me wonder greatly about the "skeptic community" and people who generally argue based on scientific consensus and credentials/credibility, vs anecdotal evidence, speculation, etc.

It seems like there are so many barriers to entry that the scientific process is often skewed toward certain results. it wouldn't even be a conspiracy theory to suggest that its skewed toward big corporations that can get through all this bureaucracy with no problem. So it's going to reflect certain interests.

If this is true, then it's hard to argue that the existing scientific evidence totally outweighs layperson speculation and theory; at least if the theory makes sense. It's also a lot more grating to hear people say "well if x illness is real, how come there isn't evidence" or "well, y person who wrote an article or blog post with references isn't published in a peer-reviewed journal, or a doctor". If the barriers to entry for these things were reasonable, than these complaints would be reasonable. But since the barriers to entry are not at all reasonable, speculation and anecdotal evidence by laypeople takes on more importance. Which is unfortunate and also skews the information toward a different end. Neither of these are totally desirable states, but they're interdependent and both caused by each other.