• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Good science: It's nice to be nice but it's more important to be honest

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Great blog on why honesty is needed in science, even though it can be hard to take criticism. It's specifically about neuroscience (which includes all those MRI and fMRI mecfs brain studies) but the principles apply far more widely.

NeuroChambers: It's nice to be nice but it's more important to be honest
...Our field is peppered with small studies pumped out by petty fiefdoms, each vying for a coveted spot in high impact journals so we can have careers and get tenure and maybe make a few discoveries along the way. It would be disingenous to say that I'm any different. I've got my own fiefdom, just like the rest. It's no less petty; I am no better than anyone else.

...When I look at fMRI studies like the one this morning, I see how far we need to come as a field.

...Some folks get upset at the direct nature of post publication peer review... they might think such criticism is an attack on the integrity of the researchers -- that robust post-publication-peer-review, pointing out probable bias or low reproducibility, is tantamount to an accusation of misconduct.

This is false because questionable practices aren't the same as fraud and bias isn't the same as misconduct. Much, if not most, research bias happens unconsciously. It can and does distort our results despite our best efforts because we're humans rather than robots. I believe many in our community are not only blind to unconscious bias, they're blind to the possibility of unconscious bias.
I've probably quoted too much, and please do read the full blog.
 

Invisible Woman

Senior Member
Messages
1,267
I was always taught that the best way to prove a theory was to do your best to disprove it. This helps to see the flaws and identify any unconscious assumptions. Also, don't let your ego stop you being the best you can - if someone takes the time to criticize then they are actually doing you a favour. Listen to what they say, evaluate it and learn. Your work will be all the better for it.
 

SOC

Senior Member
Messages
7,849
Does a researcher see their unconscious bias?

Does a fish see water?
Does [anyone, even patients] see their unconscious bias?
Comes free with being human.
All true. But that is why there is clear guidance on how to design a research study properly to minimize unconscious bias. The problem is that we're seeing so much very poorly designed research in medicine that deliberately ignores the guidance intended to protect us from researcher bias that we're starting to think it's normal and accepted. The researchers are either very, very poorly trained (shame on their universities) or they are knowingly ignoring procedures that would reduce their bias. They don't get off the hook with "everyone has bias" because science recognizes the potential for bias and has ways to account for it.
 

helen1

Senior Member
Messages
1,033
Location
Canada
This is a bit off topic but somewhat follows the title topic, which is I've noticed other ME online groups, to which I belong, have mostly members who are very keen on supporting one another in a 'nice' way and very upset about anyone 'honest' who criticizes or voices skepticism.

I've thought that isn't very helpful for learning on a personal level. I read and participate a bit in those groups to get a different perspective than I get here, but do find them biased against true sharing of thoughts and information.They have a conscious bias toward only positive comments.

I wonder what traits the ability to accept or even embrace criticism are based on.
 
Last edited:

SOC

Senior Member
Messages
7,849
This is a bit off topic but somewhat follows the title topic, which is I've noticed other ME online groups, to which I belong, have mostly members who are very keen on supporting one another in a 'nice' way and very upset about anyone 'honest' who criticizes or voices skepticism.

I've thought that isn't very helpful for learning on a personal level. I read and participate a bit in those groups to get a different perspective than I get here, but do find them biased against true sharing of thoughts and information.They have a conscious bias toward only positive comments.

I wonder what traits the ability to accept or even embrace criticism is based on.
Many people confuse 'nice' with 'always willing to agree' and 'mean' or 'hostile' with 'having a different opinion'. They see disagreement as insult. It's unfortunate because it inhibits conversation and debate, which then limits knowledge. While it's certainly possible to disagree in a rude way, disagreement by itself is not rude, it's just having a different opinion. It seems to me that a common trait among people who have these kinds of confusion is that they were never taught to distinguish between opinion and fact. Therefore, they believe that their opinion alone should be accepted as reality and that disagreeing with their opinion is just offensive stubborness, or a personal comment on their character or intelligence.
 

helen1

Senior Member
Messages
1,033
Location
Canada
I think that's probably a big part of it, seeing opinion as fact. I think it also has something to do with emotional health in the sense of lack of defensiveness.

Being able to disagree respectfully is such an important skill. Just imagine if everyone had this ability.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
This is nearly two years old, but I hadn't seen before so just putting here for now.

The Lancet: Research: increasing value, reducing waste
The Lancet presents a Series of five papers about research:
  • Iain Chalmers et al discuss how decisions about which research to fund should be based on issues relevant to users of research. [I'm guessing that even includes patients, assuming they have a use for research into treatments for their illness]
  • John Ioannidis et al consider improvements in the appropriateness of research design, methods, and analysis.
  • issues of efficient research regulation and management.
  • An-Wen Chan et al examine the role of fully accessible research information.
  • Paul Glasziou et al discuss the importance of unbiased and usable research reports.
These papers set out some of the most pressing issues, recommend how to increase value and reduce waste in biomedical research, and propose metrics for stakeholders to monitor the implementation of these recommendations.

Good to see the Lancet leading the way.
 

SOC

Senior Member
Messages
7,849
This is nearly two years old, but I hadn't seen before so just putting here for now.

The Lancet: Research: increasing value, reducing waste


Good to see the Lancet leading the way.
Only if they're going to take it seriously and apply it to what they publish. They can make themselves look good by publishing all the papers in the world about what good research and publication should look like. If they don't take it in and make it part of their policies, it's all window-dressing. Lip service does the world no good.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Messages
3,263
Being able to disagree respectfully is such an important skill.
I think we're not naturally cut out to do this, we naturally see an attack on our views as an attack on ourselves. It takes practice and discipline to separate ourselves from the picture, and even more to let go of our view and change it when necessary. I know I'm no genius at this. Just gotta keep trying.

If I feel criticised, I find that a cooling off period really helps. Doesn't feel so personal after some time out.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Great blog on why honesty is needed in science, even though it can be hard to take criticism. It's specifically about neuroscience (which includes all those MRI and fMRI mecfs brain studies) but the principles apply far more widely.

NeuroChambers: It's nice to be nice but it's more important to be honest
.

New post from Neuroskeptic on problems with fMRI (which measures which parts of the brain are active in a particular task), a brain imaging technique with a very shaky reputation for reliablility:
False Positive fMRI Revisited - Neuroskeptic

Basically, the standard method for analysing fMRI uses an awful lot of statistical processing (as do all methods, there's a huge amount of data from each subject). The standard method is supposed to give a false positive rate for 5%, the norm in research studies. Neuroskeptic blogs about a new paper showing the actual false positive rate is much higher than this, for the technique that underlies most fMRI studies. Which is likely to make fMRI findings very shaky, especially for small studies (which is almost all of them). Oops.

While I'm here, I like this too:
 
Last edited: