• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Ioannidis: This I believe in genetics: discovery can be a nuisance, replication is science...

Messages
13,774
I actually thought that this was less interesting than some of his other papers which have been highlighted on this forum previously (this could be because I'm not well informed about genetics or the controversies in this area), but it's short and open access so I thought I'd post it up

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3596761/

Also, some parts of it, about the churning out of data which doesn't increase understanding reminds me of the standard CFS papers which get pumped out, like this one I started a thread on yesterday (on parental expectations of their children's IQ): http://forums.phoenixrising.me/inde...rental-expectations-of-their-childs-iq.22629/

I'm sure we all recognise these problems:

Based on what we have started to surmise empirically, most of these claimed discoveries are likely to be either totally false preliminary observations (Ioannidis, 2005) or substantially exaggerated results (Ioannidis, 2008), a consequence of the extreme multiplicity of the probed data-space, the winner's curse (Zollner and Pritchard, 2007), and other biases. “Negative” results have almost disappeared from many scientific fields, especially those with “softer” measurements and more flexible analytical tools (Fanelli, 2010). Results procured by the most popular research sub-fields seem to have the lowest reliability (Pfeiffer and Hoffman, 2009).

I thought that this was a bit interesting on 'merit' in science, and I've been surprised by the willingness to congratulate those researchers who seem simply to have gained the funding to do certain research. I wonder if there's also an important political component to this, with those in positions of social power increasingly able to select those who gain access to funding, and thus become recognised as 'good' scientists. Ioannidis talks about bias, and of course the biases of those with social power will hold more sway than the biases of those in positions of social weakness, but there is also room for a more intentional corruption of the meritocracy of science, with funding allowing for increasing prestige to flow to those who can be 'trusted' to be on-side. It easy to talk in a way which allows others to dismiss you as a conspiracy theorist, but it seems foolish to assume that there is no political interest in promoting those with certain views to positions of scientific authority.

It seems likely that there is an extraordinary large number of small, weak effects and links (“risks” in epidemiological language), barely discernible from measurement error and diverse potential biases. Single discoveries made in single databases are likely to mean very little, they are mostly a nuisance that propagates confusion in the literature. Exceptions certainly occur, and some strong/large effects may still exist, awaiting discovery. Even then, it is unlikely that the discoverer who hits upon them will have any more merit than the thousands of other researchers who only come across the flooding multitude of weak or false effects. The process of rewarding discoverers claiming large effects (be that with grants, tenure, or Nobel prizes) may eventually become indistinguishable from running a lottery. If we add human nature, biases, and conflicts (Ioannidis, 2011), a lottery system may be even preferable.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
I haven't read this paper yet, that will have to wait till later next week, but I think your concerns about social/political bias are well founded. Such bias has nothing to do with conspiracy, and for some to argue that its just a conspiracy theory is a fallacious counter-argument in my view. I want to look more closely at these sorts of ideas in my book. It is probable public policy on science and medicine, ignoring personal influence, will distort research and introduce bias over time. NICE has a very limited focus, and that would induce bias too. Defining, measuring and accounting for such bias is one of the issues in medicine.

A much greater concern to me is the return of empiricism - build up facts in support of an argument but do not test them. This was largely discredited but its slowly returning, and its very detrimental to science. When done in experimental design it leads to verificationism, and this promotes (at a personal level) confirmation bias. I will have heaps to say about this in the next year or two.
 
Messages
13,774
Such bias has nothing to do with conspiracy, and for some to argue that its just a conspiracy theory is a fallacious counter-argument in my view

I agree. But there is a worrying tendency for some people to try to view as conspiratorial anything which questions the legitimacy of certain types of authority, so it's something I try to be aware of.

build up facts in support of an argument but do not test them.

Yeah - I find this irritating. I think that I like to try to demolish my own beliefs - find a weak spot and then pick them apart. The loss of my faith in 'science' has been the most recent destruction of a cherished (but lazy) foundational assumption. It's nice to realise how wrong one is about everything on a semi-regular basis.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Thanks for posting this, more great material from the Replication King.

Actually, I thought it was just as interesting as his other stuff, in part because he was so forthright - the freedom of an opinion piece. The problem of gene and gene expression studies is that they generate vast amouts of data, eg 1,000s of SNPs (mutations) can be screened in a single experiment: that makes it very easy to throw up false positives or false associations.

Replication offers a realistic chance of maintaining the scientific literature reasonably noise-free. Genetics has shown clearly how important this is. Human genome epidemiology was radically transformed in the last decade by the adoption of a rigorous replication culture.

While the vast majority of claims for genetic associations based on biological plausibility speculations and performed by single teams without replication were apparently wrong (Ioannidis, 2011), large meta-analyses of genome-wide association studies using agnostic platforms and sine-qua-non, rigorous replication across multiple teams and multiple datasets has yielded thousands of associations with unquestionably high credibility (Hindorff et al., 2009).

How many other scientific fields are still conducting studies based on biological plausibility speculations and performed by single teams without replication? Probably most of the literature in diverse fields has been based on these same premises and will likely collapse once rigorous replication practices are adopted.
I do like kick-ass prose at times.

A much greater concern to me is the return of empiricism - build up facts in support of an argument but do not test them. This was largely discredited but its slowly returning, and its very detrimental to science. When done in experimental design it leads to verificationism, and this promotes (at a personal level) confirmation bias. I will have heaps to say about this in the next year or two.
Good to see you back here - hope you are mended and in 'normal' health once again.

I see what you mean about 'verificationism', and I think it's true to some extent that researchers set out to prove their hypotheses right, rather than to find out what's really going on. But I think it goes further than that, where often researchers choose to ignore their own findings when it suits them - or to later cite their own evidence while ignoring the serious flaws with said evidence that they acknowledged at the time of publication. "The study is way too small and the results need confirming on larger samples; due to the cross-sectional design, no conclusions can be drawn about causation" becomes "inapprorpiate beliefs cause fatigue".
 
Messages
13,774
Actually, I thought it was just as interesting as his other stuff, in part because he was so forthright - the freedom of an opinion piece.

Pleased to hear you enjoyed it. You might be familiar enough with genetics to get more out of it.
 

Allyson

Senior Member
Messages
1,684
Location
Australia, Melbourne
Thanks for posting this, more great material from the Replication King.

Actually, I thought it was just as interesting as his other stuff, in part because he was so forthright - the freedom of an opinion piece. The problem of gene and gene expression studies is that they generate vast amouts of data, eg 1,000s of SNPs (mutations) can be screened in a single experiment: that makes it very easy to throw up false positives or false associations.
P

personally i think epigentics is our only hope for a cure and for adequate testing

i phoned for an appt at a major pblic hospital genetics depatment her in Australia and after they acepted me based on my symptoms and GP referral they took a full family tree over the phone - ie all close reelatives and any symptoms they have
and they have ordered an echocardiogram for me
and requested my BP ahdn heart tests from my BP specialist who i see for OI

that is even before my first appointment -

and all at no cost
let's hope this i a sign of improvement
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
I see what you mean about 'verificationism', and I think it's true to some extent that researchers set out to prove their hypotheses right, rather than to find out what's really going on. But I think it goes further than that, where often researchers choose to ignore their own findings when it suits them - or to later cite their own evidence while ignoring the serious flaws with said evidence that they acknowledged at the time of publication. "The study is way too small and the results need confirming on larger samples; due to the cross-sectional design, no conclusions can be drawn about causation" becomes "inapprorpiate beliefs cause fatigue".

When they design to confirm their hypotheses, shy away from testing them, and ignore their own data and other contrary evidence, I call it dogmatic verificationism. This is supported by zombie science - entrenched interests supporting science by politics and funding that fits with its agenda.

I am not home yet so my productivity is close to zero. Next week I expect to be home.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Interesting blog, Alex.

I suspect that for CFS, what will change prevailing views of the illness will not be the power of any argument, or pointing out endless flaws in theories, but hard evidence of specific alternative causes.

Usually in science it's incredibly hard to directly disprove any model, particularly broad and flexible models, because proving a negative is so hard. Whereas showing model B causes the illness usually does for model A. Eg the classic case of stomach ulcers being caused by H Pylori, rather than being the result of stress.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
Simon, I think that diagnosis, treatment and cure will come from research pretty much as you describe. What changed perspectives for me was the realization that whereas good science is the answer, politics determines how fast we get there. It influences the strength of alternative views with doctors and government, it influences publication, and it influences funding. The politics of ME is what has slowed it all down and resulted in decades of poor research effort.

H. Pylori is something I looked into. From cause to treatment was more like 108 years, largely due to dogma and politics. We face the same issues. See my blog here: here-we-go-round-the-merry-go-round-part-one

In the case of H. Pylori it was patient pressure that forced acceptance. The science was being overlooked, and there was strong pharmaceutical industry influence to continue overlooking it. Barry Marshal and his partner ( I forget his name) deserved the Nobel prize, but the medical profession was forced into accepting the result, it did not come from rapid recognition of the science.
 

Simon

Senior Member
Messages
3,789
Location
Monmouth, UK
Very interesting about H Pylori - I'd always assumed the original finding made a big impact, not least because the researcher infected himself to prove his theory. Do you have any more information about the role of patient pressure there?
 
Messages
13,774
re peptic ulcers, people might be interested in this 'Testimony' thing from the Wellcome institute. I've flicked through a few of them, reading the sections of most interest, and there's one on peptic ulcers. They tend to be quite Establishment in the approach they take and views they represent (important people testify... not patients (unless I missed them all)), but I still found it interesting. They also vary in tone and approach, and I found the peptic ulcers one to be more Establishment than some of the others.

http://archive.org/search.php?query...type:collection&sort=-avg_rating;-num_reviews