The replication crisis in science has just begun. It will be big. (Blog)

[Hutan]

I wonder to what extent the failure to replicate may be down to poor experimental methodology, and to what extent it may be down to changes in the environment and context?

For example: the well-known French paradox, where French people have a lower incidence of coronary heart disease yet a diet quite high in saturated fats, which contradicts other studies that have found saturated fats promote heart disease.

Is this a failure of replication, or simply that in the environmental context and lifestyle of France, saturated fats do not lead to heart disease, whereas they do in other countries?

Its the "psychobabble shuffle" because if you don't understand something, you can always fall back on psychobabble explanations, because they're generally undisprovable. We have no way of measuring degree of "stress" in this community vs. others. No way of refuting the explanation. Not even any agreed definition of what stress is (whatever it is, there's no guarantee its always lower in tight-knit communities. There can be a lot of pressure there).

Or maybe it is more simple

I have read somewhere that the so-called French paradox actually arises from sloppiness/differences in the attribution of causes of death. Apparently it's a big world-wide problem. Some countries are better than others. But, the low level of heart disease in France is, according to some, attributable solely to poor data collection at the time of death.

Rubbish data in leaves everyone scrabbling around for explanations of phantom effects - and leaving the door wide open for psychobabble.

 

[Woolie]

I've heard the same, @Hutan. A lot of mythology has arisen surrounding the French diet and eating culture that could just be based on misinformation.

Others here from France may be better qualified to comment. But i lived in France for a year, and I was keenly aware of the enormous pressure on French women to stay slim. It was almost like a woman was considered a failure if she "let herself go". The idea in American books - that slimness in French women is somehow due to their eating culture (you know, sharing family meals, general emphasis on quality over quantity) - did not square at all with what I saw there. Which was a lot of women eating hardly anything at all and really struggling with it.

 

[Hutan]

For those interested in wandering down this track....
http://www.who.int/healthinfo/paper12.pdf

Poorly coded death certificates are not nearly as news worthy as French cheese, or French wine, or French family eating habits providing a protective effect against heart disease of course.

Ischaemic heart disease (IHD) is estimated to be the leading cause of mortality in the world and in high-income countries it is also the leading cause of premature mortality and disability (1). Each year IHD kills an estimated 7 million people representing 13% of all male deaths and 12% of all female deaths. Moreover, 56% of those deaths occur before age 75 years (2). Recent recorded death rates from IHD vary widely across countries; age-standardized rates for males for the population aged over 30 range from more than 900 per 100,000 in some Eastern European countries (Turkmenistan, Republic of Moldova, Belarus, Ukraine) to 84 per 100,000 in Japan; for females the corresponding range is from more than 500 per 100,000 in Turkmenistan, Republic of Moldova and Ukraine to less than 50 per 100,000 in France and Japan (see Table 1).

Understanding the huge variation in IHD mortality has been the focus of intense study in the last 20 years (3, 4, 5, 6, 7, 8). An example of the interest generated by the cross-national variation in IHD mortality rates is the so called "French Paradox" - where France has a relatively high prevalence of the major risk factors for IHD such as Tobacco and fat intake but low reported IHD mortality rates (9, 10, 11). Many descriptive and analytical epidemiological studies have been inspired by the large variation in IHD mortality rates across countries (12). Not only has the cross-sectional pattern of IHD been an important stimulus to hypothesis formulation in this area but the recorded rise in many high income countries in IHD mortality rates in the 1950s and 1960s followed by declines in age-specific death rates has led to a vast body of research to understand the broader determinants of IHD incidence, case-fatality rates and mortality rates (13, 14, 15, 16, 17, 18).

Doubts have, however, been raised about the validity of cross-national comparisons and trends within countries in IHD mortality because of variation in coding practices across countries (19, 20, 21, 22, 23, 24). There are a number of cardiovascular codes in the International Classification of Diseases Ninth and Tenth Revisions (25, 26) that may be used by physicians in different countries to assign deaths that are actually due to IHD (see Table 2). These include heart failure, ventricular dysrhythmias, generalized atherosclerosis and ill- defined descriptions and complications of heart disease. IHD deaths may be assigned to these ill-defined cardiovascular codes because of insufficient clinical information at the time of death, local medical diagnostic practices or simply by error.
...
The first group of countries includes Japan1, France, Spain and Portugal while the second group includes New Zealand, Australia, Scotland, Finland, Norway and Canada. We refer to these two groups of countries as the “High ill-defined coding” and “Low ill-defined coding” groups. In addition to the problems of miscertification, part of the variation in reported IHD mortality rates (ICD-9 codes 410-414 or ICD-10 I20-I25) among countries may well be due to variations in the medical part of the death certificate, as well as miscoding.

All that said, I like saturated fats - so I'm not necessarily subscribing to the theory that saturated fats kill you either.

 

[Sean]

@HipNot even any agreed definition of what stress is (whatever it is, there's no guarantee its always lower in tight-knit communities. There can be a lot of pressure there).

Small and/or tight knit communities can be very repressive unpleasant places to live.

 

[Hip]

We have no way of measuring degree of "stress" in this community vs. others. No way of refuting the explanation.

Agreed that stress may be tricky to measure and define. Though I should think that social isolation is easier to quantify, and so the link between social isolation and ill-health is more testable and refutable.

But again, social isolation itself may not be the cause of the ill health; rather, a third factor may cause both.

I noticed that as my ME/CFS-triggering virus spread to family and friends, it led to significant personality changes in some of the people who caught it, and of particular note was a tendency to more social withdrawal in some people.

So here is an example where a chronic infection may cause social isolation via the personality changes it induces, and that same infection can also cause physical ill health. Then researchers may assume that the physical ill health was caused by social isolation, when in fact a virus may be responsible for both.

 

[Hip]

I have read somewhere that the so-called French paradox actually arises from sloppiness/differences in the attribution of causes of death.

Yes I read that too.

 

[duncan]

So here is an example where a chronic infection may cause social isolation via the personality changes it induces, and that same infection can also cause physical ill health. Then researchers may assume that the physical ill health was caused by social isolation, when in fact a virus may be responsible for both.

By this logic, sleep may cause social isolation via the personality changes it induces. For me, ditto for episodes of West Wing - when they air, I hole up in a room and cannot be bothered with the outside world. Actually, this holds true for anything written by Aaron Sorkin -- who'd have imagined he'd cause this sort of sickness behavior?