• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of, and finding treatments for, complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia, long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

On academics not checking their own citations: No One Really Reads Academic Papers

Esther12

Senior Member
Messages
13,774
One of the things a lot of people who check the citations in CFS paper find is that the citation often does not support the point being made, or say what it's meant to. I've noticed how it seems that misleading citations can 'spread' - they're used in one paper, and then papers which cite this can include the misleading citation aswell. It seems that this problem is not something which only occurs with CFS, even if it may be especially bad here.

PS: I haven't checked the work that is cited here... but the author promises that he has!

http://www.washingtonmonthly.com/college_guide/blog/academics_do_a_lot_of.php

February 19, 2013 05:17 PM
No One Really Reads Academic Papers

by Daniel Luzer

Academics do a lot of research. The pressure to perform research in order to earn tenure generates, by some estimates, about 1.5 million new articles a year.

Some scholars have critiqued the quality of this research, pointing out that only 45 percent of the articles published in top journals are cited within the first five years after publication, but scholars are supposed to build on existing knowledge and use that to develop their own thinking.

Except it turns out that most of the cited research probably isn’t read. In fact, most of the research academics actually cite in their own papers they likely haven’t looked at.

That’s according to a paper by scholars at University of California, Los Angeles indicating that some 80 percent of authors include citations to articles they probably haven’t read. As the paper explains:
We report a method for estimating what percentage of people who cited a paper had actually read it. The method is based on a stochastic modeling of the citation process that explains empirical studies of misprint distributions in citations. Our estimate is that only about 20% of citers read the original.​
The researchers apparently estimate this based on repeated misprints. In other words, if my references repeat misprints used in a study published earlier, it’s likely I haven’t actually read the study I’m misprinting but, rather, just copied the earlier study’s references.

All of this is not to say that researchers are straight-up dishonest (nowhere in an academic paper does it actually say, after all, “I certify and affirm that I have read and verified all information presented in paper and guarantee that the information is true and correct and that any documents I/we have provided with this study are genuine and that the information contained therein is also true and accurate”) but it is a little, well, disconcerting to think that there’s a possibility that most of the researchers out there are really just cutting and pasting other people’s papers, unread and unimportant, before they introduce their own material.

Disclaimer: I read the paper in question, “Read Before You Cite!” by M. V. Simkin and V. P. Roychowdhury published in Complex Systems, 14 (2003) 269-274, but it was only five pages long, and I basically skipped the equations on pgs. 272-73.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
I keep warning people about a related issue: reading only the abstract. Abstracts are often highly misleading, and only the full paper is useful - and even then not all the time. This is a problem for me due to paywalls and limited time. I guess that researchers in general have similar problems involving time and material resources. It can also be an issue that if you have a particular bias driving your research, you might interpret the paper in light of that bias. Does it really say what you think it said? I am at risk of this myself, as is everyone.
 

Esther12

Senior Member
Messages
13,774
Gee, maybe journals should hire, I dunno, editors, fact-checkers, and reviewers?

Most peer review is pretty shoddy imo. Personally I think that it can be harmful by creating a false sheen of respectability.

Also though - it does tend to be done for free. Going through and checking all of the citations in a paper is a massive piece of work, and not one that many qualified people would be willing to do in their spare time. If a claim fits with the prejudices and assumptions of reader, it will tend to go unchecked.
 

Dolphin

Senior Member
Messages
17,567
Also though - it does tend to be done for free. Going through and checking all of the citations in a paper is a massive piece of work, and not one that many qualified people would be willing to do in their spare time. If a claim fits with the prejudices and assumptions of reader, it will tend to go unchecked.
Probably all generally true.

Although personally, when reading a paper I usually find it interesting for revision purposes to check the references. And then, sometimes look up papers where I am not aware of what was referenced. I might not check every single reference of course.