Hi Kurt.. as promised...
To the issues you raised in your article (which, by the way, was a very clear and well-written statement of your opinion on the process you would like to see in XMRV research):
I have to agree with a couple other posters here that, although a formal consensus process like the one you modeled would ideally be great in the case of ME/CFS in general, it is not necessary and may indeed take too long in the case of establishing XMRV's role in pathogenesis. I do not believe in rushing science at all, of course. But multiple validation studies are not required to establish a sufficient informal consensus in the scientific community as to a given agent's role in pathogenesis (causative or otherwise) such that medical and governmental (and insurance) policies would be appropriately changed. Replication studies of an adequate number or weight are often enough to change the general consensus alone, usually without an official forum or process for consensus (such as a platform like the NIH's). In the case of XMRV detection and elucidation of pathogenic role, I do agree that some amount of validation using different techniques may be necessary to achieve a clearer picture and a broader consensus, but not much.. FAR fewer good studies (such as the one Ila Singh is working on) than 'hundreds', or even than dozens, may well be sufficient. And I should point out that the Science study had internal validation experiments (following the definition given in your article) as well, which if replicated would effectively provide validation within replication studies.
One problem with achieving consensus on the general issue of CFS is the uniquely confusing nature of poor definitions in this area.. an even bigger (and related) problem is the politics that drive much of the debate (particularly from those who argue for a psychosomatic role). The latter issue will likely make consensus far more difficult than one would objectively expect from the scientific process.
A few specific responses..
"The scientific consensus process is simply a way to describe the reaching of a consensus view by a majority of scientists for a new research finding, in essence it is the construction of a new paradigm on a given topic or sometimes an entire field. The consensus process is not a formal part of the scientific method, but often researchers, businesses and government agencies rely on a consensus opinion for policy and funding decisions. Therefore the reaching of consensus can be critical for helping a field move in new directions. Although consensus building may be a somewhat informal process, there are also formalized approaches that might present a model for a consensus process for XMRV or CFS. One of the more important consensus processes is sponsored by The US National Institute of Health. The NIH maintains a formal medical consensus committee and hosts regular conferences of experts for specific consensus development topics [7]"
But it is in precisely these government advisory consensus processes that politics come into play; the danger is that consensus hosts will be unable (or unwilling) to determine who the real experts are, and give equal voice to those who lack sufficient knowledge or understanding of the more subtle scientific issues or background.
"While research supporting the XMRV Hypothesis for CFS has not yet achieved sufficient momentum for a consensus opinion"
I would say rather that there is sufficient momentum, but insufficient data.
"In virology where testing processes can be complex, several different test designs can often be equally valid. As a result, a large number of studies may be required, to test alternate test assays and validate findings. Discovery of multiple ways to achieve the same results lowers the risk that some of the tests are producing false results (false negatives or false positives)."
This is not necessarily true.. in fact, very often a large number of studies using a variety of different assays is not required, especially with the streamlining of molecular biology techniques within specialized fields.
Your concerns about the misidentification of endogenous retroviruses that may be going on here is well noted, of course, as you have consistently presented that argument on this forum. However, I do not see why this would necessarily require a formal consensus approach in XMRV research. If researchers begin to identify the possiblity of ERV contamination, a scientific debate will automatically be established and the groups that are investigating XMRV right now (e.g. the DHHS and the CDC) will re-evaluate.
Also, I wouldn't be too keen on having the NIH itself be the forum for definitive formal consensus. Administrationally, they have a pretty poor track record when it comes to most things CFS.
As for terminology - some of the relevant experimental terminology is indeed confusing and misused by many scientists, and I think it would be nice to have some scientific consensus on
that!
"Replication Study: A study to determine the reliability of the methods used in the original discovery of a new finding. This is not generally considered a validation of the finding, but rather a verification that the procedures and methods work as claimed. This is also sometimes called a 'Verification Study'. By itself a replication study is not proof that a finding is valid, but only that an experiment can be repeated."
It does more than demonstrate the reliability of methods; it definitely lends weight to the original finding! Also, the term "validation study" is often used differently than you have defined; it can be taken to mean validation of a specific technique or protocol (more in the manner that you have defined 'replication study').
Btw, I also agree with Hope's points in a previous post regarding the consensus process.