TomK - My apologies
<Snip>
I reject the characterisation that I somehow have black-and-white thinking on scientific studies and see no value in statistics. I like mathematics and indeed was quite good at it e.g. came sixth in the Irish National Mathematics competition (for secondary school students). If my health hadn't deteriorated so much I might be doing a career involving statistics. My "crime" seems to be that I chose a book you do not think is good when all I did was base my choice on what other people wrote.
<snip>
Tom,
I want apologize. I attached your quote to my earlier post inappropriately and unfairly (your quote has now been removed). My problem was not with your considering purchasing the book. Rather, I was frustrated in the overall tone of the thread which was not of your making. As for the books that I listed, they are expensive and detailed. You don't need to purchase them to get a good feeling for the importance of their limited subject matter (power analysis in statistical analysis).
The power of a study is only one of a dozen or more different important considerations and I doubt that many of us have the energy to independently undertake a masters level course on all the methodological and statistical concerns that impact research.
Quite a lot of my mental energy is used reading ME/CFS studies and then sometimes replying to them. Not many people are doing that these days despite the fact that lots of articles can now be read online for free. If more people did that, I might well read more books on statistics as I would have more mental energy to devote to them.
I have what could be considered a very strong background in stats and research methodology (designed numerous studies, reviewed articles, years of experience actually doing the statistical analysis, probably ten or so stats courses (most at a master's or Ph.D level) and maybe a half dozen courses on research design). I've also taught sections of research design courses at a graduate level.
There is a saying that data is data. It is what you do with the data and how you characterize both the limitations, and thus what is actually revealed (if anything) by a study that counts.
Using the recent XMRV studies and the Montoya Valcyte studies as examples, there are two types of lessons to be learned from these studies.
As nearly everyone here knows, results of the recent XMRV studies in the EU differed dramatically from the WPI studie(s) - there were actually many different tests and analysis required of the WPI by Science before publication. I would be willing to concede that the EU studies were done in good faith (although there may be questions on this, it's a fight that consumes a huge amount of energy and takes focus off of what the studies actually tell us). I dont need to go into it but the problems here were more methodological than statistical. The statistical results were not close (67% versus 0%). The methodological questions revolved around issues of cohort (not a satisfactory explanation for the lack of XMRV positives in the healthy controls - Groom study excluded), testing methods - more likely (14 day culturing of cells versus roughly a day or two) and possibly, as Dr. Goff in his lecture on XMRV at CROI suggested, there may be issue with XMRV strain variation and various PCR primers not working on possibly divergent strains of XMRV or limits in the sensitivity of various assays used to detect XMRV.
Bottom line, it is extremely unlikely that stats has anything to do with the divergent XMRV studies to date (and this is as far as I would take my advice on the virology of XMRV studies - too many more details and the virology is out of my league - I suggest keeping an eye on Parvofighter's posts. Parvo is far better versed in the virology than I).
The Montoya Valcyte studies are an entirely different matter (full disclosure - I am a patient of Dr. Montoya - and I am a patient of his for very good reason). Montoya's first study of the efficacy of valcyte for CFS patients was very small but showed promise (9 of 12 patients with chronic CMV and HHV-6 responded on cognitive measures to valcyte) and the second study was slightly larger but still small (30 patients - results reported on at 2008 HHV-6 conference "indicated that patients on Valcyte experienced significant cognitive improvement" - not yet published).
The issues with the valcyte study are exactly the opposite of those with the XMRV studies - cohort and statistical power. As for cohort, not all CFS patients have chronic CMV and HHV-6 infections but they may have other chronic herpes virus (or other) infections with just as many severe effects but for these infections, valcyte is not effective.
Twelve and 30 subjects are far too few to start dividing cohorts into subsets to assess the responses of different groups of CFS patients. On top of that, much of what I personally have experienced is neurological dysfunction (autonomic and CNS with components/symptoms that remit dysfunction?) and components/symptoms that do not remit (possible neural damage?). It is known that viral replication produces neuro-toxins. A working hypothesis is that prolonged neural dysfunction due to viral toxins leads to permanent neural damage. So here you have small studies with a new agent (Valcyte) that has a measurable impact on antibody titer levels (which correlate on many measures with reported symptoms severity and degree of disability) but an end point that may or may not be entirely reversible.
On top of that, in the larger CFS population you have sub-groups with a variety of co-infections (or possibly not) that may or may not respond to each different anti-viral. Finally, CFS patients often do not respond to medications in the same manner as healthy controls. I was started on the anti-viral, acyclovir (chronic HSV-1 infection with viral encephalitis) at a ridiculously low level (100 mg / day) and then slowly increased over the course of months to a therapeutic level when most patients taking acyclovir start at 3200 mg/day. I still experienced mild side effects. My pharmacist said that it was impossible that the dose I was on was causing problems. In conversation with Dr. Montoya, 100 mg a day has caused severe problems in very sensitive patients, patients who were reduced to 40 mg a day and then very gradually increased to a therapeutic level which with significant resolution of cognitive symptoms.
The bottom line, as I said earlier, is that all of this is complicated. The sensitivity and clinical importance of your measure (anti-viral titers versus self reported symptoms), the power of you analysis based upon the magnitude of what you would deem a clinically significant effect and the ability to subdivide your entire sample and then generalize those findings to the entire population of interest (or at least carefully define to whom the results may or may not apply). All of this and more comes into play.
Rather than sitting down with a stack of textbooks, I would recommend that those who are interested may find it much more accessible and enjoyable to read (or read about) very well executed research on social phenomena impacting physiological status.
Robert Sapolosky is a scientist and author (as well as extremely bright and interesting). He is currently professor of Biological Sciences, and Professor of Neurology and Neurological Sciences, and by courtesy, Neurosurgery, at Stanford University. His early career interests included the effects of sociological factors on the development of coronary artery disease. He revolutionized thinking around the notion of the Type-A behavior pattern and heart disease. He's now focusing on the human host interactions and interdependency of chronic infectious disease (specifically toxoplasmosis) as well as stress and its impact on neuroendocrinology.
His study designs are elegant and unconventional (eg. years spent in Kenya studying the social hierarchy and dominance dynamics of baboon colonies and the relationship of these dynamics to physiological disease or well being).
Here's his link on Wikipedia:
http://en.wikipedia.org/wiki/Robert_Sapolsky
Take a look at one of his books or articles. It will probably change the way you view science.
And a few of his more memorable quotes:
"I love science, and it pains me to think that so many are terrified of the subject or feel that choosing science means you cannot also choose compassion, or the arts, or be awed by nature. Science is not meant to cure us of mystery, but to reinvent and reinvigorate it."
"Get it wrong, and we call it a cult. Get it right, and maybe, for the next few millennia, people won't have to go to work on your birthday."
