NIH intramural research program update

[aimossy]

I really do think there are some fantastically skilled people in our community who can help with patient input with respect to the science and study design that they may take on board. Nath is saying he wants the input. How that's going to roll is unclear yet but he said efforts are underway. It might be good to know what mechanism or form this will take so the community knows how to be part of that in some sort of organised way.

 

[Sasha]

They may be able to see what changed in these people longitudinally re recovery? This could also be useful especially of your taking a real in depth look at them? I mean this comes down to something with respect to study design and why these questions are good questions and useful to bring up.

Not in such a small sample, I would have thought.

 

[Aurator]

To echo what Jonathan Edwards and others have said, simple common sense suggests that even if the patients are chosen with meticulous and totally objective consideration for their parity, there's still a chance of choosing people with heterogeneous conditions. And if you strike lucky and all the patients do happen to have the same thing wrong with them in the first place, with 40 patients it's going to be absurdly easy to overestimate any finding's significance or insignificance. The most likely, and probably the only safe, conclusion will be that the conclusion is unreliable.

 

[Denise]

No matter which specialists recommend patients for this study, according to the "accidental protocol" the patients may have to travel to NIH at least twice. (The interval between initial testing and the in-patient stay was not specified.)
If so, that is a heavy burden on patients and I think it likely to select people who live closer to NIH rather than selecting patients from all over.
Additionally I would like to know who pays the travel expenses. (And if the patient needs a helper in order to travel, who pays those travel/lodging/food expenses?)

[edit to add - I realize that energy, distance and expense are a factor for most patients/for most studies.]

 

[greeneagledown]

The problem is, as Scarecrow, says, that it is very easy to miss something important with a 40 patient cohort - just for statistical reasons, considering that it is very likely that the important thing may only apply to 30% or so of your cases (maybe even only 10%) with a heterogeneous pathophysiology.

I would use a 40 patient cohort for a look see study costing $100,000 targeting one particular biomarker - where you have no Bonferoni problems, which makes it easier to avoid type 2 errors. For a high cost study looking at lots of variables I find it hard to see how you are going to get reliable stats out of 40. And the key thing is that you may well miss the crucial markers, so going on to study what showed up on bigger populations takes you further and further down a blind alley.

Does this mean you also think Phase 1 of the Big Data Study at OMI (run by Ronald Davis) is poorly designed? I believe that study is only looking at something like 20 patients. Here are details if you're not familiar: http://www.openmedicinefoundation.o...ely-Ill-Big-Data-Study-revised-12-11-2015.pdf.

The reason everyone is doing studies on smaller sample sizes is that no government agency is willing (yet) to put significant resources into ME/CFS, and large private donations have been scarce. Davis said he'd prefer to run his study on hundreds of patients but that there was no way he would be able to get the funding to do that.

 

[BurnA]

The problem is, as Scarecrow, says, that it is very easy to miss something important with a 40 patient cohort - just for statistical reasons, considering that it is very likely that the important thing may only apply to 30% or so of your cases (maybe even only 10%) with a heterogeneous pathophysiology.

I would use a 40 patient cohort for a look see study costing $100,000 targeting one particular biomarker - where you have no Bonferoni problems, which makes it easier to avoid type 2 errors. For a high cost study looking at lots of variables I find it hard to see how you are going to get reliable stats out of 40. And the key thing is that you may well miss the crucial markers, so going on to study what showed up on bigger populations takes you further and further down a blind alley.

Can this argument be presented or demonstrated mathematically ? I think that's the best chance of getting any change.

 

[viggster]

Additionally I would like to know who pays the travel expenses. (And if the patient needs a helper in order to travel, who pays those travel/lodging/food expenses?)

NIH pays travel expenses for patients sometimes...I'm not sure how they decide whom to reimburse. Also, a nonprofit group works with NIH to give temporary financial aid to patients staying at the Clinical Center who are undergoing financial hardship. Local hotels often provide discounts for family of NIH patients and there is a lodge on campus available to familes of patients (free of charge but limited capacity).

 

[Jonathan Edwards]

Are you referring to the CDC 5-site study now in its fourth or fifth year? NIH is working with those sites to recruit for the new study.

I'm not sure what other 200-person cohort you could be referring to.

I was referring to the cohort colected for the UK ME Biobank, which is NIH funded at least in part.

 

[Zombie_Lurker]

One of the questions I have about this study is: If they are going to study the effects of exercise on us why are they using fMRI as opposed to CPET? I thought CPET was the gold standard.

 

[Jonathan Edwards]

The reason everyone is doing studies on smaller sample sizes is that no government agency is willing (yet) to put significant resources into ME/CFS, and large private donations have been scarce. Davis said he'd prefer to run his study on hundreds of patients but that there was no way he would be able to get the funding to do that.

Yet the NIH is funding a large cohort exercise in the UK?

 

[greeneagledown]

Yet the NIH is funding a large cohort exercise in the UK?

I think a comprehensive, big data, biomarker discovery study with a big sample size (something like 100 patients and 100 controls) would be significantly more expensive than whatever the NIH is doing to help create the UK cohort. OMI is spending over $1 million on their study and that's with just twenty patients. We would probably be talking at least $5 million to fund a comprehensive biomarker discovery study with a large sample size. That's equivalent to the entire annual NIH budget for ME/CFS.

 

[Bob]

One of the questions I have about this study is: If they are going to study the effects of exercise on us why are they using fMRI as opposed to CPET? I thought CPET was the gold standard.

Hi Zombie, good question. We don't know the full details of the methodology yet, so they might be planning to use CPET, but it wasn't mentioned by Nath in his presentation. If they're not planning to use it, it might be because they are neurologists and they aren't familiar with CPET. It's another issue we could lobby them about.

 

[Sasha]

That's equivalent to the entire annual NIH budget for ME/CFS.

I'm not sure that's a big deal. The MRC flushed £5m down the pan with PACE. If we need a large biomarker study, we need a large biomarker study.

 

[greeneagledown]

I'm not sure that's a big deal. The MRC flushed £5m down the pan with PACE. If we need a large biomarker study, we need a large biomarker study.

Preaching to the choir. My point is that the reason investigators are doing research with small samples isn't because they believe small samples are sufficient -- it's because of a lack of funding/resources.

 

[greeneagledown]

I'm not sure that's a big deal. The MRC flushed £5m down the pan with PACE. If we need a large biomarker study, we need a large biomarker study.

The MRC-PACE trial was a unique case. In the US, at least, I don't think any organization (including NIH) has ever spent $5 million funding a single study.

 

[Jonathan Edwards]

I think a comprehensive, big data, biomarker discovery study with a big sample size (something like 100 patients and 100 controls) would be significantly more expensive than whatever the NIH is doing to help create the UK cohort. OMI is spending over $1 million on their study and that's with just twenty patients. We would probably be talking at least $5 million to fund a comprehensive biomarker discovery study with a large sample size. That's equivalent to the entire annual NIH budget for ME/CFS.

I think NIH have put about $2M into the UK Biobank already and hopefully there will be more - and it is only one project using the material. I rather doubt that sample size is a key cost factor in data generation. For most of these things once you have the protocol set up and running the sample number is a relatively small cost component.

Basically if any biomarker search is done at all I would think it should be done on 100 subjects if it is part of a major fishing exercise. The more 'comperhensive' the search the stronger the case for adequate sample size.

 

[Scarecrow]

One of the questions I have about this study is: If they are going to study the effects of exercise on us why are they using fMRI as opposed to CPET? I thought CPET was the gold standard.

AFAIK, CPET provides evidence that the failure to reproduce performance on the second day is not due to reduced effort but I don't think it provides specific information why. The fMRI will look for changes in brain function before and after exercise

 

[BurnA]

One of the questions I have about this study is: If they are going to study the effects of exercise on us why are they using fMRI as opposed to CPET? I thought CPET was the gold standard.

Gold standard for what ?

 

[greeneagledown]

I rather doubt that sample size is a key cost factor in data generation. For most of these things once you have the protocol set up and running the sample number is a relatively small cost component.

This is definitely not the case in OMI's fishing expedition. They're spending something like $25,000 per patient and Ron Davis explicitly said the only reason he's including just 20 patients is because a large study would be prohibitively expensive. When you're testing everything under the sun, adding patients to the sample is expensive.

 

[BurnA]

I think a comprehensive, big data, biomarker discovery study with a big sample size (something like 100 patients and 100 controls) would be significantly more expensive than whatever the NIH is doing to help create the UK cohort. OMI is spending over $1 million on their study and that's with just twenty patients. We would probably be talking at least $5 million to fund a comprehensive biomarker discovery study with a large sample size. That's equivalent to the entire annual NIH budget for ME/CFS.

From the OMF Severely ill Big Data project
Gaining the extensive data set will require $25,000 per patient for logistics and supplies.

Assuming similar costs $1m more will get us up to 80 patients and then if we ditch one of the control groups we're at 100