Will the IOM field test their definition?

[Andrew]

I sent the following question with comments to the IOM. I was going to wait for an answer before posting this. But the automated response said they do not reply to every email. So I'm posting now.

I was just reading the statement of work for the IOM contract regarding ME/CFS. I see no requirement that any definition you come up with be validated with real world testing. For example, the International Consensus Criteria (ICC) is currently being tested against real patients by a qualified researcher. This is to see if this criteria can actually differentiate an ME/CFS patient from a similar illness. And as this process continues, any necessary adjustments can be made before putting it into use. This is why most experts are recommending the older consensus called the Canadian Consensus Criteria (CCC), and are holding off on using their newer ICC.

Does your contract cover such validation? If not, will your recommendations include that a new consensus (if you develop one) should not be put into use unless is put through multi-site field testing and found to perform better than the CCC/ICC.

Keep in mind that an illness is defined by the manifestations of the illness in the patient population, not be the illness name. Many of the sick people who were first assigned the unfortunate term "Chronic Fatigue Syndrome" are still alive and still suffering. The only criteria that comes close to differentiating this group are the CCC/ICC. Everything else is a dilution that was driven, in part, by the illness name. Unless your definition can differentiate these people or equivalent people in real world testing, it is less qualified for use than the CCC.

 

[justinreilly]

Great point, Andrew.

 

[alex3619]

The contract cannot, I think, include any field testing. There is neither enough money nor time.

There is also the problem about how to test - what benchmark are they going to test against?

Field testing will require a further contract. Most likely they will just release it without testing. Theyz be smartz, dun need no testing here. Yet the one lesson that is central to Evidence Based Medicine is that only high quality testing in large cohorts with well developed studies and good methodology count. The rest is not quality evidence based, or class 1 evidence as described in EBM. Its definitely not Gold Standard quality.

The concept of Gold Standard is based on the notion that there is a best practice standard to compare to. The problem in ME and CFS is that we have no such accepted standard. We might have if we could get the CCC accepted. Then everything can be compared to that, including the ICC.

The closest we come to a Gold Standard, which is more like a Bronze Standard, is the Fukuda Definition. Its the most widely used for the longest time, but it doesn't stand up well to close scrutiny when compared to the evidence.

Worse, we have the Lead Standard, malleable, dull, poisonous: the Oxford Definition.

 

[Andrew]

I have a follow up letter in mind. The thing is, because patients are excluded from the decision making, I have to sit here on the outside trying to lob thoughts over the wall surrounding their ivory tower. Even if I testified directly to them as part of stakeholder feedback, it is not the same as being part of the decision making process. As part of the decision making process, one can attempt to inform on any step of the process. But in testimony, all one can do is make general suggestions from the audience.

The fact is, I think what I brought up is unarguable. If they have any scientific and humanitarian integrity at all, there is no way they can recommend that an untested criteria be let loose on sick people. And IMO, ethics require them to prove what they come up with is better than CCC. The CCC is the one that has been consensually validated by American experts based on direct experience. And the instructions state that this is to be developed with American use taking priority.

OTOH, I'm not counting on ethics here. The HHS contract ties their hands. And my big fear is this will drive us back into Fukudaland, because that's where most of the literature comes from.

@alex3619 You might want to send them your thoughts too based on your way of saying this. I think two emails are better than one, and it would be good to have this light shined clearly into their faces.

 

[alex3619]

I will think about this Andrew. Ethical issues are very important in medicine, and one possible avenue we have not explored is to raise it with the American Medical Association. Letters regarding ethics however should be broadly targeted. Don't just swat the big fly, swat all the flies.

Ethical issues are planned to be part of my book, but I have yet to do much research into this.

 

[alex3619]

Question: who is worth target on ethical issues? Has anyone investigated this?

Question: how do we want to present the ethical violations? How do we structure the argument? What should be considered?

If anyone has a background in ethics on this site then you might consider looking into these issues.

 

[Andrew]

I'm trying to appeal to their sense of ethics and responsibility and put some perspective on the relative values of criteria. I don't think any formal review body gives a hoot about this kind of thing. Just look at our history.

 

[alex3619]

I'm trying to appeal to their sense of ethics and responsibility and put some perspective on the relative values of criteria. I don't think any formal review body gives a hoot about this kind of thing. Just look at our history.

A review committee often doesn't need ethics approval. For some reason its unethical to do all sorts of things in clinical studies, but its entirely ethical to foist unproven studies or reviews on the public, doing who knows how much damage. The standard counter-argument is often that these are only guidelines, not rules. However when government and health care insurers start using such guidelines they might not treat them as provisional or suggestive. Such reviews can become prescriptive in too many cases. Just look at the DSM and how insurers treat those diagnostics.

My point about who to target is based on this principle: most of the people in a panel will probably be doctors, and so are covered by medical ethical practices. Every country has their own watchdog on medical ethics, and there are often organizations that have a view on medical ethics. They can all be contacted and made aware of these issues, but for the US for example, who are they? The AMA obviously. Who else?

What range of ethical considerations do we want to consider? I think this is worth exploring a little more than we have. We have made a start, but this is a whole new angle to pursue.

 

[Andrew]

The AMA is a professional organization to support and lobby for doctors. I think ethical violations are taken care of at the State level.

 

[alex3619]

The AMA is a professional organization to support and lobby for doctors. I think ethical violations are taken care of at the State level.

That is different to Australia and the UK then, though in the UK the disciplinary body is the GMC. If this is state based, then every state needs to be advised of possible ethical violations.