I just hope he's right. At this point, we don't know, but his work is a great start.
BY the way, I attended a science webinar talking about metabolomics. The attendees were in the thousands, from all around the world. And the CEO of Metabolon made a mention of how much of a breakthrough metabolomics profiling was a game changer for ME. The tide is turning.
My immediate reaction is the same as Kina's, but I would like to unpick that a bit.
When I did my first trial of rituximab in RA it was just 5 patients and the paper was turned down by the editor of our mainstream journal on those grounds. But I pointed out to him (in his office) that the probability of getting over a hundred objective data points all fitting in nice neat pharmacodynamic curves by coincidence was pretty much zero. So he agreed to publish. So maybe Kina was oversimplifying.
But that was a very unusual situation. In the vast majority of cases Kina is right. Five cases is unlikely to tell us anything. If we then look at the detail the worries immediately appear. The outcome measure is subjective. All the treated patients had a rash of the same sort so the study was unblinded. That pretty much writes off any useful interpretation.
That might seem too quick a conclusion so maybe we should consider some of the detail. The signature of good science is that the detail lines up well. We have reports here that children said their first sentence within a day or so of getting the drug. But the drug dose not cross the blood brain barrier and the bits of the brain responsible for language are pretty guaranteed to be behind the blood brain barrier. So it looks as if the parents' reports were maybe flavoured by optimism. That does not help either.
Then we are given a pharmacokinetic analysis with three data points on log plots. We are told that these data point fit 100% into the pharmacokinetic model. Trouble is that with a two compartment model any set of three data points fits 100% by definition. I may have misread here but again I am concerned.
We then have the complicated metabolomic analysis. The problem here is that however useful metabolomics might be for mechanism discovery it is the
wrong method in this situation because the data are statistically useless. For a small study like this the right thing to do is to measure 5-10 things at most having decided in advance which way you think they should change if your hypothesis is right.
I am not surprised that thousands of people go to metabolomics meetings. That is true for trendy subjects. For cutting edge subjects there is usually about twenty people. Metabolomics is great for trying to find an answer when you have no idea what you are looking for but it becomes irrelevant once you have a clear hypothesis. You know what you need to measure. You can measure some other things as controls but that is not metabolomics.
Another point we have discussed before is that animal models are pretty useless for identifying causal mechanisms. There are usually about 100 ways of inducing an animal to show signs similar to disease X. That means that for any one model there is a 99% chance that it does not reflect the cause of disease X.
One can always be wrong, and I have been wrong a few times but for me this study does not hang together.