1. Patients launch $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!

Trainwrecks

Blog entry posted by anciendaze, Jan 22, 2014.

While most people at some time use the term "trainwreck" to describe a situation with a particularly messy outcome, few have actually seen a real trainwreck. I have come quite close. I didn't see the actual impact, but I passed through one crossing minutes before it was blocked by wreckage, and passed through the crossing where the crash began immediately after the wreck. The only evidence of the cause where it began was the severed engine compartment of an automobile lying beside the track some distance from the road. The other wreckage was just visible over half a mile away because it was starting to burn. The cause was a grade crossing (level crossing) without an automatic gate, automobile driver inattention and a train doing 60 mph. That crossing now has a gate, but another wreck took place at another ungated crossing only 100 yards or so away last year.

This could be a metaphor for the problem of finding "the cause" of ME/CFS. There are many potential causes, but the evidence always ends up some distance away from the event that precipitates the problem. Insisting that investigation be confined to the vicinity of the wreckage makes it very hard to deduce the entire sequence of events. The six-month delay built into diagnostic criteria guarantees there will be confusion over causation. This is analogous to insisting all investigation of the trainwreck above take place at the crossing where the vast bulk of the wreckage ended up. That instance also demonstrates the problem of dealing with a cause that is not there: the automatic gate. We could even count the inattention of the driver as a semirandom occurrence common in cases that do not always result in trainwrecks. Treating that investigation like typical medical research would guarantee no action would be taken to prevent future disasters.

(This is not to say that medicine is unique in this respect. An aeronautical engineer I knew well told me "any crash investigation is bad, but when lives are lost it becomes pure hell." It takes a special kind of investigator to persist in this endeavor until the lessons learned are actually applied to save future lives. Far too many people will be primarily concerned with covering their ass.)

While it should be possible, in principle, to use Newtonian mechanics to deduce the sequence of events after the first wheels leave the rails this is virtually useless in practice. Tiny variations in the environment we would have trouble measuring, even if we knew to measure them prior to a wreck, plus variations in timing nobody actually measured, make the precise consequences of an initial impact hard to follow more than a few steps from the initiating event. You always have questions about causation: did this spike come loose from the rail before the wreck, or was it wrenched loose as a consequence? Even so, this is a great deal easier than deducing the sequence of events in the crash of an aircraft, where many more possibilities are available for a malfunctioning machine. Both are much simpler to deal with than human physiology, because we now have theories of what makes trains and airplanes work when they are functioning as desired. Physiology, by contrast, is largely descriptive, and that description mostly ignores those variations from ideal homeostasis which characterize health. We have a word, homeostasis, where a complex theory would be required for useful prediction. Even those theories which already exist are commonly neglected.

At the far end of the sequence we have the problem of classification. It is as if we argued about trainwrecks in terms of exactly which cars ended up on top of the engines, or thrown far from the tracks, treating different cases as different diseases. (We might even classify trainwrecks with chemical tank cars as entirely different from those involving ordinary box cars. The effects are different, but the etiology is the same.) In the analogy with medicine, this completely ignores the long period after some initiating event when it was literally impossible, given current medical technology, to distinguish failures in homeostasis as different as cancer, cardiovascular disease or multiple sclerosis from their poor relation CFS. All involve broken homeostasis, but we don't understand most of what goes on normally, so we ignore it. This ignorance has serious implications for more respectable diseases.

You might be forgiven for thinking that improving detection of diseases, so that they could be prevented by medical interventions, would cause a revolution in health. If you had a magic test telling you that this patient would develop cancer in another year wouldn't that mean you could avoid the whole thing? Unfortunately, existing medical interventions are also likely to be life-threatening all by themselves. Without some idea of how an individual patient will respond to chemotherapy, or what surgery is necessary to remove particular cancerous cells, any doctor attempting to treat a cancer before it becomes life-threatening would be risking a malpractice suit.

Think this is unique to cancer? Suppose, like people I have known, you have symptoms and signs of incipient multiple sclerosis (MS). This can include paresthesias carefully mapped by neurologists, disturbances in gait, optic neuritis, etc. Plenty of evidence here for diagnosis of something, but it has not crossed the line into active MS by producing conspicuous lesions visible via MRI. Now, with this evidence, and even such things as autoantibodies causing Hashimoto's thyroiditis, surely we can expect early intervention to head off the development of the full disease. Ask an honest specialist, and she/he will tell you that with a mild case of MS available medical interventions would all do more harm than good. What's wrong with this picture?

Think I've picked two atypical examples? Consider a current pinnacle in the hierarchy of medical specializations, cardiology. In three cases, where I've been close enough to people to be told details of tests on them done after a heart attack, all had evidence of damage from previous, undiagnosed heart attacks. Nothing I've read makes me think my experience in this regard is unusual. Diagnostic tests run prior to a heart attack are more likely to be reporting prior damage than to be predicting damage before it takes place, when it could be prevented. If the most effective intervention is a coronary bypass operation, which carries a substantial risk of death, there is no point in detecting any malfunction which is less than life threatening. This is all done in terms of what's best for the patient, overlooking the financial benefits for doctors of rushing in at the last minute and shouting "your money or your life!"

Truth to tell, we don't have many medical interventions in which benefits always outweigh harms. This could be a matter of historical contingency, starting from medical practice on the battlefield, but that fails to explain why this situation persists.

In the literal trainwreck described above, the first impact didn't completely derail the train. Part of the wreckage of the automobile wedged under the front wheels of the engine. All the remaining wheels stayed on the track until the engine struck the platform of a disused station near that second crossing. This second impact was what turned the engine sideways and caused the rest of the train, which was still moving rather fast, to pile up on top of it. Momentum from that crash pushed the the mass of wreckage another 100 feet. The train had traveled 1/2 mile after the first impact. To come to rest without further damage it would have had to travel close to 1 mile. Had the train avoided hitting the platform, and come to a stop, it would have only needed to back up to free the wreckage of the automobile. The impact with the platform was the proximate cause of the huge pile up. In normal operation that platform had stayed there next to the track for years without causing any problem.

In a medical context something like that platform would likely be disregarded as a cause, simply because it had not previously caused trouble. ("Evidence-based railroading"?) Also, a trip to the doctor between the time of the initial "hit" and the second impact would probably not reveal a great deal wrong. "Yes, you had an infection, and there is some persistent inflammation, but the problem will resolve naturally if you just wait."

I've come to believe many medical problems where etiology remains unknown are the result of multiple "hits". This is especially likely in chronic diseases. My first rule of thumb for a trainwreck, is that the common assumption of a single cause is suspect. Even if such a cause can be found, it may be so distant as to render a connection with the final result impossible to trace.

There is also a departure from normal homeostasis prior to recognition of the clinical disease. Unfortunately, we tend to assume homeostasis is working right up to the point where we recognize a clinical disease. Almost nobody measures how well it is working.

(You might assume a railroad does not exhibit homeostasis, but you would be wrong. Those wheels, trucks, springs and couplings are all designed to tolerate small deviations from a perfect roadbed and a balanced load without coming off the tracks. In most cases when a train strikes an object, even one as large as an automobile, the train is not derailed. You could say that train just had bad luck, though not as bad as the automobile and driver.)

One common assumption in calculating probabilities of rare events is that these are completely independent of each other. In a case like the above, the low probability of striking that platform would cause it to be discounted. In fact the probability of striking the platform turned out to be much higher after striking the automobile. This is an example of a conditional probability. Dealing with these is tricky, but the algorithms that predict what letter you are about to type into a search engine do a very good job using such conditional probabilities. Medical personnel don't do nearly as well. Probability is important because estimates of probability implicitly guide diagnosis, and diagnosis guides treatment.

There is a second trap for the unwary in combining probabilities from multiple components in a system. This is normally applied to systems engineering of machines, but medicine could learn from the insight gained through this experience.

The classic example I first learned about was the estimated reliability of the V1 "buzz bomb". Preliminary estimates were based on a careful analysis of each component the missile needed to function as intended. Engineers then took the probability of failure of each component and combined these to get an estimate of the system failure rate. This said that 70% would fail, but 30% would hit a target. This was deemed acceptable, and production went ahead. Experience then showed that only 3% hit a target (even one as large as London), a 97% failure rate. What we now know to be correct would be to multiply the probability of correct operation for each component to get the probability the system would perform as intended. Statistics concerning survival in systems without redundancy behave this way, and the proper distributions for them are not the "normal" (Gaussian) distributions we find convenient for typical parametric statistics. (Levy distributions are more likely, and I could say a great deal more about their properties.) Systems with redundancy can survive longer, but efficiency drops as they accumulate faults. This also has a medical parallel.

When human beings are considered as a system of organs, as medical specialists already tend to do, you need to combine estimates that heart, lungs, kidneys, liver, etc. will function correctly by multiplying, not adding. Problems that are not exclusively confined to a single organ will have a much larger impact on the outcome than commonly assumed. Your heart, lungs, liver and kidneys may each be working at 90% efficiency, but the combination will be only 66% of what you could expect if healthy. If you are going to treat people as having many parts you need to know how this kind of reasoning works. What happens instead in medical practice is that each organ is assessed relative to some threshold, if all pass then the patient is presumed to be healthy. Patients who insist they are not must be malingerers, the doctor thinks.

In the specific case of long-term ME/CFS patients, who often have both diastolic dysfunction and hypovolemia, they may be missing 20% of their red blood cells even though the concentration of RBCs is normal, because of hypovolemia due to a disturbed HPA axis reducing antidiuretic hormone (ADH or vasopressin). This is equivalent to loss of 20% of hemoglobin, but will not show up as anemia. It will reduce oxygen transport to 80% normal. This assumes transit time for RBCs is unaffected, but if diastolic dysfunction reduces cardiac output, transit time will increase. This could be as bad as 60% of healthy normal without triggering any alarms. The two together combine to reduce oxygen supply to cells by 0.80*0.60 = 0.48, less than half! This will not affect most readings of oxygen saturation, because oxygen is getting into the blood just fine, but the rate at which it reaches cells is way down. This is one way patients can end up going into anaerobic metabolism when they pass a low threshold during cardiopulmonary exercise testing. This is probably not the original pathology, it is more likely a consequence of accumulated damage. We are seeing a slow-motion trainwreck in progress.

So now we have two ways things combine multiplicatively, which doctors and medical statistics tend to overlook. This is not the end of the possibilities. Let me illustrate with yet another engineering example taken from operations research during WWII.

Another statistical mistake from WWII comes from mapping bullet holes in returning bombers. Bullet holes certainly combine additively, but the probability of surviving them combines multiplicatively. If you have a 90% chance of surviving a single hit the probability of surviving two is 81%, three is 72.9%, four is 65.61%, and so on. This introduces a powerful selection effect into the statistics. Because they were only counting holes in bombers that returned, the areas with the fewest holes were actually the places that needed armor.

In medical research on diseases of unknown etiology, we often don't know what "bullets" survivors have dodged or endured. If type I diabetes is caused by an infection, this might mean that those developing the clinical disease were survivors, and the data on those who died in childhood is missing because we have not connected the diseases responsible. This is like examining the bombers which returned from combat, and comparing them to aircraft which had never been in combat. You might then suspect that those aircraft with mysterious holes had genetic defects. Statistically, the additive nature of bullet holes is less important than the selection effect caused by only examining surviving aircraft.

If we were able to connect ME/CFS with a more serious, even lethal disease, like progressive MS or idiopathic myocarditis, it would change the whole interpretation of medical statistics related to this illness. Yes, we are in bad shape, but we are in better shape than those who have died. You can make the same errors in comparing those with post-polio syndrome and healthy controls. Some of the differences between us and the healthy populations used as controls may not be defects. Some healthy individuals are simply lucky they were not exposed to particular diseases.

This argument is not simply special pleading on behalf of one disenfranchised patient group. It applies to a wide range of diseases currently with unexplained etiology. Before you rule out a connection between diseases of unknown etiology you should have very good reasons for doing so, not simply convenience or economy. (There is nothing convenient or economical about long-term disability, for either the individual or society.) Many diseases which have resisted explanation for 40 or 50 years of targeted research may very well be future textbook examples of the kind of errors in reasoning I have been discussing.

When hypotheses of causation are all over the place, you have to consider that you are dealing with a trainwreck instead of a nice, neat, convenient disease.

"It's a psychological problem."
"It's an infectious disease of the CNS."
"It's an immune disfunction."
"It's dysautonomia."
"It's an autoimmune disorder."
"It's caused by an enterovirus."
"It's a mitochondrial disorder."
"It's caused by bacteria."
"It's a cardiovascular problem."
"It's caused by a parasite."
"It's a genetic disease."

If you examine the history of medicine you will see just how many ways it was possible to misinterpret evidence of causation in the cases of two chronic diseases: tuberculosis and syphilis. I would classify both as pathological trainwrecks. You might have expected doctors to do better when HIV came along because of earlier experience. You would be wrong. That dispute was resolved using knowledge and techniques that were simply unavailable earlier in history. (It would have been hard to understand retroviruses at all prior to 1970, when reverse transcription was discovered.) Without this there is no telling how long debate would have continued.
anciendaze

About the Author

As the name suggests, I am old and dazed. The avatar illustrates my rule of thumb: "Hang on! This ride isn't over."
  1. anciendaze
    Before you write things off as hopelessly complex, you might test a number of models to see how well they do. I'm opening the range of models for stable statistical distributions, and these can show surprisingly good fits to biological data, including data from humans. There are sophisticated techniques for reconstructing dynamics from data which are generally not being applied because everyone seems to be convinced there are too many interacting factors to allow scientific examination. This allows them to say, we can't possibly understand what's going on, so we have to keep doing what we're used to doing. Putting the solution of ME/CFS somewhere in the future after people understand the deep problems in psychology is simply another way of saying "we aren't going to do anything".

    All scientific theories can be viewed as analogies rather than "the Truth". (If you think physiology and medicine is unique in this respect, you don't understand the problem of interactions between electrons and the empty space around them in QED.) What people have learned in the study of systems that are complex in the sense of having many parts is that some behaviors are generic, not depending on all the details of components. There are techniques for identifying these.

    I can recall physicists who used these to discover that the weird eye movements classified as "schizophrenic gaze" had a low-dimensional attractor. This meant there was a simple explanation. When they tried to present this to medical doctors they were told, by one person after another, that this was simply impossible -- ignoring the data they had. The implication of the finding was that one aspect of the illness was a simple neurological defect involving a small number of neurons. This finding never got a hearing. Is it any wonder schizophrenia is just as intractable a problem today as it was then?
  2. alex3619
    The reason why I call the use of probabilistic multiplication analogous is that biological systems have complex interplay and feedback loops at so many levels we just don't know about all of them yet. Life is about as complex a dynamic system we have ever seen. Any such calculation is at best an approximation, and really needs to be tested. It is however useful for demonstrating the fundamental principle.

    Non-testing of convenient assumptions, or consideration of rules of thumb as having the same reliability as physical laws, is something I see a lot. The very way that many professionals, and indeed members of the public or government, see a lot of these things precludes them being examined. Much of this is fallacious thinking, but the reason I think its so prevalent is that life is too complex if you think too much. So everyone uses heuristics or even serious oversimplifications, and few ask the big questions. What deeply concerns me though is that while I can perhaps forgive an average doctor for doing this, its much less acceptable for a researcher, and its not acceptable for the profession as a whole.
  3. anciendaze
    If professionals were thinking in terms of dynamics at all, the way I was trained, they would concentrate on rates from the beginning. Just beyond that point they would recognize the importance of ratios of rates. Then it would be possible to talk about whether interactions were linear or nonlinear, etc. This would put some solid basis behind the word homeostasis.

    In practice almost nobody is even testing convenient assumptions to see if they are true. They are interested in mean heart rate, and consider variations to be random. If this were true you would get the same behavior from a time series of intervals between heart beats if you did a random permutation of the order in which they occur. You can't get any more random than completely random. Try this with any data set from a real human and the result will be visibly different from the original data, ergo the variation is not random. (You can find this example, and many more, in the works of Bruce J. West, like "Where Medicine Went Wrong".)

    I've argued repeatedly that convenient assumptions about normal distributions continue to be used even when they are clearly violated. If separate processes combine by multiplication the result is going to be like the Levy distributions I mentioned. The only reason you will get a finite variance (or standard deviation) from this are limitations on the allowed range and number of sample points. This convinces me that many of the researchers most able to extract desired results from data are actually doing unconscious data manipulation. Virtually every criterion necessary for applying the Central Limit Theorem to justify normal distributions is violated in practice, over and over again. Many data sets plot as a straight line on a log-log graph, an indication of scale-free behavior. The wonder is that few people in research seem to be aware of this pervasive problem and what it implies.
  4. anciendaze
    Alex, the combination problem is more than analogy. If you are going to take things apart in order to examine the function of components you have to have some idea of how to put the results back together. If doctors were seriously thinking about the additive combination implicit in the choice of normal distributions they wouldn't make the same blunders they do in thinking about thresholds for pathology. What they are doing is pattern matching, and if it isn't a pattern they find convenient, then it is "somebody else's problem".

    Incidentally, when people disparage "reductionist science" they typically forget that Newton had to invent calculus to put the idealized components back together. Simply taking things apart without putting them back together is like much of my childhood exploration of mechanisms.
  5. alex3619
    The combination analogy is one I have been using for many years. Organs have to be considered as a system.

    I have said many times that we often find something, presume its important, proclaim it as the cause, and then ... silence. So far most of the causes have turned out to be pathophysiology, only piece in the puzzle, or wrong, or only partially right. At some point one or more will be proven right though, its just hard to more than guess which "cause" will really be the cause/s. I started talking about multiple hit ideas about nine years ago. The idea has rarely been greeted well.

    The issue with low blood volume has been discussed before, and is indeed a big issue. Doctors just don't consider alternatives, they use rapid recognition and minimal testing to draw conclusions, they are usually (and there are exceptions) not problem solvers.

    Departure from homeostasis can be described under either chaos theory or catastrophe theory.

    There is a pronounced dearth of systemic analysis in modern science. Most who do that are in business, computer science or engineering. The standard scientific approach is either reductionist or statistical. Nearly all of psychology is statistical.
  6. Little Bluestem
    Loved the train wreck analogy.

    I have had only basic statistic courses, but the ubiquitous assumptions of standard normal distributions is irritating even to me.
  7. Christopher
    @anciendaze - Hopefully what will happen is enough people suffering from these disorders are able to recover enough functionality somehow with the help of currently available therapies in order to spearhead the necessary change.

    Joey Tuan is an example by starting up his healclick website. Kyle Day another trying to help people with methylation. Terry Wahls with her diet. There are way too many brilliant sick people for this not to change.
  8. anciendaze
    Sorry about being "a little depressing". I haven't figured out how to avoid that when discussing the state of the art in medicine. At this point I'm not eager to take on the entire profession, which has a definite tendency toward ad hominem attacks and reasoning from authority. It is much easier to see current status quo as acceptable if it is paying your salary.

    There is an editorial in BMJ which considers "evidence-based medicine" as imperfect, but necessary as the only game in town. Once again they are fighting old battles, like the veterans of yesterday. The idea that the fundamental basis of the essential statistical tools in use is invalid because separate distributions do not combine as supposed is simply not yet in sight from that Olympian vantage point.

    I've tried to do more than simply say things are broken. I've pointed out a specific gap which people invoking the Central Limit Theorem to justify assumptions about normal distributions are missing. I've suggested a class of distributions which provide better fits. I've demonstrated some simple calculations which are simply not being done in practice. I've brought in conditional probabilities, which work very well in an application all of us here are using (search engines). I've also touched on selection effects which can override the original distribution even when it is valid.

    What I am not capable of doing is cleaning the Augean stables of medical research, especially in the face of determined opposition.
  9. Marco
    Very well written and extremely thought provoking (if a little depressing).

    Could you submit this to the Lancet/AMA?

    Funny that you should use the 'wreck' analogy. I used a similar one a while back when I suggested that 'differential diagnosis' and the research that follows was akin to arriving at an auto/car junkyard and sorting them into 'frontenders', 'rearenders', 'rollers' etc.

    You would then proceed to try to determine what caused the front end of these ones to deform or the windscreen of these others to break. The common thread that all were involved in a collision is lost in the details.

    Loved the bomber example BTW. Logical when you 'think' about it!
    Little Bluestem likes this.