Discovery Forum 2017: Dr. Nancy Klimas

[used_to_race]

I wonder if she would have better, faster, or more insightful results of she uses A.I. (artificial intelligence)/Deep Learning and AI-chips (or GPUs) instead of super-computing. It's the latest stuff coming out of Silicon Valley and is supposedly more efficient in pattern recognition and optimal solution seeking. I'm far from an expert, but perhaps someone on here knows more about that field and could provide some insight.

I work in Machine Learning/AI and have a couple recent publications, but am far from an expert. The issue is that you need a lot of data. For medical stuff the data requirements actually seem to be a bit less than what I'm used to because our bodies aren't changing at high frequency I guess. You really don't need "supercomputers" to do the kind of stuff they mention, and I have no idea why Klimas puts a picture of some folks standing in a server room in her presentation for "0.3TB of data". I'm not even sure what she means when she says the 7000 CPUs "generate" all this data. It's kind of laughable to be honest. A modern laptop with a GPU and lots of RAM will do okay with some deep learning tasks. It's nice to have maybe a single server with some GPUs but this would be far from a supercomputer. The machine I use at work has 8 high end GPUs, about 700GB of RAM, and (I think) 56 logical CPU cores, and 7 or 8 engineers and data scientists are able to use it for simultaneous tasks, but we just bought a second.

Ultimately it goes back to having a lot of data and being confident that it's labeled correctly. The software makes everything else pretty easy. I see how Stanford is generating large datasets, but nobody is looking at using ML yet as far as I know.

It's funny they talk about biomarkers in these complex diseases, but you could train a classifier to identify other autoimmune diseases like Crohn's, UC, RA, and others maybe 90-95% of the time without a powerful machine or even a deep learning approach. Here's a paper where they did that.

 

[Gemini]

I started talking about homeostasis and reset decades ago...
This is about chaos theory to a large extent. I was discussing this in the 90s. The early models I was working with were about homeostasis but based on hormones and hypoxia.

I remember @alex3619! You were way ahead of the times!

Klimas' "integrative multi-systems model" deals with the HPA, HPG (hormones) and immune system.

For IT experts amongst us details of her model are in this free paper: @used_to_race

A Role for Homeostatic Drive in the Perpetuation of Complex Chronic Illness: Gulf War Illness and Chronic Fatigue Syndrome, Plos, 2014

https://www.ncbi.nlm.nih.gov/pubmed/24416298

 

[junkcrap50]

I work in Machine Learning/AI and have a couple recent publications, but am far from an expert....

Thanks so much for replying. That was very interesting. Maybe she can raise more money if she uses terms like "supercomputer" and stuff like that. My impression is that she runs tons of simulations, basically trial and erroring changing 1 thing at a time, which prompted the super computer. Regardless, interesting.

Do you think that there's any role for machine learning/AI in analyzing CFS data? Would the metabolomics used by Naviaux data be big enough? What if they do a mega study and use every single testing and analysis each CFS researcher uses in their narrow studies, 100 patients, and Klimas's 9 serum draw exercise test - literally every test ever done on CFS patients and combine them into one huge data set?

Thanks.

 

[used_to_race]

Thanks so much for replying. That was very interesting. Maybe she can raise more money if she uses terms like "supercomputer" and stuff like that. My impression is that she runs tons of simulations, basically trial and erroring changing 1 thing at a time, which prompted the super computer. Regardless, interesting.

Yeah I think that there's an element of woo in tech when you are speaking with a nontechnical audience. Everyone does it but these days you will get called out if you're spouting crap like this and that's counterproductive. Even people in fields outside computer science have a grasp of machine learning and I fear it harms the credibility of ME/CFS research when folks talk like this.

Do you think that there's any role for machine learning/AI in analyzing CFS data? Would the metabolomics used by Naviaux data be big enough? What if they do a mega study and use every single testing and analysis each CFS researcher uses in their narrow studies, 100 patients, and Klimas's 9 serum draw exercise test - literally every test ever done on CFS patients and combine them into one huge data set?

Yes, I do think so. I linked the study above, but there are many other such studies as well. ML can be used in various ways for medical research:

1. Diagnostically: you use serological, genomic/multi-omic, and clinical data as inputs to a classifier and train it using patients and controls. The classifier can then serve as a diagnostic tool. The other way this is effective is when applied to imaging. Here's a paper where some Stanford electrical engineering students (no medical training at all but with the help of people in the med school) trained a neural net to identify and classify skin cancer with better performance than a couple of board-certified dermatologists. They are actually using a pretrained model and fine-tuning it, so that lessens the need for large datasets. This could be applied to images of cells in ME/CFS vs. controls, for example. The issue (and this is a central issue in AI as a field) is that how the classifier makes these classifications is largely inscrutable. So if you talk to a doctor she might say "well this image shows a skin lesion that has parameters x, y, and z, and given the patient's family history and history of sun exposure, I think it's likely to be a melanoma of type A or C." The doc might be 80% accurate at doing this. The classifier could be 95% accurate, but you will get essentially no information about how the decision is made.
   
2. To guide treatment and predict drug response: if you have lots of clinical data on patient responses to certain drugs, you can train a model with the patients' clinical data and their responder/non-responder status. For example, this has been done to predict response to anti-TNF therapy in some autoimmune diseases. With ME/CFS we do not have this clinical data, because clinical trials of drugs mostly haven't happened and the information out there on antivirals, IVIg, and whatever else is very informal. However with the rituximab trials and cycloME we might be able to apply this if they collected enough data on the patients (which I would think they did). And for future clinical trials this will be possible. This is a nice type of project because you don't need to do deep learning to get good accuracy. It's exactly the type of classifier you could put together, train, and document in a couple of days.
   
3. To identify subgroups: you might be able to do some 'clustering' (which in CS is just a type of unsupervised machine learning) on the data to get a largely unbiased estimate of how your cohort is organized into subgroups. Lately I have been doing a lot of this at work, and there are lots of algorithms out there. Some are better than others, but the issue is taking data with many dimensions (i.e. metabolomic data or cytokine panels) and "de-noising" it. Some of the stuff measured is going to be pretty low importance, so you have to reduce the dimensionality in a way that preserves the useful stuff. This is historically a hard problem.

 

[alex3619]

Would the metabolomics used by Naviaux data be big enough?

Maybe too big. AI methods often struggle with gigantic datasets ... unless they are used on a supercomputer. There is a place for AI methods, and I would bet that many are used in bioinformatics. For a long time I wanted to combine my degrees in AI and biochemistry and get into bioinformatics, but eventually I realised I was not even close to well enough.

 

[alex3619]

The issue (and this is a central issue in AI as a field) is that how the classifier makes these classifications is largely inscrutable

When I was looking into this in the 90s there were attempts made to translate neural networks into rules. I am not sure they were successful. Other classifier systems like genetic algorithms are intrinsically easy to convert to rules.

 

[used_to_race]

When I was looking into this in the 90s there were attempts made to translate neural networks into rules. I am not sure they were successful. Other classifier systems like genetic algorithms are intrinsically easy to convert to rules.

The thing is that people usually use very raw data as inputs to neural networks, like pixels in an image or raw I/Q of a signal or something of that nature. So not every feature has the same importance each time. And indeed with recurrent and convolutional neural nets this is kind of the point. It's a really tricky problem. You can do feature importance analysis way more easily on shallow learning models, which is why the Random Forest classifier is so popular these days. I'm honestly not familiar with genetic algorithms and would like to learn more about them.

A lot has changed since the 90s. There's even automated machine learning now, where a separate process tries a bunch of pipelines and parameters and selects the best one for the data and outcome measures. TPOT is the variant I've used.

 

[alex3619]

So not every feature has the same importance each time.

Finding the appropriate input data structure is one of the trickiest problems in neural nets. My PhD, which I had to abandon as my ME worsened and destroyed my capacity for mathematics, and even reading, was about composite neural networks. Really about structures of networks which could be combined (then retrained) on a reusable basis, much like software modules.

 

[junkcrap50]

Thanks @alex3619 and @used_to_race, I appreciate your insights very much.

I thought that what Klimas is doing, using the in-silico simulations to come up with a model and then figure out a treatment path was real impressive. And it seems like the complexity of CFS and the mass amounts of data would necessitate more in-silico work. Hopefully, the latest tech can be used for it. Thanks again.

 

[Belbyr]

I just watched some newer videos of her, she was saying she is ready to start trials now (Jan. of this year) and mentioned she doesn't need a mouse model like she did in GWS.

 

[wigglethemouse]

I just watched some newer videos of her

Can you post the links please

 

[junkcrap50]

I just watched some newer videos of her, she was saying she is ready to start trials now (Jan. of this year) and mentioned she doesn't need a mouse model like she did in GWS.

Yes. Please post the links. I searched youtube to see if there more recent videos of her and couldn't find any. And you're talking about she being ready to start CFS/ME trials, right? Does she mention what her approach for the study would be? What supplements or drugs or what they're acting on?

 

[wigglethemouse]

This is another link from the panel discussion following the screening of Unrest at the first ME/CFS Canadian Collaborative Team Conference in Montreal on May 3, 2018

Nancy Klimas said she will be starting ME clinical trial in women in Oct, men hopefully by end of year. Unfortunately I didn't note the time stamp where she said this.

 

[alex3619]

The trial related dates are at about eight minutes.

 

[alex3619]

On computational issues with ME data, Ron Davis has like a billion data points per patient, if I am not misremembering, and that is growing. Analysis of this stuff is computationally heavy, particularly if you are running many different analyses, and comparisons or machine learning using the entire data set. What is different is that they have tracked a great many different chemicals, using different tests, and done this many times. Most of what I have seen Ron present is pathway analysis, showing how biochemical process are activated or inhibited. Of course with healthy controls added in this dataset is even bigger. I have no idea how big it all is now, but it will keep growing because they keep testing.

 

[Mary]

@Diwi9 - thanks so much for starting this thread! I just came across it tonight and watched the video you linked - it's amazing and Nancy Klimas is amazing!

 

[Gemini]

On computational issues with ME data, Ron Davis has like a billion data points per patient, if I am not misremembering, and that is growing. Analysis of this stuff is computationally heavy, particularly if you are running many different analyses, and comparisons or machine learning using the entire data set.

I was thinking along the same lines@alex3619.

When Ron described matching patients' gene expression data to gene expression data for every known disease (closest match being Sleeping Sickness), I was trying to visualize the amount of data and systems/computation power they used which must have been substantial. He was speaking at the London Conference in June.

www.investinme.eu/IIMEC13.shtml#report

Wonder if @Ben H could do a "Science Wednesday" on these state-of-the-art computer technologies?

 

[FMMM1]

I was thinking along the same lines@alex3619.

When Ron described matching patients' gene expression data to gene expression data for every known disease (closest match being Sleeping Sickness), I was trying to visualize the amount of data and systems/computation power they used which must have been substantial. He was speaking at the London Conference in June.

www.investinme.eu/IIMEC13.shtml#report

Wonder if @Ben H could do a "Science Wednesday" on these state-of-the-art computer technologies?

I think Wengzhong Xiao presented gene expression data (at the OMF Community Symposium 2017) which showed that the closest match was sepsis [https://www.omf.ngo/2018/07/10/healthrising-new-harvard-me-cfs-research-center/ - search for sepsis].

 

[Gemini]

I think Wengzhong Xiao presented gene expression data (at the OMF Community Symposium 2017) which showed that the closest match was sepsis

Nice catch @FMMM1!

Took another look at Xiao's 2017 presentation (14:20 on the video) where he presents a gene expression chart:

Up front he said it was "preliminary" data, so wonder if during the past year they did further analysis? Sleeping sickness is on the chart just further down the list.

Xiao is among speakers @BenH just posted for this year's Symposium. Looking forward to his presentation.

And wondering if Nancy Klimas' findings complement these?

 

[Gemini]

ML can be used in various ways for medical research....with the rituximab trials and cycloME we might be able to apply this if they collected enough data on the patients (which I would think they did). And for future clinical trials this will be possible.

Great examples of machine learning applications @used_to_race! Thanks for posting!

Nancy Klimas has said on many occasions we have plenty of "biomarkers" (after decades of research). And Ron Davis' team is collecting "Big Data" and making it available on-line to researchers worldwide.

Is this paving the way for ML applications in ME/CFS? Is there a way to connect with ML researchers who might be interested in applying it to treatments like rituximab or working with ME/CFS research centers?