FORWARD ME GROUP: Minutes for the meeting held at the House of Lords on December 7th 2016

[charles shepherd]

FORWARD ME GROUP:

Minutes for the meeting held at the House of Lords on December 7th 2016

This meeting included a presentation from Professor Stephen Holgate on the MEGA research study, which as followed by questions

From the Minutes section on Forward ME group website:

http://www.forward-me.org.uk/7th December 2016.htm

 

[Sasha]

Thanks for posting this, @charles shepherd.

6. NICE

6.1Proposed review of NICE Guideline. The Chairman referred to a communication from NICE received the day before seeking any comments we wished to make on this subject. It was agreed we would send any such information to the Chairman who would collate it and forward it to NICE.

6.2Dr Charles Shepherd wondered whether there was any deadline for NICE receiving comments. He had the impression that NICE’s checking process would probably go on for several months. He added that NICE seemed to be seeking published papers that throw new light on clinical assessment features or management features of the illness, NOT opinions.

When they say they want comments, what does that mean? Are they asking for things in the published academic literature, such as the new paper in Fatigue critiquing PACE's "recovery" rates?

Is the MEA or anyone intending to collate a list?

 

[Esther12]

Thanks. First thing I did was search for 'Crawely' and didn't find anything, but will be back to look at the rest later.

 

[charles shepherd]

Thanks for posting this, @charles shepherd.



When they say they want comments, what does that mean? Are they asking for things in the published academic literature, such as the new paper in Fatigue critiquing PACE's "recovery" rates?

Is the MEA or anyone intending to collate a list?

Sorry - this isn't as clear as it could be

We can comment on the process that is taking place - which we have done

But any evidence that is going to be submitted to the 'evidence checking proces' that is about to take place at NICE, the outcome of which will form the basis for a decision on whether or not to formally review the NICE guideline later in 2017, will have to be in the form of research that has been published since 2007 that relates to either clinical assessment or management

NICE are not really interested in the debate into causation of ME/CFS - as this is not their remit

And NICE are not interested (at this stage) in opinion

The MEA will be producing a list of key findings that need to be considered by whoever is looking at this evidence

 

[A.B.]

Holgate still doesn't realise that by protecting Crawley, he has erased his own credibility.

It is very clear that a large number of patients have no trust whatsoever in Crawley (see all the comments on both petitions). By pretending that this distrust and these concerns do not exist, Holgate is taking the side of Crawley and thus our distrust extends to him. No kind words, promises or grand plans can undo this as we only expect lies from the BPS school. They know how to be persuasive, but they are unable to honestly acknowledge problems and respecting patients. Just like Holgate in this story.

 

[Sasha]

But any evidence that is going to be submitted to the 'evidence checking proces' that is about to take place at NICE, the outcome of which will form the basis for a decision on whether or not to formally review the NICE guideline later in 2017, will have to be in the form of research that has been published since 2007 that relates to either clinical assessment or management

NICE are not really interested in the debate into causation of ME/CFS - as this is not their remit

And NICE are not interested (at this stage) in opinion

The MEA will be producing a list of key findings that need to be considered by whoever is looking at this evidence

Thanks. That sounds as though it should include the PACE "recovery" critique paper, for instance, since it's a critique of a management trial.

 

[Barry53]

And NICE are not interested (at this stage) in opinion

If only they had recognised opinion for what it was all those years ago, when they established the ME/CFS guidelines.

 

[Barry53]

The method involves collecting together large numbers of patients who show different “expressions” of the disease in question (such as different degrees of severity, different times of onset). You then examine them and perform various measurements within those groups, use standard clinical tests and then enter all the data into a computer programme to identify sub-types. This is called “cluster analysis”.The method makes no assumption about existing criteria such as Fukuda or Canada – although a number of participants in the study will conform to those criteria, but others will not. This lack of restraint had been very successful with migraine recently. It was very important, he said, that we respect heterogeneity.

Provided the data includes good representative sampling of all degrees of severity (from 0% to 100%), then the above makes very good sense. Cluster analysis is a data mining technique that starts off with no preconceptions, teasing out data clusters that may not be at all obvious from conventional analysis; highlighting similarities within each cluster, whilst highlighting differences between clusters. But of course the "cr*p in, cr*p out" principle applies, so the information out can only be as good as the data in. Would have to be very careful that those at the high end of the severity spectrum are adequately represented, and not just a token representation. And because there must be no preconceptions about where the spectrum terminates at either end, the low-to-no severity end of it must also be sensibly represented. The point here is that the cluster analysis itself should be teasing out what the spectrum comprises, and what its limits are, and not be presumed by researchers constraining the input data according to their own preconceptions.

It is pretty much agreed by all that ME/CFS encompasses a number of sub-types ... trouble is no-one has a clue what they are or how many. Cluster analysis could well unlock the secret of what those subsets are, along with what distinguishes them from each other, which would of course be a huge leap forward.

The minutes make good sense and are actually really encouraging, so it is just such a pity that MEGA is dragged right down by the presence of EC, and Prof Holgate's continuing support for her and her ways.

 

[TiredSam]

The work of MEGA was more akin to astro-physics. Think about discovering black holes in space.

Not the first time MEGA has been compared to a black hole.

The Chairman thanked Professor Holgate for being so helpful, and asked to whom we should send questions. The Professor said we should send them to the website.

Well there's our first black hole discovered right there.

Sorry to be so flippant, but I was struck by how many questions were unasked or unaddressed.

SC wasn't even there, something else must have had a higher priority than representing patient charities. What was her role in MEGA again?

 

[charles shepherd]

Thanks. That sounds as though it should include the PACE "recovery" critique paper, for instance, since it's a critique of a management trial.

The 'powers that be' at NICE are aware of the re-analysis of the PACE trial results following the FoI tribunal

 

[charles shepherd]

Not the first time MEGA has been compared to a black hole.



Well there's our first black hole discovered right there.

Sorry to be so flippant, but I was struck by how many questions were unasked or unaddressed.

SC wasn't even there, something else must have had a higher priority than representing patient charities. What was her role in MEGA again?

This meeting was time limited to about an hour - so although quite a lot of questions were raised it wasn't possible to deal with every question

 

[charles shepherd]

Not the first time MEGA has been compared to a black hole.



Well there's our first black hole discovered right there.

Sorry to be so flippant, but I was struck by how many questions were unasked or unaddressed.

SC wasn't even there, something else must have had a higher priority than representing patient charities. What was her role in MEGA again?

SC could not attend because she was out of the country

 

[charles shepherd]

Provided the data includes good representative sampling of all degrees of severity (from 0% to 100%), then the above makes very good sense. Cluster analysis is a data mining technique that starts off with no preconceptions, teasing out data clusters that may not be at all obvious from conventional analysis; highlighting similarities within each cluster, whilst highlighting differences between clusters. But of course the "cr*p in, cr*p out" principle applies, so the information out can only be as good as the data in. Would have to be very careful that those at the high end of the severity spectrum are adequately represented, and not just a token representation. And because there must be no preconceptions about where the spectrum terminates at either end, the low-to-no severity end of it must also be sensibly represented. The point here is that the cluster analysis itself should be teasing out what the spectrum comprises, and what its limits are, and not be presumed by researchers constraining the input data according to their own preconceptions.

It is pretty much agreed by all that ME/CFS encompasses a number of sub-types ... trouble is no-one has a clue what they are or how many. Cluster analysis could well unlock the secret of what those subsets are, along with what distinguishes them from each other, which would of course be a huge leap forward.

The minutes make good sense and are actually really encouraging, so it is just such a pity that MEGA is dragged right down by the presence of EC, and Prof Holgate's continuing support for her and her ways.

Sounds as though you know something about cluster analysis in relation to this dreadful term 'big data' !

Do you have academic expertise in this area of statistical analysis?

 

[Sasha]

The 'powers that be' at NICE are aware of the re-analysis of the PACE trial results following the FoI tribunal

Not just the Matthees et al. preliminary analysis but also the Wilshire et al. critique just published in Fatigue that explains why that reanalysis was necessary.

I hope they understand why the main outcomes reanalysis was necessary too.

 

[alex3619]

Requiring analyses to be published in peer review journals before being considered by NICE or any other agency is bureaucratic double speak. Don't they have people capable of evaluating scientific papers? Or is everyone lacking the necessary competence? Or is it that their bureaucratic rules ignore scientific commentary? Not everything in science is published in journals. The flaws of PACE are both severe and obvious. How can it be missed and whoever is missing it claiming to be competent? This is more likely to be a political than a scientific issue.

 

[Valentijn]

Seriously. It sounds like "We will ignore blatant and fatal flaws until they are exposed in another publication."

It makes the reviewers look shockingly incompetent - if a bunch of patients can analyze research, why can't they? Do they just read all of the abstracts and assume they're complete and accurate, unless some other abstract says otherwise?

 

[TiredSam]

Sounds as though you know something about cluster analysis in relation to this dreadful term 'big data' !

Do you have academic expertise in this area of statistical analysis?

By the same token:

The work of MEGA was more akin to astro-physics. Think about discovering black holes in space. He said this was probably why some people had difficulty with the concept.

I found Prof. Holgate's attempt to bluff his way in astro-physics in order to portray the arguments of patients with legitimate concerns about MEGA as "some people" (there's that phrase again, beloved of old ladies in the queue at butchers loudly gossiping about the woman behind them) having "difficulty with the concept" to be patronizing bluster.

Leaving aside the fact that many on this forum could give Prof. Holgate a run for his money in the astro-physics department, it just looks to me like another example of the standard BPS technique of stealing from the language of science in order to shroud their wisdom in mystery too complex for mere mortals, or, heaven forfend, patients.

 

[trishrhymes]

If anyone's interested, Holgate's answers on MEGA at this meeting are also being discussed on the 'New MEGA study website' PR thread - I won't repeat my comments here...

 

[TiredSam]

If anyone's interested, Holgate's answers on MEGA at this meeting are also being discussed on the 'New MEGA study website' PR thread - I won't repeat my comments here...

Well in that case allow me to repeat one of your comments from that thread:

By insisting on starting the project with questionnaire and sample collection as the sole purpose of the grant application, with no guarantee of further funding even being applied for until all that collecting has been completed, we may not see any actual biomedical research at all. At best we will have to wait some years while MEGA gets its act together and collects 12000 patients' data and samples.

Because it's a perfect example of one of the concerns that wasn't even raised at the meeting.

 

[user9876]

By the same token:



I found Prof. Holgate's attempt to bluff his way in astro-physics in order to portray the arguments of patients with ligitimate concerns about MEGA as "some people" (there's that phrase again, beloved of old ladies in the queue at butchers loudly gossiping about the woman behind them) having "difficulty with the concept" to be patronizing bluster.

Leaving aside the fact that many on this forum could give Prof. Holgate a run for his money in the astro-physics department, it just looks to me like another example of the standard BPS technique of stealing from the language of science in order to shroud their wisdom in mystery too complex for mere mortals, or, heaven forfend, patients.

I think there are big issues around the quality of the data prior to using any sort of cluster analysis or machine learning techniques. Physicists look at their data and what to collect using knowledge of underlying theories and of the noise and receiver characteristics of measurement equipment. None of this seems to be being done here,

I think there are issues around:
1) The dynamics of ME and understanding when and what state patients were in when samples were taken. (Depends what is being measured but are the samples different after exertion (perhaps getting to the test place), what about when PEM is a factor).
2) By not carefully choosing a sample there could be issues around noise added into the data. For example, if co-morbid disease is not taken into account (as well as the dynamic issues above).
3) Non-linearities in the data particularly poor quality questionnaires such as CFQ and the SF36 where if they are trying to use these to assess the level of disability then issues with the scales could cause algorithms to fail.

More generally most cluster analysis techniques are not particularly robust to noise (especially non-Gaussian noise) and this can be an issue. Although they could be designed to deal with that by using alternative distance metrics/outlier rejection etc. But without carefully thinking through the data characteristics then it is dangerous.

I know few people working in data analytics who would apply algorithms without having a very good understanding from subject matter experts. Which leaves me worrying at Holgate's comments.