ME Analysis - Patient created Web document relating to the PACE study

In your analysis you talk about using better measures than the mean and standard deviation. I would go further and argue that the mean and standard deviation, and median and percentiles are not valid for the way they have processes the data from the chandler fatigue scale. Basically they say lets treat it as a likert scale and add up all the values and use this as a measure. That is simply wrong!

The chandler fatigue scale is a multi factor scale (including influances of physical fatigue and mental fatigue - look at the principle component analysis quoted in their original paper). A likert scale requires all the questions to relate to one underlying thing or attitude. You should show there is a high degree of correlation between the answers of all the questions. What they have created is a scale that is not necessarily monotonic - that is an increase the things you are measuring will not necessarily lead to an increase in the measure they are using. They could talk about the mode but that wouldn't be very useful.


Even if they had a single thing they were measuring they need to worry about whether it is an ordinal or interval scale. That is is the scale linear - does the gap between 1 and 2 represent the same gap as between 4 and 5. This is complicated by two factors, firstly the scale on which questions are answered. People answering the scale need to understand it as even and the scale needs to represent the full extremes of the possible answers. If we see bunching at the end of a scale then the end points probably represent a much bigger interval than the middle points.

The second factor is when you add up all the results of different questions, since there are no units it is hard to say that the resulting sum evenly covers the overall scale. I beleive psyciatrists tend to say but our intervals are of the same order of magnatide hence its ok.

If the scale is really an interval scale then you can use the mean and std and do things like regression on the results however if it is not you cann't.

As long as the results are ordered even if not evenly then you have an ordinal scale which you can quote the median and percentiles for. But the mean and std make no sense.

I would argue that if their questionaires only measure one (unseperable) concept then they have an ordinal scale. They may be able to argue that they have an interval scale but it is up to them to demonstrate that they have the correct properties. Where they have things like the chandler fatigue scale then there analysis is just wrong as its measuring some unknown function of two or more different things.

Agreed. Just on a very simple level, an improvement of 4 points on the fatigue scale can mean so many different things. But I suppose that this is also true of exam marks. An improvement from grade E to grade D gives absolutely no indication of what a person can and can't do.

Your argument reflects the way I think about the whole approach, but it is very mathematical in nature and we wanted to make this report accessible, as opposed to the more technical and finely accurate analyses produced elsewhere. It took me a long time to find a way to present the Bimodal/Likert conflict in graphical form, and yet now it looks so obvious: I did think about the "ordinal scale", but I just couldn't come up with a way of tying it all together in an accessible way. If you come up with anything, I'd be very interested to see it.

The main point is, of course, simply that even on their terms, there is no evidence of CBT or GET being worthwhile, and it is possible that their assessments made it harder to record harm. If we had been given information about medians and quartiles, we would have had a better picture of what happened to the majority of scores, and it would have underlined the fact that most patients hardly improved. As I see it, if we are not careful we can overwhelm that simple message by being too precise.

If they had stuck to the original Bimodal scoring, then the fatigue scale would simply have been a list of how many items gave difficulties. That would have been much clearer, but my guess is that no-one would have recorded much of a change in scores over the year.

Can I just add that it seems to me impossible to come up with anything of value from PACE. Yes from mostly bedbound - gradual increase in activity - my Doc and Osteopath encouraging, muscles moving and walking a little more, feeling better - then crash. Back to bed, feeling worse (after about 6 months of improvement). My Osteopath declined to treat and my Doc likewise. Additional problems set in - passings out, severe GI problems etc - all as we all know. Measure of fatigue in isolation from the rest of the illness is nonesense to me. Or to be blunt pathetic. Long term study like Graham's much more revealing.

For those of you who have been involved in the survey, and have a link to the campaign page, I have added a copy of the email that I sent to Richard Horton, editor of The Lancet, on 17th May, in response to his comment in an interview where he issued a challenge to the patient community to engage in a proper scientific discussion of the issues raised by the PACE trial. I haven't had any reply.

I then wrote to the deputy editor, Dr James, asking for an acknowledgement that the email had been received. That was a week ago - no reply.

It is an interesting interpretation of the term "discussion".

Now to consider the next moves.

Graham, you will probably never receive a reply from Horton. AFAIK, no one here who has emailed him has ever received one. I received an auto-reply when he was on holiday, which indicates that I got the address correct. Also, his "challenge" was issued over a year ago, it is unfortunate that due to illness it takes us so long to respond in detail, but he probably expected a response within weeks, so for him it would be old news by now. However, if the Lancet publish another PACE paper or related article/editorial then it might be a good time to bring the issue up again.

True enough. To be honest, I didn't really expect one. But it was the proper thing for me to do as a first step. If he chooses to ignore it, then it leaves me more free to take the next steps.

On the issue of time though, as it has taken a well-funded, well-supported and healthy PACE team over a year now to fail to come up with their next level of analysis of the data, I don't think we have done too badly

I agree. They have several papers planned, and this FOI request suggests another paper is getting closer to submission for publication (if there is no recovery or employment data in the next paper we will know something is wrong) :

http://www.whatdotheyknow.com/request/request_for_deterioration_rates

Yes my argument is very mathematical, I spend most of my time talking with engineers and mathematicions, and I'm not good at making things accessable. In other areas we are trying to use simulation, visualisation and games to get across complex ideas. I'm trying to think about how such an approach could be used with things like the PACE trial.

I don't think we will ever change their minds but it would be good if we could get the wider medical community to understand how bad their methodologies are.

How about trying:

Erik Engstrom
Reed Elsevier
1-3 Strand
London WC2N 5JR
United Kingdom


He is the CEO of Reed Elsevier who own the Lancet. I would have thought that they would have an interest that one of there major brands could become devalued because their review and editorial process is so weak that they publish papers with bad stats and where the entry conditions of someone being sick overlaps with exit conditions of someone within a "normal" range of activity.

Another thought is to push links to your we site on shareholder's message boards such as
http://uk.messages.finance.yahoo.com/UK_Stocks/forumview?bn=UKF-Stocks-REL.L

Seems to me that publishers will only really take critisism seriously if it hits their reputation and share price.

Nice suggestions! I have added them to my "to do" list (it seems to keep growing no matter how much I get done!).

I cannot get my head around why the medical world find this so difficult to understand. You and I clearly think alike on this. If I translate the situation to my world as a maths teacher, it looks utterly stupid.

If I had a class of 30 students who struggled with maths who were in their last year coming up to major exams, and I knew that 4 of them suffered badly from exam nerves and would benefit from six months of concentrating on exam techniques, relaxation tricks, improved pacing and timing, rereading questions after answering to make sure that they have done what was asked etc., the current ME strategy would then be to devote all the lessons for the next six months on exam techniques. Obviously it would boost those 4 students' marks, and may help some of the others pick up a couple of extra marks. But it would be utterly unprofessional and stupid for three main reasons.

Firstly, I could not justify spending so much time with the whole class on this when only a few need it. The appropriate thing to do would be to identify those 4 students and deal with them separately.

The second is that it would prevent me from actively helping the majority, and spending time and resources on their needs.

The third and biggest fault is that exam nerves are not a maths problem: they can affect any subject. The appropriate action would be for the school to set up special classes for those who are identified as having exam nerves, freeing me to deal with the specific problems of helping students to improve their mathematical understanding. There would be a risk there of interpreting exam nerves as being a maths problem, rather than an exam problem. Why can't they see that?

Can you just imagine the uproar from the parents if I spent 6 months not doing any maths but just concentrating on exam techniques? And if I produced evidence at the end that suggested there was a weak improvement over those 6 months (mostly caused by the great improvement of those 4 averaged out over the whole group), who would fall for it?

By the way, 4 in 30 is the correct proportion. CBT and GET, according to PACE, have a "number needed to treat" of 7 and 8 (or it might be 8 and 7).

Rant over!

What really galls is that these numbskulls have and do turn a blind eye to all the scientific discoveries/pathologies found over years, let alone medical treatments now being used for some symptoms. Who are the "trained" Docs, GP's they speak of - the NHS doesn't carry out the range of testing needed to identify the problems found everywhere else but the UK - so what are these apparently trained people trained in.

FWIW, the issue of the reporting of harms in the PACE Trial is dealt with in quite a lot of detail in section 6 of this paper which was published in a peer-reviewed journal:

If people think important points have been missed, they could write a letter to the editor for this journal.

One idea to get published would be the Chalder Fatigue Scale data. That is a scale that is used in a lot of studies. Then in the introduction and discussion sections, would could potentially discuss how this might relate to the PACE Trial.
Here is an example of such a study:

Even if you didn't manage to make all your points in any one paper, it could be quoted in future studies/papers/letters.

Thanks for the suggestion, Dolphin. I have two difficulties with this though. The first is that there is masses of good-quality, scientific-style debunking of the psychological interpretation of ME, but it doesn't seem to be getting through to mainstream understanding. So would it be worth my time and effort (especially considering my second difficulty)? Or is it something to postpone? Is it better for me to focus on accessible stuff for the time being?

The second is that I have no experience of how to go about it, and my linguistic style is far from what is required. Putting the ideas into that format would be tough, very tough, for me. Stand me in front of a group of a hundred people who need to be convinced, give me a pen and a board, and I'd do it! I've had a life of talking, not writing.

No particular rush on it I suppose.

But basically, the world of the Lancet and the like I don't think deals with data in the format you have. And indeed for some of us who try to enter the world a bit through published pieces of one sort or another, it would be difficult to quote any data or analysis you have until it is published somewhere.

Keep at it, Dolphin. I've been telling Graham from the beginning that if we really want to reach GP's/PCP's, we've got to produce a version which can be submitted to a peer-reviewed journal. (preferably one which is already indexed on PubMed, no offense to IACFS/ME)

It's a great idea to aim to get it published, but would Graham have to think about ethical considerations, and designing consent forms?

Ok. I don't want to put any unnecessary pressure on you. But I have read quite a lot of papers and sometimes, it is obvious that English isn't the writer's/writers' first language.

Of course, probably for many high-impact journals e.g. the Lancet or BMJ, the English can be quite good. But they aren't the only journals out there. Also, there can be people out there who can help with writing once drafts are done: different people have different skills and knowledge.

Not giving any guarantees I'll volunteer myself I'm afraid, but if a draft came my way, I might help in some way (maybe more with references, although I can sometimes coming up with the odd synonym and the like to make something look a bit fancier. And I might know others who might help too. And WillowJ is good with words.

Afraid I'm not an expert on that end of things. One could contact other people who have published questionnaire data and see what they did.

Hmm, consent forms. I can't even get in touch with all the people who took part in the study. Two of them have vanished off the radar (i.e. left PR or changed their email addresses). But would the initial announcement be sufficient? (The statement that I would use their responses totally anonymously in our study.)

As for the offer of help, Dolphin, and the suggestion of Willow J wielding a whip over me while I sort out my grammar – I would have had you both earmarked for that anyway!

I have just finished the first of a possible short series of YouTube videos enticing people to our study, and at the moment am phrasing a letter to MPs in the light of the fact that the consultation period "Liberating the NHS: Greater Choice and Control" runs out on 20th July, and I want to draw their attention to the fact that patient power at grassroots level is no good if there is no coherent structure that enables us to ensure that all the GP consortia are following proper guidelines and actually providing support (cue the AfME report). What with that and starting up a Facebook page, I'm really pushing the boundaries of what a tired old ex-maths teacher is able to get his brain around.

The YouTube video combined a standard video, a couple of graphics and the race animation, taken at different sizes and frame rates. The software I used, Blender, was free, very powerful, and has hideously poor documentation. My brain hurts!