• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

PACE Trial and PACE Trial Protocol

Dolphin

Senior Member
Messages
17,567
Graph of fatigue in the general Norwegian population

It might be interesting to mark in the PACE Trial results (and perhaps for other trials also) on this graph

Loge1998-fatigueinthegeneralNorwegianpopulation.jpg


From:
J Psychosom Res. 1998 Jul;45(1 Spec No):53-65.
Fatigue in the general Norwegian population: normative data and associations.
Loge JH, Ekeberg O, Kaasa S.

Note: these aren't the healthy scores - the mean for those with "no disease/current health problem" was 11.2 (no standard deviations were given in the paper).
 

Dolphin

Senior Member
Messages
17,567
You know me and analogies: well I think I have found one that illustrates why I am not very bothered whether Cohen's formula shows that the effect is big or small. All the following numbers given are properly scaled versions of the GET/SMC Chalder scores in terms of exam percentages (where 33 on the Fatigue Scale is 0%, and 0 is 100%).

Imagine that you desperately want to be a doctor, in fact a psychiatrist, but you are told that you need a grade A at A-level Statistics to be accepted on the course. You took the exam last summer and got 15%. You are told that, on average, people who join the School's Maths Class for the year, and work hard, increase that mark to 29%. Those who also have a dozen or so of Graham's Expensive Tuition sessions in addition to the usual SMC, on average, increase their mark to 38% (which could be a grade E or D).

Paying for all those extra maths lessons on average adds a further 9% to the students' marks. Cohen would tell you that these improvements are large in relation to how bad you were at statistics, but are they worthwhile?

What if you are told that 13 out of every 15 students that have the extra lessons only increase by 1% to 30%, but that 2 students increase to 91%? That Graham has lots of data about the students but hasn't put the time in to determine which characteristics ensure success? That now everyone who wants to retake Statistics has to pay for all the extra lessons on the grounds that some of them will make it? And, just incidently, Graham will make a lot of money out of it.

My attitude is that statistics should simplify and illuminate, but we should not let calculations take the place of value judgements.
Keep doing the analogies, Graham - they're great.

One small point that I'm not sure I highlighted before - I know I restrained myself when I initially saw your video as I didn't want to dampened your enthusiasm: the Chalder fatigue scale is an odd scale: if you answer "same as usual" (which with people with ME/CFS should be "same as when healthy") you score 11 rather than 0. So 11 could be considered 100% (or 90 (?)% anyway)

(Aside: in an MS CBT Trial that somebody (Snow Leopard?) highlighted, involving Rona Moss-Morris and Trudie Chalder IIRC, lots were scoring under 11 which was a bit odd!).
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
Modern Alchemy: Turning GOLD into LEAD.

Hi, it was suggested to me to advertise my blog on what the PACE trial could have been but failed to be:

http://forums.phoenixrising.me/entry.php?940-Modern-Alchemy-Turning-GOLD-into-LEAD.

This is a discussion of the use of ME in advocacy, but intended for a global audience, and of research protocols - and why the PACE trials are more the "lead" standard than the "gold" standard.

I know who the "other guy" is, I just don't mention him. Mark would get the reference I think.

Bye
Alex
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
Keep doing the analogies, Graham - they're great.

One small point that I'm not sure I highlighted before - I know I restrained myself when I initially saw your video as I didn't want to dampened your enthusiasm: the Chalder fatigue scale is an odd scale: if you answer "same as usual" (which with people with ME/CFS should be "same as when healthy") you score 11 rather than 0. So 11 could be considered 100% (or 90 (?)% anyway)

(Aside: in an MS CBT Trial that somebody (Snow Leopard?) highlighted, involving Rona Moss-Morris and Trudie Chalder IIRC, lots were scoring under 11 which was a bit odd!).

It's confusing, but I came to the conclusion that the authors of the PACE Trial aren't aware of the subtleties of the Chalder Fatigue scale, and that 'zero' is considered to be well, and that '11' is considered to be unwell/fatigued. I think it becomes too confusing to work to any other interpretation, especially as it is not stated in any of the PACE Trial literature.
 

Graham

Senior Moment
Messages
5,188
Location
Sussex, UK
One small point that I'm not sure I highlighted before - I know I restrained myself when I initially saw your video as I didn't want to dampened your enthusiasm: the Chalder fatigue scale is an odd scale: if you answer "same as usual" (which with people with ME/CFS should be "same as when healthy") you score 11 rather than 0. So 11 could be considered 100% (or 90 (?)% anyway)

Glad you enjoyed the analogy: it's the teacher in me - no matter what I do, it keeps resurfacing.

Thanks for the comment about the Chalder scale. As Bob said, we did discuss it, but it is such a mess - what on earth does "no more than usual" convey to someone like my son who can never actually remember what it is like to be well? Let's face it, I'm getting rather hazy about what it was like to be healthy.

When I was in the peak of health, how could I ever have answered "Do you feel weak?" as "less than usual"?

Stepping back from how it was actually used, the options given for the answers read as if anyone who is in a stable state, even if they are very ill, should score 11. No wonder it is confusing.

So I fudged the issue by using a colour gradient.

P.S. Dampening my enthusiasm is a very difficult task! There's no need to be careful about that. Ask Bob!
 

Dolphin

Senior Member
Messages
17,567
It's confusing, but I came to the conclusion that the authors of the PACE Trial aren't aware of the subtleties of the Chalder Fatigue scale, and that 'zero' is considered to be well, and that '11' is considered to be unwell/fatigued. I think it becomes too confusing to work to any other interpretation, especially as it is not stated in any of the PACE Trial literature.
I'm not convinced of this - have you evidence to suggest this?/what did you read that suggested this?
 

Dolphin

Senior Member
Messages
17,567
Glad you enjoyed the analogy: it's the teacher in me - no matter what I do, it keeps resurfacing.

Thanks for the comment about the Chalder scale. As Bob said, we did discuss it, but it is such a mess - what on earth does "no more than usual" convey to someone like my son who can never actually remember what it is like to be well? Let's face it, I'm getting rather hazy about what it was like to be healthy.

When I was in the peak of health, how could I ever have answered "Do you feel weak?" as "less than usual"?

Stepping back from how it was actually used, the options given for the answers read as if anyone who is in a stable state, even if they are very ill, should score 11. No wonder it is confusing.
Here is what it says over the questionnaire:
We would like to know more about any problems you have had with feeling tired,
weak or lacking in energy in the last month. Please answer ALL the questions by
ticking the answer which applies to you most closely. If you have been feeling tired for
a long while, then compare yourself to how you felt when you were last well. (Please
tick only one box per line).

I agree the wording is not great. And as somebody who has been ill 22 years since I was 16 and severely affected 17 years, I would also find it hard to remember. But it does say "compare yourself to how you felt when you were last well".

But the MS results, where people scored less than 11, are interesting.

So I fudged the issue by using a colour gradient.
Ok, good to know you had that in mind. Although I think it could be challenged by somebody who wanted to disagree which is why I think using 11 would be better.
 

Graham

Senior Moment
Messages
5,188
Location
Sussex, UK
But the MS results, where people scored less than 11, are interesting.

Ok, good to know you had that in mind. Although I think it could be challenged by somebody who wanted to disagree which is why I think using 11 would be better.

You could well be right: I'll have to put more thought into it. I have two problems with accepting that 11 is healthy though. The first is that it would mean that the Chalder scale is really a 22 point scale, 11 to 33, and the second, that the only way to score less than 11 when you are perfectly healthy is by consuming some form of illegal substance (of which, of course, as an ex-teacher and an upstanding member of the community, I have no knowledge whatsoever). But then the problems with "difficulty in speaking" "finding the correct word" and memory questions would counter the euphoria.

I reckon the Chalder scale was originally devised by Charles Dodson as a practice run for the Mad Hatter's Tea Party.
 

Dolphin

Senior Member
Messages
17,567
You could well be right: I'll have to put more thought into it. I have two problems with accepting that 11 is healthy though. The first is that it would mean that the Chalder scale is really a 22 point scale, 11 to 33, and the second, that the only way to score less than 11 when you are perfectly healthy is by consuming some form of illegal substance (of which, of course, as an ex-teacher and an upstanding member of the community, I have no knowledge whatsoever). But then the problems with "difficulty in speaking" "finding the correct word" and memory questions would counter the euphoria.

I reckon the Chalder scale was originally devised by Charles Dodson as a practice run for the Mad Hatter's Tea Party.
Yes, it is odd to have it scores less than 11 for the reason you give.

Just to point out again, that in Norway, the mean for those with "no disease/current health problem" was 11.2 (no standard deviations were given in the paper).

See also the graph above for the distribution across society.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
I'm not convinced of this - have you evidence to suggest this?/what did you read that suggested this?

Yes, you're right not to be convinced Dolphin. I said it as a casual observation of the literature that I've read.

It's just that whilst dealing with the PACE Trial literature, and other information regarding the Chalder scale, I've never come across a suggestion that a score of 11 would be full health, except in this discussion thread, and I can't remember if anyone had found any official or peer reviewed information about the subject? It's very possible that I've missed something vital that you are aware of?

As far as I'm aware, the PACE Trial interprets a decreased score on the fatigue scale as success for the trial, no matter what the score is, at least statistically speaking. When we are analysing the data, then any drop in an individual participant's score would contribute to the overall drop in the average score.

If researchers use a score of 11 as a perfect health, then the scale becomes extremely unintuitive, and I question how helpful it would be, in terms of statistical analysis etc.

ETA: I've just read your comments re Norway, and other helpful comments. But I'd like to see some other relevant data before I am convinced that the authors and editors of the PACE Trial have any insight into any special significance of a score of '11'.

None of the PACE Trial analysis is based on a scale of 22 (11 to 33.)
 

Dolphin

Senior Member
Messages
17,567
Yes, you're right not to be convinced Dolphin. I said it as a casual observation of the literature that I've read.

It's just that whilst dealing with the PACE Trial literature, and other information regarding the Chalder scale, I've never come across a suggestion that a score of 11 would be full health, except in this discussion thread, and I can't remember if anyone had found any official or peer reviewed information about the subject? It's very possible that I've missed something vital that you are aware of?

As far as I'm aware, the PACE Trial interprets a decreased score on the fatigue scale as success for the trial, no matter what the score is, at least statistically speaking. When we are analysing the data, then any drop in an individual participant's score would contribute to the overall drop in the average score.

If researchers use a score of 11 as a perfect health, then the scale becomes extremely unintuitive, and I question how helpful it would be, in terms of statistical analysis etc.

ETA: I've just read your comments re Norway, and other helpful comments. But I'd like to see some other relevant data before I am convinced that the authors and editors of the PACE Trial have any insight into any special significance of a score of '11'.
A lot of the earlier work was done using the bimodal score. In this scale, scores of 0 and 1 are not differentiated. On this scale, a score of 4 or more was considered to show "fatigue caseness". This was validated by comparing the results with another scale as I recall.

None of the PACE Trial analysis is based on a scale of 22 (11 to 33.)
Did any of them explicitly say 0 to 33 i.e. have you specific points in mind?

However, I am coming around to your position a little: It does bring up the point that if somebody were to score less than 11 (as happened with the MS trial http://www.psychosomaticmedicine.org/content/70/2/205.full.pdf+html), this would artificially (in my opinion) bring down the average.

And thinking about it, although scores of less than 11 are probably still going to be unusual overall in a CFS trial, there are probably going to be people who, for whatever reason, tick "less than usual" (0) for the odd question.

Given the data from the MS study http://www.psychosomaticmedicine.org/content/70/2/205.full.pdf+html, it makes me wonder whether a score of 15 (say) from somebody in the general population and a score of 15 from somebody who has done a trial might not necessarily represent the same score: somebody who did CBT (or GET) (say) might more likely to tick some 0 boxes (even if it explicitly said answer it compared to before you became ill), while a healthy person might simply say they had the symptom "no more than usual" scoring 1 for the same question even though the level of impairment (or lack of it), might be the same. This would mean referring to population norms using the scale would not be appropriate* (if it occurred).

However, overall I think saying the scale is 0 to 33 [and ignoring the fact that if somebody said they were said "no more than usual" (where usual is before they became ill) for the 11 questions they would score 11] is a vulnerable position that could be criticised, especially given the Norwegian data.

But this conversation has got me to think a bit more about the issue.

* I already think that there may also problems with the SF-36 physical functioning questionnaire when comparing scores from people with ME/CFS with healthy people. Some people with ME/CFS might say "they have no problem" walking half a mile/a few blocks/0.5km** or whatever - because they can do it once a day or whatever (and this would lead to a score 10 points out of 10 on that item). But this may be the main activity that they do - lots may not be working at the same time. This comes up with disability assessments where somebody can walk 100m once so it is put down that they have no difficulty doing it - but they couldn't do it if they had to do it dozens of times in the course of their working day.

** The SF36 actually varies on this question from countries that use yards/miles, blocks (US) and kilometres (most countries I think).
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
A lot of the earlier work was done using the bimodal score. In this scale, scores of 0 and 1 are not differentiated. On this scale, a score of 4 or more was considered to show "fatigue caseness". This was validated by comparing the results with another scale as I recall.

Yes, it didn't help, at all, that they changed the scoring system.

Bob said:
None of the PACE Trial analysis is based on a scale of 22 (11 to 33.)
Did any of them explicitly say 0 to 33 i.e. have you specific points in mind?

Nothing in the PACE Trial explicitly says that the scale should be considered to be a 22 point scale, or that 11 points should be considered recovered and healthy. (As far as I'm aware).
As the scale is so unintuitive, I would have thought that it was vital for this to be explained somewhere, if that is how they are interpreting it.
All the graphs have a scale of 0 to 33, for example.

However, I am coming around to your position a little: It does bring up the point that if somebody were to score less than 11 (as happened with the MS trial http://www.psychosomaticmedicine.org/content/70/2/205.full.pdf+html), this would artificially (in my opinion) bring down the average.

Yes, the authors only seem particularly interested in the average scores coming down, and in their own definition of 'recovery'.
If they say anything about a score of '11' in the 'recovery' paper that they say they are going to publish, then obviously we will have to take that into account.
If they don't say anything, then I think we can safely use the 0-33 scale.

And thinking about it, although scores of less than 11 are probably still going to be unusual overall in a CFS trial, there are probably going to be people who, for whatever reason, tick "less than usual" (0) for the odd question.

Given the data from the MS study http://www.psychosomaticmedicine.org/content/70/2/205.full.pdf+html, it makes me wonder whether a score of 15 (say) from somebody in the general population and a score of 15 from somebody who has done a trial might not necessarily represent the same score: somebody who did CBT (or GET) (say) might more likely to tick some 0 boxes (even if it explicitly said answer it compared to before you became ill), while a healthy person might simply say they had the symptom "no more than usual" scoring 1 for the same question even though the level of impairment (or lack of it), might be the same. This would mean referring to population norms using the scale would not be appropriate (if it occurred).

Yes, it's a subjective scale anyway isn't it.

However, overall I think saying the scale is 0 to 33 [and ignoring the fact that if somebody said they were said "no more than usual" (where usual is before they became ill) for the 11 questions they would score 11] is a vulnerable position that could be criticised, especially given the Norwegian data.

Yes, I agree. As Graham said, we did discuss it, and decided to go with the simple and intuitive option.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)

Graham

Senior Moment
Messages
5,188
Location
Sussex, UK
Sean gave me a link to a study in the BJ of Psychiatry looking at the improvement of patients in ME centres. I was primarily interested in their prejudice against support groups, but spotted this interesting bit - they used the Chalder bimodal scale. For them scores of more than 3 indicate excessive fatigue, which could convert on the continuous scale to 6 to 9, so perhaps 7.5 would be their borderline for excessive fatigue on the continuous scale!!!

Their entry criterion was a score on the SF36 physical functioning subscale of less than 25, where this sub-scale has a scoring range of 10 to 30, where 10 indicates maximum physical limitation in self-care and 30 indicates the ability to do vigorous sports.

Always happy to stir things up!
 

Dolphin

Senior Member
Messages
17,567
Their entry criterion was a score on the SF36 physical functioning subscale of less than 25, where this sub-scale has a scoring range of 10 to 30, where 10 indicates maximum physical limitation in self-care and 30 indicates the ability to do vigorous sports.
The way to convert scores to the 0-100 scale is to take 10 away and multiply by 5 i.e. it's the percentage of the gap from 10-30. It involves the same questions as in the PACE Trial.
 

Dolphin

Senior Member
Messages
17,567
Here is the second correction from the Lancet that I'm aware of. This one relates to the Bleijenberg and Knoop editorial:
http://niceguidelines.blogspot.com/2011/06/lancet-corrects-bleijenberg-knoop.html

The first correction was relating to the SF-36 normal population data.
http://www.meactionuk.org.uk/whitereply.htm

If there are any more corrections then I think maybe we should start a thread specifically for PACE Trial corrections.
:Sign Good Job: Well done to those who have pushed to get these corrections recorded.
 

Dolphin

Senior Member
Messages
17,567
Sean gave me a link to a study in the BJ of Psychiatry looking at the improvement of patients in ME centres. I was primarily interested in their prejudice against support groups, but spotted this interesting bit - they used the Chalder bimodal scale. For them scores of more than 3 indicate excessive fatigue, which could convert on the continuous scale to 6 to 9, so perhaps 7.5 would be their borderline for excessive fatigue on the continuous scale!!!
To be exact, they say:
"scores greater than 3 indicate excessive fatigue"
This is the normal definition of "fatigue caseness" using the Chalder fatigue scale - it was the one that was supposed to be used in the PACE Trial.

A score of 4 can represent anything from 8 (4x2, 7x0) to 19 (4x3, 7x1) on the Likert scale.
 

oceanblue

Guest
Messages
1,383
Location
UK
:Sign Good Job: Well done to those who have pushed to get these corrections recorded.
Indeed.

It's worth noting that no significant new information has come to light since the original complaints about the 'Recovery' error. The Lancet paper clearly does not use the word 'recovery' at any stage. The protocol did. The Lancet should have been capable of reading paper and protocol and making their own minds up on that information - not relying on the authors to say they didn't report on recovery before acting. Surely the whole point is for the lancet to investigate independently.

I wonder if the Lancet will now take a different view of the PACE authors' failure to correct the Knoop/Beijenberg editorial's use of 'Recovery'? The PACE authors saw the editorial and its error and chose to say nothing, now the Lancet has to issue an apology and the Lancet might not be too thrilled about that.

Oh, the correction is good news, but I'm pissed off about how we are getting sops long after the damage has been done in the media and among among doctors who read the articles.
 

Sean

Senior Member
Messages
7,378
Sort of nice at least one of the more egregious errors is being properly corrected. Maybe we can get one or two more out of them. [/wishful thinking]

But have to agree with oceanblue, it really is too little too late, especially for such an obvious, outrageous, and damaging falsehood, one that is difficult to interpret as other than a deliberate, calculated misrepresentation by vested interests.

Any correction now in the mainstream media of those appallingly misleading headlines will almost certainly be minimal at best, in very small print, safely tucked away on page 87, just below the results of the local junior knitting club annual general meeting. (No offence to the fine and honourable craft of knitting.)

I very, very strongly suggest to the PACE authors, editorial writers, and The Lancet, that if they do not like being accused of incompetence and/or unethical conduct, then they behave competently and ethically and never again allow anything like this to occur. However aggrieved you may feel about the harsh words you copped from patients over this, it is as nothing compared to the sometimes brutal real-world consequences patients have to deal with as a result of your 'errors'.
 

Bob

Senior Member
Messages
16,455
Location
England (south coast)
I very, very strongly suggest to the PACE authors, editorial writers, and The Lancet, that if they do not like being accused of incompetence and/or unethical conduct, then they behave competently and ethically and never again allow anything like this to occur. However aggrieved you may feel about the harsh words you copped from patients over this, it is as nothing compared to the sometimes brutal real-world consequences patients have to deal with as a result of your 'errors'.

Well said Sean!