PACE Trial and PACE Trial Protocol

[Bob]

BTW, I apologise to everyone if this has already been gone over in detail before...
I often miss things on the thread, but I just can't read through the entire thread again!
So I'm relying on everyone else's memories!!! (Hopeless?)

 

[Esther12]

But thanks for that... It could explain it... Except that they have said they used the 'English adult population'... And the total sample mean is given as 89.6 for the ONS data.

I don't really understand why the data in table 3 and 4 seems to differ. It looks like the data in table 4 has yet fewer people included too.

Ah, OK, maybe that could explain it. Thanks for that Esther.
It still doesn't explain their correction though, does it.

No probs. I'm not sure which correction you mean?

 

[Esther12]

BTW, I apologise to everyone if this has already been gone over in detail before...
I often miss things on the thread, but I just can't read through the entire thread again!
So I'm relying on everyone else's memories!!! (Hopeless?)

We're all in the same boat there. It's often handy to look again at things anyway. I don't think that I'd noticed the difference between the data for all the population, and male+female previously.

 

[Bob]

No probs. I'm not sure which correction you mean?

In their published correction, they said they used the English adult population, which only correlates to HSE.

We're all in the same boat there. It's often handy to look again at things anyway. I don't think that I'd noticed the difference between the data for all the population, and male+female previously.

Yes, I agree, it can be very helpful to touch on subjects again, can't it.

 

[Esther12]

Thanks for that biophile... I'd forgotten it was a formally published correction.
"(in our paper we stated that this was a UK working-age population, whereas more accurately this should have been an English adult population)"
http://www.thelancet.com/journals/lancet/article/PIIS0140-6736(11)60651-X/fulltext

Ah... I see. That was a rather curt correction for such a significant error, which played such an important role in affecting the way in which their results were reported.

Phrasing it the way they did could lead some reader to think that the correction was a trivial shift from UK to English norms, rather than the more significant abandonment of the working age population, for one where 25% of the population was aged over 65.

 

[Esther12]

biophile: It could only be me who is troubled by this, but the larger text font you use makes your posts slightly harder to read for me. I've just noticed myself skimming over what you've written a bit, even though it seemed interesting. It's a bit of a petty thing to even mention, but I thought I would in case anyone else found that same thing but was not saying so.

 

[Graham]

Can I chip in as a mathematician and express my despair at using statistical methods for determining who is healthy? Standard deviations are used in well-defined normal/Gaussian distributions, like the height of adult Caucasian males. You don't even combine men and women's heights into one distribution! Standard deviations are not like some all purpose measuring tape: they have to be used with great care, and combining different data distributions is a minefield.

I've used this analogy before, but when I started teaching, in examinations they used to grant O-level passes to the "top" 20% of the school population in maths (and 40% in English). Then, as they moved to GCSEs, they moved to a standard based assessment, like driving tests, where student had to reach prescribed levels to obtain the grades. That's obviously the right way to determine who passes and who fails. When all the marked scripts come together, experienced teachers and exam moderators decide for each exam, at what stage pupils can be said to have reached those prescribed levels.

So why can't experienced doctors, looking at the results of a large number of healthy and of ill patients determine the "pass" levels without playing around with statistics that they don't understand? Have they no faith in their own ability to tell healthy from ill, or do they think that using complicated calculations just makes things look better?

I also suspect there is some "weasel word" work going on around the use of the word "normal" rather than "healthy". How abnormal is it to be ill?

 

[biophile]

biophile: It could only be me who is troubled by this, but the larger text font you use makes your posts slightly harder to read for me. I've just noticed myself skimming over what you've written a bit, even though it seemed interesting. It's a bit of a petty thing to even mention, but I thought I would in case anyone else found that same thing but was not saying so.

Hmmm, that hasn't been my intention at all, in fact the opposite. Thanks for letting me know. Maybe it is a setting, but the default text size on the default theme looks big to me so I chose what looks like 10pt Verdana to shrink down the larger posts, although come to think of it "size 3" in HTML is bigger than that so something must be set wrong at my end?

 

[Bob]

Hmmm, that hasn't been my intention at all, in fact the opposite. Thanks for letting me know. Maybe it is a setting, but the default text size on the default theme looks big to me so I chose what looks like 10pt Verdana to shrink down the larger posts, although come to think of it "size 3" in HTML is bigger than that so something must be set wrong at my end?

Aw, I quite liked your font. It seemed neat and tidy. But if it's not helpful for Esther then best not to use it.
Just to let you know, the text in this post (quoted) looks like everyone elses.
And your other font looked a similar size to the standard font, to me. (Maybe a slightly smaller font, but more spaced out, horizontally, between each character and each word.)

 

[Bob]

Can I chip in as a mathematician and express my despair at using statistical methods for determining who is healthy? Standard deviations are used in well-defined normal/Gaussian distributions, like the height of adult Caucasian males. You don't even combine men and women's heights into one distribution! Standard deviations are not like some all purpose measuring tape: they have to be used with great care, and combining different data distributions is a minefield.

I've used this analogy before, but when I started teaching, in examinations they used to grant O-level passes to the "top" 20% of the school population in maths (and 40% in English). Then, as they moved to GCSEs, they moved to a standard based assessment, like driving tests, where student had to reach prescribed levels to obtain the grades. That's obviously the right way to determine who passes and who fails. When all the marked scripts come together, experienced teachers and exam moderators decide for each exam, at what stage pupils can be said to have reached those prescribed levels.

So why can't experienced doctors, looking at the results of a large number of healthy and of ill patients determine the "pass" levels without playing around with statistics that they don't understand? Have they no faith in their own ability to tell healthy from ill, or do they think that using complicated calculations just makes things look better?

I also suspect there is some "weasel word" work going on around the use of the word "normal" rather than "healthy". How abnormal is it to be ill?

So you mean that you can't lump together healthy people, ill people, and disabled people, and then do a simple statistical analysis and come up with an answer for who is 'normal'?

Then I question the purpose of maths!


Edit: my dodgy humour doesn't always translate on forums, so I'd better just make it clear that I said this with a strong sense of irony.

 

[Esther12]

Maybe it is a setting, but the default text size on the default theme looks big to me so I chose what looks like 10pt Verdana to shrink down the larger posts, although come to think of it "size 3" in HTML is bigger than that so something must be set wrong at my end?

It could be some combination of our different settings then. Sorry to be a pain!

I use ctrl+ and ctrl- to change the level of zoom in my browser, so if you're finding everyone's writing a bit too big, that could be a way a dealing with it.

So you mean that you can't lump together healthy people, ill people, and disabled people, and then do a simple statistical analysis and come up with an answer for who is 'normal'?

Don't worry, it's perfectly 'normal' to die of cancer. The treatment has been a success!

Graham: I sometimes forget that their statistical trickery is meant to be related to meaningful real-world outcomes. Dolphin has made a few posts with suggestions as to how measures of recovery could be sensibly defined, and his comments often seem so obviously sensible that I'm amazed that sort of approach is not used as the universal default.

 

[Graham]

Hey Bob, the purpose of making everybody do maths is to make virtually everybody feel inadequate (except a geeky few, who nobody envies). As to what use it is - many mathematicians pride themselves on just how useless their research is. I forget his name, but one mathematician worked on primes and decided it was utterly abstract. It is now used for (brainfog moment - putting things in code - computer security - can only be read by recipient).

 

[user9876]

. It is now used for (brainfog moment - putting things in code - computer security - can only be read by recipient).

I think the word you are searching for is Cryptography.

I don't think they are using maths in their papers to make everybody feel small but rather to frighten people away from challenging them. Seems to me like they do things by rote without understanding hence they don't realise their errors.

I looked very briefly at the FINE trial paper. I noticed Graham Dunn was the statistition involved (I've found one of his books very useful in the past). I also noticed that he doesn't list the paper with his other publications on his web page. Made me wonder if it was shame or if he doesn't know about the paper.

 

[Graham]

I know from experience that hardly anyone ever challenges a mathematician when it comes to calculations. Years ago the head of my school wanted to buy a key system for the photocopier that counted and regulated the number of photocopies that each head of department could use. It was expensive, and I did a quick calculation as to how many photocopies we would have to save to make it worthwhile. When I told them, they all agree it was a foolish idea. Only later that evening did I realise that I was out by a factor of ten!

 

[Esther12]

I know from experience that hardly anyone ever challenges a mathematician when it comes to calculations.

Quick OT: This is a really worrying human trait imo. It's normally incredibly difficult to really win political or moral arguments when just talking with people... however start referencing studies or guff about neural correlates and people end up totally convinced of what you're saying, without taking a moment to think about whether the studies sound well designed, have been replicated, etc. I keep noticing examples of piss-poor studies being used to sway political and moral arguments recently, and it's scary to see how people have developed a deference to new kinds of authority as religion, politicians, the police, etc have become increasingly discredited.

 

[Dolphin]

I know from experience that hardly anyone ever challenges a mathematician when it comes to calculations. Years ago the head of my school wanted to buy a key system for the photocopier that counted and regulated the number of photocopies that each head of department could use. It was expensive, and I did a quick calculation as to how many photocopies we would have to save to make it worthwhile. When I told them, they all agree it was a foolish idea. Only later that evening did I realise that I was out by a factor of ten!

I think being scared of making an error is one of the reasons clinicians or researchers don't write more letters to the editor. There has been lots of rubbish written in the field for most of the 2000 that went unchallenged (except a few studies in the BMJ, Lancet, etc.). Anyway, as I've said before I think it's up to us to challenge most papers - we can't depend on clinicians or other researchers to do it.

 

[oceanblue]

Peter White's earlier, more reasonable and published view on 'normal' SF36 scores

This from the 1997 evaluation of graded exercise for CFS, which found the mean SF-36 PF score improved from 48.5 at baseline to 69 after 12 weeks of treatment:

... SF36 physical function score
[and other measures] were significantly
better after exercise than after flexibility treatment,
though none of these measures had returned to
normal

So here a Sf-36 PF score of 69 is not a return to normal, compared with the PACE threshold of 'normal' of just 60. Strange.

 

[Don Quichotte]

So why can't experienced doctors, looking at the results of a large number of healthy and of ill patients determine the "pass" levels without playing around with statistics that they don't understand? Have they no faith in their own ability to tell healthy from ill, or do they think that using complicated calculations just makes things look better?

It is both. Many modern physicians have no faith in their clinical skills, if what they think is not supported by " objective" evidence. I still find it hard to understand why some so called leading experts think that seeing (using their eyes) a graph showing the electrical activity of the muscle is more reliable and objective than seeing (using the same eyes) a patient unable to move and breath.
And yes, if I say that half of the patients responded to a certain treatment based on my clinical impression, it is not seen as important data, but if I say that 1/ 1000 patients had a statistically significant response, that is important evidence which justifies the use of this treatment in numerous patients.

 

[Esther12]

Peter White's earlier, more reasonable and published view on 'normal' SF36 scores

This from the 1997 evaluation of graded exercise for CFS, which found the mean SF-36 PF score improved from 48.5 at baseline to 69 after 12 weeks of treatment:


So here a Sf-36 PF score of 69 is not a return to normal, compared with the PACE threshold of 'normal' of just 60. Strange.

Thanks OB. There are a few quotes like this. I wish a kept all this PACE stuff filed away in a usable form. There's just too much relevant stuff!

 

[Graham]

In case you missed it on the other thread, there is a live link here (see the home page) to the federal advisory committee on chronic fatigue syndrome (CFSAC), which meets twice a year to propose recommendations and interview and prod federal officials to do more to meet the enormous needs of the chronic fatigue syndrome community.

Janelle has a 5 minute slot and will be talking about our project on the PACE study, and about Tom's harms paper. Cort (talking about PR), and Marly Silverman will also mention the project in their talks. Remember to add 5 hours to the times if you are in the UK. Janelle is due to speak somewhere between 3:30 and 4:30 (i.e. 8:30 and 9:30 GMT).