PACE Trial follow-up: Here's the table looking at the effects of having CBT or GET after 52 weeks

[Dolphin]

@Simon, maybe you could also post that to the thread on the paper: http://forums.phoenixrising.me/inde...me-long-term-follow-up-from-pace-trial.40817/

 

[elliepeabody]

One of the points raised by Coyne is that the follow up data is uninterpretable because it's no longer gathered in a randomized clinical trial.
For example, we can see patients who were assigned to SMC but didn't do CBT and GET had lower fatigue scores than those who did, but we don't know what this means because things are no longer randomized.

So does that mean for the data to have any validity the patients would have had to be randomly assigned to the three groups (no CBT / GET, 1-9 sessions of CBT / GET, or more than 10 sessions of CBT / GET)?

Many thanks to @wdb and @Simon for the graphs, combination of colour and bar charts does make the data a lot easier to understand.

One of the interesting things about the numbers in each group is by far the largest group is the no further CBT / GET. If there was a complex negotiation between the researchers and the patients it appears that patients voted with their feet and the majority refused to participate in further CBT / GET despite the 'good news' newsletters, wanting to please their therapist and also I suspect some pressure to assist further with the research.

Is the data in the 1-9 sessions not significant because it is too big a range of sessions to determine effect? Why would this group show a bigger improvement than the no CBT / GET group?

 

[Simon]

Things I do to keep @Dolphin happy...

Combining the 1-9 and 10+ groups together gives a comparison of means for 'no treatment' vs 'any treatment', below
Slight advantage for no treatment for fatigue scores, slightly bigger advantage for treatment for physical function. But neither will be statistically significant (because of wide confidence intervals on 'no treatment' group alone).

Added
Worth noting that roughly half of each group later had any CBT/GET (57% for APT group. 47% for SMC group).

For that additional treatment to explain why APT/SMC groups effectively caught up with the CBT/GET groups, the half that went on to have CBT/GET would have to do spectacularly well, with no change among those that didn't. In fact, there was effectively no difference between the two. This shows up the authors' wishful thinking that it was the wonder of CBT/GEt that enabled to have the APT/SMC groups to catch up by 2.5 years.

 

[Dolphin]

Things I do to keep @Dolphin happy...

Thanks.

If you or anyone else felt inclined, maybe the SMC and APT groups could be combined, using the appropriate weightings (i.e. the sample sizes). Just an idea.

 

[Valentijn]

If you or anyone else felt inclined, maybe the SMC and APT groups could be combined, using the appropriate weightings (i.e. the sample sizes). Just an idea.

Would you like a cappuccino and foot massage with that, sir?

 

[Snow Leopard]

Just to add that the participants in the Pace trial were majority women, mean age 38 years.

Not only that, they have systematically excluded other medical disorders when screening patients for the trial. This means that the true comparison group is not the "working age population", but the healthy group in the 35-45 age range. This group has a SF-36 PF mean of the low-mid 90s and an SD of ~10 at most. This is how the original threshold of 85 was derived.

My guess is they always planned on watering down the measures, even when writing the protocol.

 

[Snow Leopard]

For that additional treatment to explain why APT/SMC groups effectively caught up with the CBT/GET groups, the half that went on to have CBT/GET would have to do spectacularly well, with no change among those that didn't. In fact, there was effectively no difference between the two. This shows up the authors' wishful thinking that it was the wonder of CBT/GEt that enabled to have the APT/SMC groups to catch up by 2.5 years.

The APT group who had 1-9 sessions and SMC/APT who had 10 sessions had the lowest scores, so they had the most 'potential' for improvement. The SMC/APT group which received no additional treatment started with higher scores...

The key is though, if you didn't label any of the groups and just gave me the raw numbers, I'd claim that the data just shows a classic regression to the mean.

 

[Simon]

If you or anyone else felt inclined, maybe the SMC and APT groups could be combined, using the appropriate weightings (i.e. the sample sizes). Just an idea.

No trouble at all, Sir: see below - cappucino and foot massage are in the post, my apologies for omitting last time. (Actually, I'm just annoyed I didn't think of this graph myself )

Would appreciate anyone checking the numbers look sensible, and any suggestions for improving the presentation as I'd like to tweet this later.

Ideally I'd put the fatigue access on a secondary axis and reverse the scale (so both sets of bars go upwards), and scale the fatigue bigger (since it's 0-33 vs 0-100 for SF36). But I can't work out how to do this in Excel 2013. I'll add a note that no differences are statistically significant.

 

[Snow Leopard]

No trouble at all, Sir: see below - cappucino and foot massage are in the post, my apologies for omitting last time. (Actually, I'm just annoyed I didn't think of this graph myself )

View attachment 13324

Would appreciate anyone checking the numbers look sensible, and any suggestions for improving the presentation as I'd like to tweet this later.

Ideally I'd put the fatigue access on a secondary axis and reverse the scale (so both sets of bars go upwards), and scale the fatigue bigger (since it's 0-33 vs 0-100 for SF36). But I can't work out how to do this in Excel 2013. I'll add a note that no differences are statistically significant.

Calculate the p-values? (the difference doesn't look significant to me, but a weak p-value would confirm it.)

 

[Valentijn]

Would appreciate anyone checking the numbers look sensible, and any suggestions for improving the presentation as I'd like to tweet this later.

It'd be nice if the "None" bars were grey instead of white, since there's a white background.

I'd also like a cappuccino. Maple syrup, almond milk, and organic coffee, with cinnamon and dark chocolate sprinkles on top will suffice

 

[user9876]

I'd also like a cappuccino. Maple syrup, almond milk, and organic coffee, with cinnamon and dark chocolate sprinkles on top will suffice

That drink sounds worse than the PACE methodology. (but then I don't approve of milk in coffee!).

 

[Valentijn]

That drink sounds worse than the PACE methodology. (but then I don't approve of milk in coffee!).

I like a little coffee with my sweet beverage

 

[Dolphin]

No trouble at all, Sir: see below - cappucino and foot massage are in the post, my apologies for omitting last time. (Actually, I'm just annoyed I didn't think of this graph myself )

View attachment 13324

Would appreciate anyone checking the numbers look sensible, and any suggestions for improving the presentation as I'd like to tweet this later.

Ideally I'd put the fatigue access on a secondary axis and reverse the scale (so both sets of bars go upwards), and scale the fatigue bigger (since it's 0-33 vs 0-100 for SF36). But I can't work out how to do this in Excel 2013. I'll add a note that no differences are statistically significant.

Thank you.

In the Lancet paper (2011), they say:

A clinically useful difference between the means of the primary outcomes was defined as 0·5 of the SD of these measures at baseline,31 equating to 2 points for Chalder fatigue questionnaire and 8 points for short form-36. A secondary post-hoc analysis compared the proportions of participants who had improved between baseline and 52 weeks by 2 or more points of the Chalder fatigue questionnaire, 8 or more points of the short form-36, and improved on both.

So for the Chalder Fatigue scale, the no CBT or GET treatment group did 0.2* of a clinically useful difference better than the CBT/GET treatment group. (Or 0.1 SD).

And for the SF-36, the the CBT/GET treatment group did 0.14* (0.1375)* of a clinically useful difference better than no CBT or GET treatment group. (or 0.07 SD)



*There was probably some rounding with the original figures.

 

[Sean]

Ideally I'd put the fatigue access on a secondary axis and reverse the scale (so both sets of bars go upwards), and scale the fatigue bigger (since it's 0-33 vs 0-100 for SF36). But I can't work out how to do this in Excel 2013. I'll add a note that no differences are statistically significant.

I'd add an extra note to clarify that improvement is in opposite directions for each measure. One up, one down. Does sort of say that in the title, but needs to be more explicit, I think.

 

[wdb]

Would appreciate anyone checking the numbers look sensible, and any suggestions for improving the presentation as I'd like to tweet this later.

Was thinking, rather than present the difference with no context it might be better to show the before and after mean for the two groups (no post treatment and some post treatment) ,we have after and diff so easy to calculate the before. Or maybe show the difference as a percentage improvement rather than absolute values.

 

[Simon]

OK, so not as blindingly obvious as I'd hoped - clearly I've seen too many research papers in my life.

Calculate the p-values? (the difference doesn't look significant to me, but a weak p-value would confirm it.)

Sadly that's pretty-well impossible without the raw data, though it's clear from the small differences and wide confidence intervals it wouldn't be significant (even allowing for CIs being smaller in combined samples).

So for the Chalder Fatigue scale, the no CBT or GET treatment group did 0.2* of a clinically useful difference better than the CBT/GET treatment group. (Or 0.1 SD).

And for the SF-36, the the CBT/GET treatment group did 0.14* (0.1375)* of a clinically useful difference better than no CBT or GET treatment group. (or 0.07 SD)

That's probably the most practical approach. Here's my attempt at graphing that, just for fatigue.

Does this work any better, and/or any suggestions? Note I reversed the fatigue score axis so both it and physical function would point the same way, with higher as better.

I could put a dotted line for clinically useful difference instead, drawn at the height of the top of the 'clinically useful difference' bar i.e. -2.0 ahead of the 'Any CBT/GET' bar. (And maybe have a second dotted line at the top of that bar, with 'clincally useful difference' as the label on the top line.)

 

[Sean]

Nice.

The word "drop" in the title clashes with the direction of the graphic (upwards). Alter either:

- the title, e.g. 'Mean change in...', or

- the direction of the graphic, (invert top to bottom, with the bars, scale, and arrow falling). More intuitive?

Don't need the grey line connecting the 0.4 figure to its box, pretty obvious those two go together.

Put '(favours no treatment)' on a new line below Difference.

You can do a higher res version?

 

[Sean]

I could put a dotted line for clinically useful difference instead, drawn at the height of the top of the 'clinically useful difference' bar i.e. -2.0 ahead of the 'Any CBT/GET' bar. (And maybe have a second dotted line at the top of that bar, with 'clincally useful difference' as the label on the top line.)

Is that 0.4 'Difference' statistically significant? If not, I would ditch it.

The graphic comparison between the two groups says it all: CBT/GET offered no advantage.

Couldn't be starker.

Keep the CUD box.

 

[Simon]

Nice

Thanks! (But what do you have to do to get a 'like' round here )

Thanks for all the suggestions too.

- the direction of the graphic, (invert top to bottom, with the bars, scale, and arrow falling). More intuitive?

Yes, more intuitive, but confusing with Physical Function going up - and this is aimed at a wide audience, not just research geeks. Have changed title from 'fall' to 'change' as you are right about contradiction.

Is that 0.4 'Difference' statistically significant? If not, I would ditch it.

Almost certainly not, but can't quote p values and confirm as can't compute (see my previous post).

However, another option is to use lines, as below, to indicate CUD. I could then drop both the difference and the CUD bar (if I did that, I could probably do both fatigue and function on the same graph too). I know the lines are not exactly aligned.

You can do a higher res version?

Probably! Just copied and pasted from excel, guess if I made graph bigger in excel, it would make a higher res picture when copied.

 

[Sean]

(But what do you have to do to get a 'like' round here )

Why, just ask...