(UK) (Propaganda!?) Report from CCRNC for APPG inquiry into patient services (2009)

[Dolphin]

(I figured this was the best section for this)

Report from CCRNC for APPG inquiry into patient services

Executive Summary is signed by Esther Crawley, Chair, CRNCC
http://bit.ly/gfpiFO i.e. http://www.afme.org.uk/res/img/resources/CCRNC report to the APPG Inquiry 2009.pdf

Although it is portrayed as an impartial report, I don't think it can be seen as such:

The section, "Quantitative patient satisfaction and outcome data from routine surveys", contains all sorts of data - a small bit from each service.
It is difficult to believe that this is the only data available.
Maybe there is a reason for it, but we are not told why there is no quantitative data for services 8, 14, 15 & 17 (there is qualitative data for these services)

Also, for the section, "Qualitative accounts of patient experiences of Specialist CFS/ME services", it seems like we are only getting some of the data or if not, they didn't ask the best questions to get all sorts of feedback
e.g.
Service 10: Results from patient survey: "What was most helpful about the service you have received?"
Service 14: Service evaluation: "what is really good about your care?"
Maybe there is a reason for it, but we are not told why there is no qualitative data for services 1, 2, 11, 12 & 16 (there is quantitative data for these services)

So I think the claims (copied below) that this is better quality data than data from patient surveys are dubious.

---------

Some notes I made for myself (I don't claim they represent the whole report - they don't - they would be unrepresentative although the same question wasn't generally asked):- Service 13 (paeds) seems like Esther Crawley's service (coloured charts) -anyone know/have an educated guess at other ones?

----

Comment on Service 3: "I hope there is continued funding and eventually have clinics everywhere" (p. 19)

Vs

Comment on Service 5: "I hope there is continued funding and eventually have clinics everywhere" (p. 20)

----

(Not exciting) Made worse: two I noticed:
(i) 10% (n=5) (Service 6, p.10)
(ii) 11% (Service 11, p.13)

----

(Service 16) Level of goal achievement: Patients are asked what percentage of their goal has been achieved (p.14)
Fully (around 100%): 16 (39.02%)
Mainly (around 80%): 12 (29.27%)
Partly (around 40%): 8 (19.51%)
Only slightly (around 20%): 5 (12.20%)
Not (0%): 0

(I measured these) (n=41)

----

(This is probably the most interesting and complete data)

Service 16: 12 Month Outcome Data

Initial Follow up
Average hours of employment per week: 5.5 8.9
Average SF36 Function Scale scores (Range 100 - 0): 39.8 44.6
Average Pain VAS Scores: 38.2 34.0
Mean Chalder Physical Fatigue Scale score (Range 0 - 7): 6.4 3.4
Mean Chalder Mental Fatigue Scale score (Range 0 - 4): 3.5 2.1
Mean Chalder Fatigue combined score (Range 0 - 11): 6.9 5.5
Average Hospital Anxiety Scale score: 10.4 8.9
Average Hospital Depression Scale score: 10.1 7.8

8.9 hours a week on average (3.4) at 12 months still isn't a huge figure
especially as many aren't that ill and some are probably glandular fever and
other post-infectious cases in the other stages who would have made a good
improvement in that time:

The mean Chalder Fatigue Score going in was 6.9.
This is a scale where most people with moderate or severe ME score 10 or 11
out of 11 - see Figure 1 at:
http://bit.ly/g7Tvup i.e. http://www.iacfsme.org/BULLETINFALL2008/Fall08GoudsmitFatigueinMyalgicEnceph/tabid/292/Default.aspx

That study found for the different types of ME, the average Chalder Fatigue
Score (bimodal) was:
Severe (n=11): 10.82
Moderate (n=10): 9.42
Mild (n=3): 7.67
So even the mild ME group had worse Chalder fatigue scores than the group
attending service 16.

----------------------------------------------------------

Why is the data described in this report so different to those described in some patient charity member surveys?

Both the quantitative and the qualitative data presented here describe very high levels (much higher than normal NHS surveys) of patient satisfaction, improved outcomes and positive experiences with specialist CFS/ME services. This is inconsistent with some patient charity member surveys.

The most likely explanations for these differences are due to bias and methodological differences. Bias occurs when particular opinions are obtained due to ascertainment processes in surveys which increase the chances of certain views being over represented. Bias is reduced if opinions are systematically collected across the whole sample of interest with a high response rate. The data represented here has been systematically collected annually across many services and represents a high response rate.

The differences between the patient experience represented here and that seen in patient charity member surveys could be due to several sources of bias. Patients who access the services but do not improve may be more likely to respond to patient charity surveys. Secondly, patients responding to patient charity surveys may not have had the treatment offered by specialist services or may be different in other ways, for example being ill for longer. Those responding to charity surveys may not have been diagnosed by a doctor as having CFS/ME, but may be self-diagnosed. Patients may respond to patient charity surveys about the services described even when they have chosen not to attend. The corollary is that those responding to patient service surveys may also be biased in that they may be more likely to respond positively. However this is less likely because of the high percentage of patients that responded, the anonymised collection of data, and the consistent outcome in data collection across many teams.

 

[Esther12]

They say:

The absence of complaints on treatment approaches is striking.

I can only agree.

I wasn't asked to fill in any questionnaire. I wonder how they choose which patients to ask?

Surely they should have data for 'no of patients seen' vs 'no of questionnaires submitted'. I couldn't see it though, and some of the figure's reported looked really low considering the time span involved.

I'm still open to the possibility that I just saw a terrible pair of therapists, and everyone else is wonderfully happy with the approach taken by these CFS centers, but I'm not going to have faith in them when they could so easily provide more compelling data if they really had it.

Are they claiming that they can cure CFS, or just teach us to accept being ill?

I am working on my aim which is to accept that I have CFS/ME and re-adjust my lifestyle
to a more acceptable level. Only doing what is most important to me and the family, learn
to say not to things and people when I know this makes the symptoms worse.

Learning to pace, relax and time management. Meeting people in the same situation.
Coming to terms with this illness and my own limitations. Getting more control with the
illness.

I do not feel alone with this terrible condition. The advice given to me on filling in the benefit
forms meant I didnt have to go to an appeal

Some of the comments imply that they're being encouraged to adopt a sickness role! I know some CFS clinics reject the CBT/GET appraoch - it would be interesting to see how different clinics compared against one another.

Also, while it may look impressive that a high percent of patients feel they achieved their long-term goals, it needs to be seen in context of what those goals were:

eg: Drive to do the shopping (long-term goal).

I don't want to do down this patient's achievement, and if the psycho-socialists were suitably modest in their claims about their ability to understand and treat CFS I'd applaud whatever help they were able to offer... but these sorts of results do not justify the dominance of research spending they have received or the sorts of claims they have been making about the nature of CFS.

Was that really meant to be an independent report? It read like a sales pitch - and that's fine, so long as they're honest about what they're doing.

 

[biophile]

Surely they should have data for 'no of patients seen' vs 'no of questionnaires submitted'. I couldn't see it though, and some of the figure's reported looked really low considering the time span involved.

Ah, so much for the "systematic collection" of information they kept going on about and the "bias" they claimed to compensate for.

 

[Dolphin]

Are they claiming that they can cure CFS, or just teach us to accept being ill?

I am working on my aim which is to accept that I have CFS/ME and re-adjust my lifestyle
to a more acceptable level. Only doing what is most important to me and the family, learn
to say not to things and people when I know this makes the symptoms worse.

Learning to pace, relax and time management. Meeting people in the same situation.
Coming to terms with this illness and my own limitations. Getting more control with the
illness.

I do not feel alone with this terrible condition. The advice given to me on filling in the benefit
forms meant I didnt have to go to an appeal

Some of the comments imply that they're being encouraged to adopt a sickness role! I know some CFS clinics reject the CBT/GET appraoch - it would be interesting to see how different clinics compared against one another.

Yes, I have heard that a lot of the practitioners aren't sticking to the strict CBT/GET view of the world that some promote.

Also, while it may look impressive that a high percent of patients feel they achieved their long-term goals, it needs to be seen in context of what those goals were:

eg: Drive to do the shopping (long-term goal).

I don't want to do down this patient's achievement, and if the psycho-socialists were suitably modest in their claims about their ability to understand and treat CFS I'd applaud whatever help they were able to offer... but these sorts of results do not justify the dominance of research spending they have received or the sorts of claims they have been making about the nature of CFS.

Yes, this is the issue. CBT/GET is sometimes portrayed as all that patients require - that the model explains everything that is happening with patients. Results on the ground suggest that it tends to lead to few cases of full recovery which one might expect if the model was good as it is presented.

Was that really meant to be an independent report? It read like a sales pitch - and that's fine, so long as they're honest about what they're doing.

I'm not sure if you read the last bit I quoted but they try to claim it's impartial and much better data than data patient group's have collected.

 

[Dolphin]

The APPG service review's questionnaire is at: http://worcsmegroup.weebly.com/uplo...09-090718-pct_me_survey_finalrecdfromsg-s.pdf

Although my guess is that a lot of the data wasn't collected from patients for this specifically but for the minimum data set, etc.

 

[Esther12]

I'm not sure if you read the last bit I quoted but they try to claim it's impartial and much better data than data patient group's have collected.

I understand them thinking that their survey is better than charity surveys... but all are pretty easily distorted.

Surely this reads like a transparent PR exercise rather than a serious attempt to judge critically whether these services are a worthwhile use of money. If they wanted to look independent, they've not done a good job imo.

If you see this as them putting their best face foreward, and trying to get/maintain their funding, then the weakness of the case they present seems pretty condemning.

The 'time spend in paid employment data' looked pretty poor. If these centres are dealing with a representative sample of Oxford Criteria CFS patients, that sort of improvement sounds similar to what you'd expect from standard care (I really wish they included some meaninful comparison figures... the fact they didn't will surely mean anyone reading the report will be less willing to give them the benefit of the doubt). The whole thing seems really flimsy to me, but maybe MPs/decision makers fall for this sort of stuff.

 

[Dolphin]

I understand them thinking that their survey is better than charity surveys... but all are pretty easily distorted.

Surely this reads like a transparent PR exercise rather than a serious attempt to judge critically whether these services are a worthwhile use of money. If they wanted to look independent, they've not done a good job imo.

If you see this as them putting their best face foreward, and trying to get/maintain their funding, then the weakness of the case they present seems pretty condemning.

The 'time spend in paid employment data' looked pretty poor. If these centres are dealing with a representative sample of Oxford Criteria CFS patients, that sort of improvement sounds similar to what you'd expect from standard care (I really wish they included some meaninful comparison figures... the fact they didn't will surely mean anyone reading the report will be less willing to give them the benefit of the doubt). The whole thing seems really flimsy to me, but maybe MPs/decision makers fall for this sort of stuff.

Good points.

Patient surveys can have problems. But I think the MEA survey http://www.meassociation.org.uk/?page_id=1345 http://www.meassociation.org.uk/wp-content/uploads/2010/09/2010-survey-report-lo-res10.pdf and indeed the AfME/AYME survey http://afme.wordpress.com/ are presented in a much more thorough/transparent manner than this survey.

A large fundraising event raised money for Action for M.E. and the organisers wanted some of it to go to research. Action for M.E. looked at various projects and decided to fund research into setting up a tissue bank (very worthy) and a national outcomes database

National outcomes database
Dr Esther Crawley and colleagues from the University of Bristol, University of Nottingham and Bart's Hospital, London will set up a National Outcomes Database which will allow the 46 clinical teams that are part of the NHS CFS/M.E.

http://www.afme.org.uk/news.asp?newsid=369

Esther Crawley, one will recall, is the person who put her name to this report.

She previously condemned the AFME/AYME 2008 survey:
http://meagenda.wordpress.com/2008/05/16/dr-esther-crawley-condemns-afme-survey-as-unreliable/

But while Action for ME accepted the treatment could help some, it said that 34 per cent of ME patients in its survey said graded exercise therapy had made their ME worse.

However Dr Esther Crawley, an ME specialist based at the Royal National Hospital for Rheumatic Diseases in Bath and who helped to draw up the Nice guidelines, dismissed the findings, saying the survey was unreliable.

“This survey is based on a biased sample of people who have had an issue with treatment and we cannot deduce who had graded exercise therapy delivered by a specialist, as Nice recommends,” she said.

“Delivered by a specialist, it actually reduces activity to a stable level and then to a small amount each day. We have had above 85 per cent of people who said they have been happy with their treatment.

“We do not say to people they have to have the therapy, we give them a range of options and they choose.”

I have little confidence in how she will present data that she will collect. I also have little confidence in Bart's Hospital and how they might deal with the data (see my sig for that service's comments on the draft NICE guidelines). I think Nottingham may be a biased place also.

I think there needs to be an independent audit of the English/UK services, like happened in Belgium http://forums.aboutmecfs.org/showth...f-Belgian-CBT-GET-rehab-clinics-(large-sample) , where the results were not very impressive (they used Fukuda criteria and amount of hours worked actually decreased!).

 

[Esther12]

I think there needs to be an independent audit of the English/UK services, like happened in Belgium http://forums.aboutmecfs.org/showth...f-Belgian-CBT-GET-rehab-clinics-(large-sample) , where the results were not very impressive (they used Fukuda criteria and amount of hours worked actually decreased!).

Absolutely. When you have the proponents of a treatment assessing it's efficacy, you have a recipe for quackery and dishonesty. I find it hard to blame them for it... if I was marking my own work I'd spin it as positively as possible to - especially with cutbacks due.

(I'm having PC trouble, so haven't looked at those links you provided. Thanks for them though.)

Also - was this the report that took ages to come out because of the extensive analysis of results they wanted to do? Did I miss the clever and enlightening statistical analysis they'd put all that work in to?

 

[Dolphin]

Absolutely. When you have the proponents of a treatment assessing it's efficacy, you have a recipe for quackery and dishonesty. I find it hard to blame them for it... if I was marking my own work I'd spin it as positively as possible to - especially with cutbacks due.

Which shows one might not have that much difficulty convincing an impartial person that an independent audit would be more accurate.

Also - was this the report that took ages to come out because of the extensive analysis of results they wanted to do? Did I miss the clever and enlightening statistical analysis they'd put all that work in to?

Are you thinking of the (5m) PACE Trial (which was approved for funding in 2003 and where the results were expected in the second half of 2010)?

 

[Esther12]

Which shows one might not have that much difficulty convincing an impartial person that an independent audit would be more accurate.

I get the impression that no-one impartial is interested in CFS one way or the other.

There's that NHS consultation happening now. My PC trouble's prevented me looking at it today, but if I'm able to get on-line again tomorrow I've been planning to submit stuff. I expect it will all be ignored, but may be worth a go.

Are you thinking of the (5m) PACE Trial (which was approved for funding in 2003 and where the results were expected in the second half of 2010)?

No. I semi-remember reading Esther (other esther) talking about releasing follow up information from patients of these centers, and how complicated and difficult it will be to compile and analyse the results. I was expecting more. I've no idea where I read it (possibly on here?), and could well have got the wrong end of the stick, but it wasn't with PACE.

 

[Dolphin]

I get the impression that no-one impartial is interested in CFS one way or the other.

Maybe so.
I meant somebody contracted to audit.
But there might be a bias as it's an NHS employee.
With the Belgian clinics, an outside body (the state health insurance company - think that would be a description that might give an idea what it was - don't full understand the system) set the outcome measures. So there were things like exercise testing and hours worked rather than simply questionnaires. I heard something like that anyway, that it wasn't the clinics who set the outcome measures, etc (nor did they write the report, I think).

No. I thought I remember reading Esther (other esther) talking about releasing follow up information from patients of these centres, and complicated and difficult it will be to compile the results. I was expecting more. I've no idea where I read it (possibly on here?), and could well have got the wrong end of the stick, but it wasn't with PACE.

They have been collecting the "Minimum Data Set" for years. Data from the services has been promised at some stage. Maybe there was a delay - not sure I heard.

 

[Esther12]

With the Belgian clinics, an outside body (the state health insurance company - think that would be a description that might give an idea what it was - don't full understand the system) set the outcome measures. So there were things like exercise testing and hours worked rather than simply questionnaires. I heard something like that anyway, that it wasn't the clinics who set the outcome measures, etc (nor did they write the report, I think).

That sounds the way to do it. Especially if the practitioners don't know what measures are being used. I think I could run a clinic for CFS that could lead to significant improvements in any particular measure, just by encouraging patients to prioritise that over other aspects of their life. Very often, that seems to be good enough. Even an independent auditor could slip up if they didn't know what they were doing, or failed to think about the claims being made critically enough.

I should re-read that Belgian stuff. I read it a while back, and have the thread saved to my favourites to go back over.

 

[Dolphin]

That sounds the way to do it. Especially if the practitioners don't know what measures are being used. I think I could run a clinic for CFS that could lead to significant improvements in any particular measure, just by encouraging patients to prioritise that over other aspects of their life. Very often, that seems to be good enough. Even an independent auditor could slip up if they didn't know what they were doing, or failed to think about the claims being made critically enough.

I think your point about training for particular measures is a good one.
However, I should say, that I don't think with the Belgian clinics the outcome measures were hidden from the people running the clinics, just that the people running the clinics didn't get to pick them (or some of them anyway, or at least, not without discussion - I'm not 100% sure which as I'm going on second/third-hand information).