• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Measuring disability in patients with CFS: reliability and validity of the WSAS (PACE

Dolphin

Senior Member
Messages
17,567
Measuring disability in patients with chronic fatigue syndrome: reliability and validity of the Work and Social Adjustment Scale.

Journal of Psychosomatic Research xx (2011) xxxxxx

Matteo Cella, Michael Sharpe, and Trudie Chalder

Received 23 September 2010;
revised 2 February 2011;
accepted 24 February 2011.
Available online 3 April 2011.

Abstract

Background

Disability is a defining feature of chronic conditions, and it is an increasingly used measure of therapy effectiveness. The Work and Social Adjustment Scale (WSAS) is a simple and clear measure of disability. Although the scale is widely used, no study has yet investigated its psychometric properties in patients with chronic fatigue syndrome (CFS).

Methods

Data from two samples of patients were used, one from a multicenter randomized controlled clinical trial of treatments for CFS (n =639) and the other from a clinic that specializes in CFS (n=384). All patients completed the WSAS as well as other measures.

Results

Internal consistency and the SpearmanBrown split-half coefficient values indicated that the scale is reliable. CFS patients who had comorbid diagnoses of depression, anxiety or fibromyalgia had higher WSAS scores. High levels of disability were associated with high number of physical symptoms, severe fatigue, depression, anxiety, poor sleep quality and poor physical fitness, with correlation coefficients ranging between 0.41 and 0.11. Lower scores on the WSAS were modestly associated with better physical functioning as well as higher levels of physical capacity as assessed by a walking test. Sensitivity to change was evaluated in a subgroup of patients who had undergone a course of cognitive behavioral therapy. Disability significantly decreased after therapy and remained stable at follow-ups.

Conclusion

The WSAS is a reliable and valid assessment tool for disability in patients with CFS
Note, this study reports some PACE results - but it is only data from before the interventions so not so interesting.
 

Dolphin

Senior Member
Messages
17,567
List of WSAS questions

I always prefer to have a look at the questionnaires when I can to see what they are measuring:

(From full (unpublished) PACE Trial protocol)

Appendix 6: Case Report Forms
ISRCTN54285094
A6.34
Work and Social Adjustment Scale
Participant Initials Date completed

Centre Participant Fore. Midd. Sur. Day Month Year

Please read each of the following questions and tick the appropriate box to indicate

No impairment Very Severe Impairment

0 1 2 3 4 5 6 7 8

Because of my CFS/ME, my ability to work is
impaired.

Because of my CFS/ME, my home
management (cleaning, tidying, shopping,
cooking, looking after home or children,
paying bills) is impaired.

Because of my CFS/ME, my social leisure
activities (with other people, such as parties,
bars, clubs, outings, visits, dating, home
entertainment) are impaired.

Because of my CFS/ME, my private leisure
activities (done alone, such as reading,
gardening, collecting, sewing, walking alone)
are impaired.

Because of my CFS/ME, my ability to form
and maintain close relationships with others,
including those I live with, is impaired.
 

Dolphin

Senior Member
Messages
17,567
Correlations before treatment may not be the safe as correlations after treatment

Firstly, I think a problem is that no truly objective outcome measures are used.

I think better data would have been got by comparing the WSAS scores with:
The Client Service Receipt Inventory (CSRI), adapted for use in CFS/ME [31], will measure hours of employment/study, wages and benefits received, allowing another more objective measure of function.
which is an outcome measure in the PACE Trial (but wasn't reported in the Lancet paper), so they could have done the comparison either before and/or after treatment.

Similarly they do have actometer data before treatment in the PACE Trial but this was not reported.

The 6 minute walking test and 2 minute step test have some value. But they could be influenced by how well you think you are rather than how well you actually are.

I think a problem with this study is that no correlations are reported for after treatment.

We are supposed to assume that the same correlations will apply; however, they may not - there may be response biases (including specific to a therapy like CBT) that cause people to answer certain questionnaires differently e.g. to please the therapist. Maybe they would all still correlate together as they are all somewhat fairly subjective, or maybe they would not.

The accuracy of a questionnaire after an intervention is at least equally, if not more, important.

ETA: They explicitly say:
The short and the uncomplicated nature of the scale
makes the WSAS an excellent and quick assessment tool for
the degree to which people with CFS can participate in life,
an aspect of disability. Its use is recommended both for
assessment and for treatment evaluation
.

By the way, I think there is a good chance that this paper was written with the PACE Trial results in mind, just as I think
Measuring fatigue in clinical and community settings.
Cella M, Chalder T.
J Psychosom Res. 2010 Jul;69(1):17-22. Epub 2009 Dec 11.
was published with the PACE Trial results in mind (it actually used data from the mid-1990s); people will recall that it was used to give the mean and standard deviation for the Chalder Fatigue scale for the general population (based on people who had attended GPs in the previous year i.e. it excluded a lot of the healthier people) when the protocol paper already quoted previous research by Trudie Chalder et al which gave a threshold for "fatigue casesness" - a definition which most likely didn't give the results they wanted (it is quite strict while the new one is much looser).
 
Messages
13,774
Thanks for looking through this Dolphin. From my quick look at the paper, I couldn't see any interesting data that I wanted to dig into. I wish we could access the raw data.

edit: I totally agree about the possibility of different treatments, particularly CBT, having an impact upon the way these different outcome measure correlate with one another. But there's nothing in this paper that would let us assess whether that's the case.
 

Dolphin

Senior Member
Messages
17,567
The results in Figure 1 might look less impressive if we had all the data

I thought I'd highlight the following as people might not be aware of it if they just flick through the paper.

There are two cohorts used, the PACE Trial participants and another group:
Cohort 2: CFS patients attending a CFS specialist unit
Data from 384 patients assessed in a UK secondary care
specialist clinic were used to form cohort 2. These patients
were initially referred from primary or secondary care and
assessed by a consultant psychiatrist or senior therapist who
confirmed the diagnosis of CFS. Patients included had a
minimum age of 18 years at intake and a diagnosis of CFS
according to the Oxford criteria [19]. Patients with severe
psychiatric comorbidity (e.g., severe depression) are generally
not treated by the CFS clinic; nevertheless, a number of
comorbid psychiatric problems such as moderate depression
and anxiety are frequently observed in CFS patients. All
patients in cohort 2 received a course of CBT from an
experienced clinical psychologist or nurse. Data collection
for cohort 2 was conducted as part of the clinic audit and
service evaluation.

Patients in cohort 2 received between 10 and 15 sessions
of individual CBT and completed the WSAS and the Chalder
fatigue scale (CFQ) both before and after therapy and again
at 6- and 12-month follow-up after discharge from treatment.
A number of cohort 2 patients had missing follow-up
measures. One hundred sixty-one did not complete the
WSAS after treatment, 174 did not complete the measure at
the first follow-up and 159 did not complete it at the second
follow-up.
Reasons for not completing the measure were as
follows: poor compliance and delay in returning the
measures (e.g., more than 2 months from the due date).
Only 114 patients completed the WSAS at 4 time points.

They say:
Preliminary analysis conducted on baseline levels of WSAS,
fatigue and basic demographic variables (i.e., age and
gender) did not reveal any significant difference between
patients who complete the measures at all assessment points
and those who did not.
However, those aren't the only variables they could have checked for - Table 1 shows they had various other variables - I think if they should have simply listed them all if it were the case they did check them. Also, having similar baseline scores doesn't mean they all responded to therapy in a similar way.
 

Dolphin

Senior Member
Messages
17,567
Hyping CBT and GET:
However, occupational outcomes tend
to improve substantially for CFS patients who receive
treatment such as cognitive behavioral therapy and graded
exercise therapy [6].

Rimes KA, Chalder T. Treatments for chronic fatigue syndrome.
Occup Med (Lond) 2005;55:328.

(for 2005 paper)
Department of Psychological Medicine, Institute of Psychiatry, London, UK.

Correspondence to: Dr K. A. Rimes, Academic Department of Psychological
Medicine, Institute of Psychiatry and GKT School of Medicine, Weston
Education Centre, 10 Cutcombe Road, London SE5 9RJ, UK.
e-mail: k.rimes@iop.kcl.ac.uk
I can't remember that paper 2005 off the top of my head but most studies haven't published this data. Indeed, the famous Sharpe et al. (1996) CBT study in the BMJ was supposed to publish the data but they didn't - funny that.
 

Dolphin

Senior Member
Messages
17,567
They sneak in points that don't seem to be proven:
Disability was not found to differ between genders but
was found to be higher in older patients. This is perhaps
unsurprising as self-rated disability has been shown to
increase with age [37]. However, mean differences in the
disability scores between the age classes were not very high,
and the correlation showed a significant but very modest
trend in the expected direction. What is perhaps more
interesting is that levels of disability were comparable
between persons of different marital status, suggesting that
the potential support from partners did not have an effect on
self-reported levels of disability. Previous research has
shown that oversolicitous behavior is associated with
worse disability in CFS.
[38,39]
Simply having a partner (or not as the case may be) tells one very little about "oversolicitous behaviour".
 

Dolphin

Senior Member
Messages
17,567
The WSAS correlated most strong with depression in the (untreated) CFS patients

I think the authors may be correct that the WSAS may partly measure, in some people anyway, depression and anxiety. However, as predicted, I don't particularly like the way they make the point (in particular the underlined part).

Construct validity as assessed with correlations and
variance analysis showed moderate but statistically
significant values. The correlation coefficients, although
modest, pointed to a stronger association between higher
levels of disability with higher depression and fatigue scores
and lower scores on the SF-36 physical functioning scale.
These associations in particular may be relevant to the
construct validity of the WSAS as both depression and
fatigue are constructs that per se are associated with
increased disability. Higher levels of disability may be an
indication that clinicians should consider screening for
anxiety and depression. Other associations were rather weak
but in the expected direction. Associations between objective
tests of physical fitness, such as the walking and the step test,
although significant, were small. This may be indicative of a
degree of dissociation between physical capacity and
patient's perception of their disability levels. Fear of
exercising and overly cautious attitudes to physical activity
is a common thinking style observed in individuals with CFS
[38]. An alternative explanation for the low correlation
observed between physical capacity and WSAS scores may
be in the nature of the disability the scale measures. It is
possible that physical capacity, as assessed by the walking
and step test, is not associated with WSAS items such as
ability to work, as most jobs would not require high levels of
fitness to be performed. Similarly, leisure and social activity
may not be associated, for some people, with physical
fitness. Also, the higher values of disability observed in
those with a concomitant diagnosis of depression and anxiety
may suggest that the disability concept as measured by the
WSAS may depart from the more traditional physically
related concept of disability. The higher correlations with
depression, in particular, hint at the possibility that the
WSAS scale may measure a slightly different concept of
disability more associated with the capacity to enjoy and
participate in life. Therefore, the use of the WSAS may be
more appropriate for those investigations with a specific
target on a concept of disability less reliant strictly on
physical capacity and accounting more for the capacity to
be active.

38. Deale A, Chalder T, Wessely S. Illness beliefs and treatment outcome
in chronic fatigue syndrome. J Psychosom Res 1998;45:7783.

Fear of exercising and overly cautious attitudes to physical activity
is a common thinking style observed in individuals with CFS
I think a "cautious attitude to physical activity" would have made the basic point.
-----
On the correlation with depression, etc, they say in the introduction:
The scale was first introduced by Marks [9], but the first study
evaluating its psychometric proprieties in psychiatric groups
[i.e., obsessive-compulsive disorder (OCD) and depression]
was conducted by Mundt et al. [10]. These authors found that
the WSAS had good internal consistency (i.e., Cronbach's ?
ranging between .70 and .90) and that it positively correlated
with depressive symptom severity in depressed individuals
as well as with obsessive-compulsive symptom severity in
a group of patients with OCD.
The corollary of this is that improvements may represent improvements in depression.
 
Messages
13,774
I noticed the over solicitous thing as well, and seemed to remember the previous evidence in support of this being amusingly weak.

You might know this better than I, but from what I remember, the emergence of the idea that 'over solicitous' relationships were bad for outcomes came from a study in which they predicted that those who were less happy in their relationships would be more at risk of CFS (stress, depression, etc), and then actually found that it was very marginally the opposite. So they then came up with a new psychosocial story (as one always can).

They didn't seem terribly concerned about the psychosocial harm that may be done to carers who then had to worry that their attempts to support their loved one were actually perpetuating their illness. That's a psychosocial strain!
 

Dolphin

Senior Member
Messages
17,567
What data were we not shown?

As I pointed out above, we are not given correlations with some measures we know were used in the PACE Trial. One reason for this is that some may not have been measured in the other cohort. However, as with a lot of these studies, they may decide not to show us some data that doesn't suit them.
 

Hope123

Senior Member
Messages
1,266
Dolphin, I looked at this scale briefly before and believed I e-mailed you on this prior but the improvement on the WSAS scores in PACE post-intervention was on the level of something like 3 points (don't remember exactly) and some past studies, done in other conditions, seem to indicate than a 7-10 point difference (or some number around there) is needed to mean something significant.

This is somewhere in the deep recesses of my mind but I also recall that WSAS was primarily developed/ validated/ used in psychiatric disorders rather than other disorders so the question might be how valid is it for ME/CFS.

Haven't read this paper yet.
 

Dolphin

Senior Member
Messages
17,567
Dolphin, I looked at this scale briefly before and believed I e-mailed you on this prior but the improvement on the WSAS scores in PACE post-intervention was on the level of something like 3 points (don't remember exactly) and some past studies, done in other conditions, seem to indicate than a 7-10 point difference (or some number around there) is needed to mean something significant.

This is somewhere in the deep recesses of my mind but I also recall that WSAS was primarily developed/ validated/ used in psychiatric disorders rather than other disorders so the question might be how valid is it for ME/CFS.

Haven't read this paper yet.
Yes, you did send me something before - thanks. Here it is:
Clinical signficance with the 6-minute Walk Test has been discussed with the changes seen as pretty minimal. I've looked a bit at the work/ social adjustment scale but can't find a paper talking about what is considered a clinically signficant change. One paper about the WSAS and its repsonsiveness to true clinical change indicated 9-12 points as indicating signficant change; this is more than the 6-7 points with CBT/GET. The WSAS has mostly been used with psychiatric illnesses.The scale though was not compared to people's work/ leisure hours or asking family, etc. about whether self-report changes were equivalent to objective changes.

http://bjp.rcpsych.org/cgi/content/abstract/180/5/461
Also, the SMC group went up by 3 points so, when adjustments were made to baseline scores, CBT was 3.6 points better than SMC and GET was 3.2 points better. I think it's easier to criticise those sorts of figures as perhaps not being of much significance (especially if one factors in any response biases, and perhaps improvements in depression, etc.).

I wouldn't say this is the most exciting paper in the world to read. I just can read these sorts of papers relatively easily now as I know the questionnaires, etc quite well so like to read them.
 

oceanblue

Guest
Messages
1,383
Location
UK
I noticed the over solicitous thing as well, and seemed to remember the previous evidence in support of this being amusingly weak.

You might know this better than I, but from what I remember, the emergence of the idea that 'over solicitous' relationships were bad for outcomes came from a study in which they predicted that those who were less happy in their relationships would be more at risk of CFS (stress, depression, etc), and then actually found that it was very marginally the opposite. So they then came up with a new psychosocial story (as one always can).

They didn't seem terribly concerned about the psychosocial harm that may be done to carers who then had to worry that their attempts to support their loved one were actually perpetuating their illness. That's a psychosocial strain!
True, but worth noting that they are now claiming the WSAS data refutes this bunkum, so we can quote this if the 'loved ones being too supportive' theory is used again.
 

oceanblue

Guest
Messages
1,383
Location
UK
Now some doubt over relationship between reported disability and physical capacity:

Associations between objective
tests of physical fitness, such as the walking and the step test,
although significant, were small. This may be indicative of a
degree of dissociation between physical capacity and
patient's perception of their disability levels.
shame they didn't discuss this in the PACE paper.
 
Messages
13,774
Now some doubt over relationship between reported disability and physical capacity:


shame they didn't discuss this in the PACE paper.

Or go in to detail here! Have I missed the data for this point? On my read through, I couldn't see any figures that let us know exactly what was being referred too (I am feeling particularly dim at the moment though).
 

Dolphin

Senior Member
Messages
17,567
I noticed the over solicitous thing as well, and seemed to remember the previous evidence in support of this being amusingly weak.

You might know this better than I, but from what I remember, the emergence of the idea that 'over solicitous' relationships were bad for outcomes came from a study in which they predicted that those who were less happy in their relationships would be more at risk of CFS (stress, depression, etc), and then actually found that it was very marginally the opposite. So they then came up with a new psychosocial story (as one always can).

They didn't seem terribly concerned about the psychosocial harm that may be done to carers who then had to worry that their attempts to support their loved one were actually perpetuating their illness. That's a psychosocial strain!
Yes, interesting point about strain on partner.
I don't recall off-hand what research they might be referring to; they reference 38 & 39:
[38] Deale A, Chalder T, Wessely S. Illness beliefs and treatment outcome
in chronic fatigue syndrome. J Psychosom Res 1998;45:7783.
[39] Romano JM, Jensen MP, Schmaling KB, Hops H, Buchwald DS.
Illness behaviors in patients with unexplained chronic fatigue are
associated with significant other responses. J Behav Med 2009;32:
55869.
I imagine the data could be interpreted in other ways.
 

oceanblue

Guest
Messages
1,383
Location
UK
Or go in to detail here! Have I missed the data for this point? On my read through, I couldn't see any figures that let us know exactly what was being referred too (I am feeling particularly dim at the moment though).
Table 1 shows only a weak correlation between the subjective WSAS score and the objective 6MWT (r=-0.261) and Step test(r=0.128). As both should be related to true underlying disability/physical capability, you would expect a bigger correlation than this, hence their suggestion that reported disability might not be such an accurate measure of actual disability.
 
Messages
13,774
I don't recall off-hand what research they might be referring to; they reference 38 & 39:

I imagine the data could be interpreted in other ways.

Hmmm... they look later than the paper I was thinking of. It could have been as late as 98, but I didn't think the initial paper involved Wessely. I read it long before I was keeping notes on this stuff though (and I've recently lost most of my notes!). Maybe I've just started a CFS myth!
 
Messages
13,774
Table 1 shows only a weak correlation between the subjective WSAS score and the objective 6MWT (r=-0.261) and Step test(r=0.128). As both should be related to true underlying disability/physical capability, you would expect a bigger correlation than this, hence their suggestion that reported disability might not be such an accurate measure of actual disability.


I'm a bit shoddy on statistics stuff, so don't really understand how these things are calculated, or having much understanding of their significance. Dolphin suggested 'The Cartoon Guide to Statistics' as an appropriate text for my level of ignorance, but I've yet to get around to it.

I should really take the time to re-read the paper properly, and do some googling. I didn't take much in from it first time around. I'm off to read the wikipedia page on the 'Pearson product-moment correlation coefficient'.