1. Patients launch a $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
ME/CFS and the Magic of the Canine Factor
There's been plenty of research indicating that having pets is good for your health. I never really noticed any particular benefits to having cats, though that may have had more to do with my cats. They've been fairly indifferent to my presence and we've shared a live-and-let-live...
Discuss the article on the Forums.

Goldsmith et al piece on PACE Trial inc. link to free txt (help sought to explain it)

Discussion in 'Latest ME/CFS Research' started by Dolphin, Dec 27, 2011.

  1. Dolphin

    Dolphin Senior Member

    Messages:
    6,439
    Likes:
    4,665
    I just happened to come across the following:


    It's free at: http://www.trialsjournal.com/content/12/S1/A144

    It's late here so on a first read I'm having difficulty understanding it. Perhaps together we can break it down.
  2. Snow Leopard

    Snow Leopard Senior Member

    Messages:
    2,203
    Likes:
    1,535
    Australia
    I don't really understand it either since the actual Oral presentation isn't provided.

    The conclusion is interesting. Apparently there were no instrumental variables (IVs) that were stong mediators of improvement. A maximum correlation coefficient of R=0.03 shows these variables are unlikely to have any (practical) significance whatsoever.
  3. Sean

    Sean Senior Member

    Messages:
    1,152
    Likes:
    1,663
    I am struggling to understand much of that one at all. Seriously beyond my math level.

    Though whatever it is they are measuring/calculating it doesn't look like the final numbers give the PACE crowd much comfort.

    Frankly, if they are not able to get a clear substantial result from basic statistical tests and analysis of the raw data, then it is unlikely they will get one from it using more advanced subtle methods. We are not talking about a 10 000 variable matrix here, like some gene analyses, where a 1% difference between variables can be important. A good solid result from PACE type trials should stand out without needing sophisticated stats work (or dubious manipulations of definitions and statistical thresh holds).

    How many ways can you fail to find a practically meaningful increase in distance walked or hours worked? If it ain't there, it just ain't there.
  4. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    342
    UK
    Good luck with this, I don't have the energy to look in detail but here are a few comments:

    What makes mediation analysis special is that it aims to establish causal relationships. It seems to be related to Structural Equation Modelling and Path Analysis, techniques covered in the final, most difficult section of my Biostatistics textbook (not a good sign) and I never got that far.

    The first few pages of "Mediation Analysis in Social Psychology: Current Practices and New Recommendations" might be helpful, and possibly this critique of Meditation analysis too.

    It might not be worth the effort, though, given the limited findings:
  5. Marco

    Marco Old blackguard

    Messages:
    1,136
    Likes:
    737
    Near Cognac, France
    Post the link on Bad Science and ask them to explain.

    I'm sure there are many clever people there.
    peggy-sue likes this.
  6. Marco

    Marco Old blackguard

    Messages:
    1,136
    Likes:
    737
    Near Cognac, France
    I don't honestly expect my previous post would accomplish anything useful.

    I've had a quick look around and it seems to me that the authors don't doubt that CBT and GET are effective but are trying to understand perhaps what the common intervening (mediating) variables might be.

    The following abstract is from a text for using instrumental variable analysis to explore why certain social policy instruments might achieve the hoped for social behaviours. By randomly assigning certain groups to with/without policy conditions you can tell that a policy works in changing the desired behaviour but you don't know why it works. Knowing why might help find more effective ways of intervening (that is the intervention effectiveness might be improved).

    In the case of PACE they believe that changing thought processes via CBT and GET (both emphasise that ME/CFS is benign and reversible) results in an improvement in the measured outcomes of fatigue and physical function. It may be that much of the improvement (sic) however is due to a mediating behaviour (e.g. getting out of the house more) so perhaps encouraging people to 'get out more' might be a more direct and cost effective treatment than costly CBT therapy or supervised GET.

    So they run a range on models in which they insert a mediating variable that they can measure and correlate with the input and output variables (roughly speaking) and any strong co-variance might suggest a likely mediating variable.

    They apparently didn't find any strong candidates which restricts their ability to make their CBT/GET routine more cost effective.

    That's my take anyway.

  7. Dolphin

    Dolphin Senior Member

    Messages:
    6,439
    Likes:
    4,665
    Thanks for input. Will read now.

    Just to say that I've spotted what I think is an error ...
    However, I don't reckon they'll be giving me a Nobel prize for thinking that one of the authors was called Peter White, not Paul White.
  8. Marco

    Marco Old blackguard

    Messages:
    1,136
    Likes:
    737
    Near Cognac, France
    Just to add that one behavioural variable I'm sure they didn't consider in their models is any social desirability bias when responding to subjective self rating questionnaires of fatigue and physical functioning following therapies that aim to encourage participants to think of their symptoms as trivial and transient and that they will recover.

    As I recall participants' ratings of therapists was generally high. I would expect this mediator to account for much more than 20% of the variance.
    peggy-sue likes this.
  9. Dolphin

    Dolphin Senior Member

    Messages:
    6,439
    Likes:
    4,665
    One statistic which makes me thinks CBT might have influenced what the participants reported is:
  10. Esther12

    Esther12 Senior Member

    Messages:
    5,088
    Likes:
    4,857
    I'm sure that they would have addressed such concerns... otherwise that could be just promoting a placebo dressed up in quackery!
    peggy-sue likes this.
  11. Dolphin

    Dolphin Senior Member

    Messages:
    6,439
    Likes:
    4,665
    This is how one person has interpreted the findings. They said it could be re-posted:

  12. Sean

    Sean Senior Member

    Messages:
    1,152
    Likes:
    1,663
    We have been deliberately, persistently, systematically and very effectively disempowered, marginalised and stigmatised, for decades, almost certainly by the very people who are giving that advice, and who hold virtually all of the important advisory positions in this field. Yet they try to make out we are the ones in the driving seat, with massive amounts of undue influence?

    Who do the insurance companies, governments, media, etc, go to for advice on ME/CFS? Not us patients.

    A genuinely Orwellian use of language.
    justinreilly and oceanblue like this.
  13. Dolphin

    Dolphin Senior Member

    Messages:
    6,439
    Likes:
    4,665
    Note: I wasn't concentrating and didn't post the first part of the last post, which was the part I wanted to highlight. Also the reason for the rest of it might not have been clear without it. Anyway, I have edited it in now.
  14. CBS

    CBS Senior Member

    Messages:
    1,356
    Likes:
    216
    Western US
    I don't see how any meaningful interpretation of this "article" can be made without a list of which baseline variables were used as instrumental variables.

    Secondly, while I have not done instrumental variable analysis myself, a quick read of the Wiki entry on IV analysis (http://en.wikipedia.org/wiki/Instrumental_variable) would suggest that IV's present at least the possibility of exaggerating one's impression of the treatment effect by over emphasizing "response" of those who may have had some positive change as opposed to those who showed no benefit from treatment.

    According to Imbens and Angrist (1994):
    In layman's terms, LATE is typically used in social science research where randomization of subjects to treatment groups is impractical or unethical "and where controlled experiments are not available." It is designed to identify subgroups of patients where the treatment effect may have had an impact that otherwise would have gone undetected. This should raise questions about whether or not IV analysis was appropriate for the PACE data and whether or not the use of IV analysis met the necessary assumption "that the causal effect of interest does not vary across observations."

    It would take a bit more digging but I'm left wondering what the authors could have possibly used as "instrumental variables" and if IV analysis was even remotely appropriate. That said, if the R-squared really was 0.03, it would seem that this would raise even larger questions about the efficacy of CBT and GET to increase activity or reduce fatigue given the potential that the PACE trial violates the underlying assumptions of IV analysis and that IV analysis exaggerates the influence of only those subjects whose behavior actually changed.
    WillowJ and Dolphin like this.
  15. Dolphin

    Dolphin Senior Member

    Messages:
    6,439
    Likes:
    4,665
    For anyone who wants it, PACE Trial protocol is at: http://www.biomedcentral.com/1471-2377/7/6

    Here's the list of Process variables they had:

  16. oceanblue

    oceanblue Senior Member

    Messages:
    1,174
    Likes:
    342
    UK
    An interesting point, though of course as they made no measure of social desirability bias they couldn't include it in their model - so it wouldn't have an influence on the model findings (unless social desirability bias was correlated with one of the baseline variables they did include).

    I'm sure there's some (non-CFS) research out there where researchers did include independent measures of social desirability bias, to help them interpret the results of questionnaires etc. I think they even recommended this should be standard practice, but of course I can't remember where I saw the research. Such an independent measure of social desirability bias would have been really helpful in making sense of the PACE results.
  17. Dolphin

    Dolphin Senior Member

    Messages:
    6,439
    Likes:
    4,665
    Yes, some (non-CFS) research does use such measures.

    I started a thread on a paper on this here http://forums.phoenixrising.me/show...ability-response-bias-in-self-report-research .
    Here's the first post:
    oceanblue and WillowJ like this.
  18. Marco

    Marco Old blackguard

    Messages:
    1,136
    Likes:
    737
    Near Cognac, France
    When I was a psych undergrad we covered quite a bit on questionnaire development, validation, reliability etc and one of the problems in designing them is to try to avoid issues such as social desirability bias, acquiesence bias (where people always agree with a statement regardless of the content) and even lying (perhaps again relating to social desirability).

    One way around it is to include items in the questionnaire that set out to detect these biases rather than pertaining directly to the subject under study (e.g. "I have never lied in my life" in clearly a false statement and anyone agreeing with it is likely to also give false answers to other questions).

    Getting back to PACE its a slightly different matter as they are not being asked to respond to statements but to self rate their levels of fatigue and physical function. There is no easy means here to include 'lie scale' or other items to measure any social desirability bias so if such bias was to be discounted from the results some other means would have to be found.

    Perhaps if each individual's rating of their therapist and the trial therapy arm had been recorded these could be modelled as mediating variables but I doubt they have.

    Which again leads us back to the only way to discount these potential biases when dealing with cognitive therapies and subjective outcome measures is to use objective activity measures.
    oceanblue and Dolphin like this.
  19. Esther12

    Esther12 Senior Member

    Messages:
    5,088
    Likes:
    4,857
    Thanks D. So this will not have included things like membership of a self-help group, or receipt of benefits? I can't tell... anyone else like to hazard a guess?
  20. Dolphin

    Dolphin Senior Member

    Messages:
    6,439
    Likes:
    4,665
    I notice Kimberly Goldsmith is the second name on the Lancet PACE Trial paper suggesting a major role. I wonder how much she was involved with and/or agreed with the changes made, definition of normal functioning, etc.

See more popular forum discussions.

Share This Page