Julie Rehmeyer's 'Through the Shadowlands'
Writer Never Give Up talks about Julie Rehmeyer's new book "Through the Shadowlands: A Science Writer's Odyssey into an Illness Science Doesn't Understand" and shares an interview with Julie ...
Discuss the article on the Forums.

Is Trial Registratn Working to Prevent Publicatn of Unregistered Trials & selective outcome reportng

Discussion in 'Other Health News and Research' started by Dolphin, Aug 20, 2015.

  1. Dolphin

    Dolphin Senior Member

    Messages:
    10,574
    Likes:
    27,659
    (A minority interest)
    (Had to abbreviate title: that's all the characters I had)

    POMs=Primary Outcome Measures

    Free full text: http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0133718


     
    Sean, sarah darwins, Bob and 4 others like this.
  2. SOC

    SOC

    Messages:
    7,802
    Likes:
    16,454
    Why am I not surprised? :meh:
     
    A.B., ahmo and Dolphin like this.
  3. alex3619

    alex3619 Senior Member

    Messages:
    12,285
    Likes:
    33,808
    Logan, Queensland, Australia
    (I realize this is rhetorical, but ... ) Because we saw it in the PACE trial and almost nobody cared? PACE was before this initiative, but they did not use their proposed plan, selectively reported data, and abandoned an effective and objective measure with actometers. The real issue is they were largely not held to account, or appropriately criticized, for these issues.
     
    sarah darwins and A.B. like this.
  4. SOC

    SOC

    Messages:
    7,802
    Likes:
    16,454
    Oh yeah, that's why. :rolleyes:
    :)
     
    sarah darwins likes this.
  5. Simon

    Simon

    Messages:
    1,913
    Likes:
    14,348
    Monmouth, UK
    To summarise, then:
    • Although standards are in place to improve prospective registration and transparency in clinical trials, less than 15% of psychiatry trials were prospectively registered with no changes in POMs (Published Primary Outcome Measures).
    • Most trials were either not prospectively registered, changed POMs or the timeframes at some point after registration or changed participant numbers.
    • Prospective registration with no changes in POMs occurred more frequently with pharmaceutical funding.
    Marvellous
     
    Last edited: Aug 21, 2015
  6. Sasha

    Sasha Fine, thank you

    Messages:
    12,759
    Likes:
    33,927
    UK
    I remember that White (somewhere) defended changing the planned outcome measures in PACE that had been registered in the protocol, by saying that it was very common to make such changes. I was astonished by this claim at the time (particularly in relation to primary outcome analyses and particularly in a 641-patient, £5m trial) but I wonder if it's true in psychiatry? Are there comparative figures for non-psychiatric medicine?
     
    sarah darwins and SOC like this.
  7. worldbackwards

    worldbackwards A unique snowflake

    Messages:
    2,091
    Likes:
    10,354
    Earth
    It's a classic - It's OK, we're all frauds. The whole thing's a fraud. What's up with that?
     
    sarah darwins and Dolphin like this.
  8. Bob

    Bob

    Messages:
    9,846
    Likes:
    33,923
    England (south coast)
    Yes, that was rather mind-boggling. I think it's OK to decide that your primary outcome measures aren't helpful or appropriate after the trial is complete, but they still need to be published. Any post-hoc outcome measures can be published alongside the primary measures that are set out in the protocol or trial registry, and they should be clearly labelled as secondary and post-hoc. A discussion can be included about why the primary outcome measures proved to be unhelpful.
     
    Dolphin and SOC like this.
  9. Sasha

    Sasha Fine, thank you

    Messages:
    12,759
    Likes:
    33,927
    UK
    I think it's more complicated than that. White says (as I understand it) that the new analyses weren't post hoc because they hadn't seen the data (although they had seen the failure of the same analyses in the FINE trial). I don't think that one needs to assume post-hockery to find a shedload of reasons not to trust his interpretation of the data.

    But such wholesale disregard for the per-protocol analysis (especially in the "recovery" paper) has undermined trust in the trial and analysis according to the original results should now be produced.
     
    Valentijn likes this.
  10. Bob

    Bob

    Messages:
    9,846
    Likes:
    33,923
    England (south coast)
    Yes, he claimed that about the primary outcomes, but his disregard of rigorous process (i.e. by not publishing the proposed primary outcomes) negates the purpose of the protocol and trial registry. If we take his claims at face value, it's still problematic, and they should still publish the primary outcomes as proposed in the protocol. Otherwise there's no point in protocols. The whole point in the pre trial publications is to safe-guard against dubious practice, and to ensure rigour. It gives the public confidence that proper process has been followed. As it is, we have to accept the word of those involved in the trial that proper process was followed. We might have no reason to doubt them, but the public needs to have confidence. So it's complicated, but actually not that complicated. Good practice is to publish the outcomes as set out in the trial protocol. Changes are sometimes necessary, but the original primary outcome measures should still be published alongside any new outcome measures so that the public can judge the merit and value of the changes.

    That's my interpretation of good practice anyway. But perhaps others don't agree?
     
    Last edited: Aug 21, 2015
    Valentijn, Dolphin, SOC and 2 others like this.
  11. Sasha

    Sasha Fine, thank you

    Messages:
    12,759
    Likes:
    33,927
    UK
    The protocol has more than one purpose. One is as part of the submission of the trial to a review committee to see if it should be funded. If people want to make protocol changes later, they need to run it past another committee to get the OK.

    The weird thing about PACE, though, is that (in my experience, anyway) changes are only made to analyses when the data are in and the stattos find that the actual data violate the statistical assumptions of the planned analysis (the usual problem is that they're not normally distributed) - but I think the change is generally then to normalise the data by transforming it (log scale, etc.). It doesn't require that the original analyses be done too (I think? Unless that's done as a sensitivity analysis? I'm rusty).

    This isn't what happened in PACE, though - especially in the "recovery" paper, which took a wrecking ball to the planned analyses.

    It would be interesting to know what the establishment view is of what should happen when a protocol has been published and the analyses change. Cochrane would be a good source for best practice.
     
    Valentijn and Bob like this.
  12. Dolphin

    Dolphin Senior Member

    Messages:
    10,574
    Likes:
    27,659
    I think it's more likely POMs=Primary Outcome Measures. That's what is usually looked at in papers on outcome reporting bias.
     
    Simon and sarah darwins like this.

See more popular forum discussions.

Share This Page