1. Patients launch a $1.27 million crowdfunding campaign for ME/CFS gut microbiome study.
    Check out the website, Facebook and Twitter. Join in donate and spread the word!
The Chronic Fatigue Initiative and Interview with Mady Hornig
In a follow-up article to the recent IACFS/ME conference presentation in San Francisco, and after speaking at length with Dr. Mady Hornig, 'searcher' delves deeper into the impressive work being completed by the Chronic Fatigue Initiative, and focuses in on those cytokine results ...
Discuss the article on the Forums.

Freedom of Information Request for 6MWT data for "recovered" in PACE Trial turned down

Discussion in 'General ME/CFS News' started by Tom Kindlon, Sep 2, 2013.

  1. Tom Kindlon

    Tom Kindlon Senior Member

    Messages:
    205
    Likes:
    554
    https://www.whatdotheyknow.com/request/6min_walking_test_results_for_re#comment-42596


    --------
    -------

    --------

    --------

  2. Tom Kindlon

    Tom Kindlon Senior Member

    Messages:
    205
    Likes:
    554
    I can't say I find this response very satisfactory: "I can confirm that the information you have requested is not held by the College."

    Anna just said on Twitter it was frustrating as their letter wasn't out till after the 40-day appeal period was up (see: https://twitter.com/Annakwood/status/374585141508968448 ).

    One can see the PACE Trial investigators' letter on the recovery paper here: http://www.meassociation.org.uk/?p=16209 - it doesn't give the data - no major reason (they may only have 6MWT data for 72% of those that did GET, but they still published that data in the Lancet paper).

    People can comment underneath the FOI request if so inclined.
    Valentijn likes this.
  3. Mark

    Mark Acting CEO

    Messages:
    4,412
    Likes:
    1,718
    Sofa, UK
    I think this gets quite complicated quite quickly. One of the standard FoI exemptions is where the information is not held. However, in this case, the underlying data clearly is held, and using that data the analysis could certainly be performed to provide the information requested. But any such analysis would require work, so should FoIs be expected to cover all possible (or reasonable) statistical analyses of the raw data, or the raw data itself, or what? My answer: the question demonstrates that a credible concept of the scientific method demands open data, even if that requires planning for anonymisation of source data from the outset as a matter of course.

    The raw data cannot all be released because the entire raw dataset contains information that could identify individuals. But despite protestations from MRC staff who I spoke to at an Open Data conference last year, I am not at all convinced that in a study like PACE it is not feasible to release an entire anonymised dataset. On the contrary, I think that the release of such anonymised datasets should be the minimum standard that should be required for scientific credibility.

    PACE brings this issue into sharp relief for those who are paying close attention to it. It is no more than scientific spin to leave the entire dataset - publicly funded - in the hands of researchers with a clear vested interest in the outcome, and to allow them to determine the protocols for what analysis is published. They are not publishing this crucial data that has been requested, nor are they publishing any analysis of the data that would answer this very pertinent question. They are, quite simply, given free reign to spin the data however they choose. They can decide in great detail what questions they will - and won't - answer, and they are analysing it in ways that enable them to say the things they want to be able to say. They have obvious conflicts of interest and lifelong careers invested in the success of what they claim to be studying. This is not, by any stretch of imagination for any intelligent observer, worthy of the name 'Science'. It is simply an exercise in spin. That their conclusions are so pathetic even after this exercise in spin is a damning indictment of the alleged 'treatment' on which their careers are based.


    Very much so, and they seem to have played a disingenuous game to say the least. I would think there should be a basis for an appeal, or a fresh request, based on this timeline. It's a strong argument just in terms of raw facts of the timeline if it could be laid out clearly. Similarly with most of the other FOI requests: they keep on arguing that they are going to present the answers some time, some day...but they never do, and surely eventually when this is all pointed out to an FOI tribunal, an unbiased judge should be able to see that this is not on.
  4. Tom Kindlon

    Tom Kindlon Senior Member

    Messages:
    205
    Likes:
    554
    Ok, but one is setting a low standard to escape FOIs if a holder of data can claim that means (averages) aren't held when they hold the raw data. One doesn't need to be a mathematician or statistician to do this - somebody could do this by hand. Having to calculate a mean is only a tiny bit more than a request to ask "how many of X happened/whatever" (SDs can easily be computed also with a small bit of help from a computer as you know).

    Anyway, hopefully somebody will request the data.
    Valentijn likes this.
  5. Bob

    Bob

    Messages:
    6,357
    Likes:
    6,668
    England, UK
    Mark

    There is legal guidance about when data should be released if it isn't stored in the exact format asked for in a FOI request.
    I can't remember the exact details, but I think it's defined in the number of hours needed to convert the data.
    If it can be converted within a certain number of person-hours, then not having the data in the exact format requested is not an adequate reason for denying the data.

    As for making a fresh FOI request for the same data, that is always possible. If the other party refuses to release the data, that is only the beginning of the process. The next step of the process is to take the request (make a 'complaint') to the Information Commissioner's Office (ICO). The ICO will then decide, independently, whether the data should be released or not. After that decision is made, either party can appeal to a tribunal service, where a judge will make the decision.
    Valentijn, ukxmrv and Tom Kindlon like this.
  6. Sean

    Sean Senior Member

    Messages:
    1,048
    Likes:
    1,304
    It is a total sham. The most basic requirement of all in science is that your data and methods must be absolutely transparent for others to review. Anything less is either incompetence, laziness, or fraud.
    leela, peggy-sue, Shell and 3 others like this.
  7. Mark

    Mark Acting CEO

    Messages:
    4,412
    Likes:
    1,718
    Sofa, UK
    Thanks Bob, I didn't know that about the number of hours needed to do the processing. It did seem like a rather grey area to me and I never read anything about that question before. Defining it in terms of the number of hours required to process seems about as good a rule as I can imagine to address this issue.

    So it's clearly important to be smart about this in formulating FOI requests - making each individual request as simple to calculate as possible should cut off this kind of excuse. For example, if it were possible to specifically request data that didn't require any intelligent processing in order to provide it, but which could be processed later to provide the information, maybe if this were done exactly right they would not be able to give the excuse that the data 'isn't held'. In effect, work out which columns in the existing database you can ask for to get the desired data, and don't ask them to do any calculations on them because they could argue it would take them too long. Having said that, I think this request was probably as simple as it could be: I guess the calculation they applied to determine who was supposedly 'recovered' would be way too complicated to determine for oneself. So asking for the 6MWT data for those specific individuals was a really good question IMO and I can't think of a way to improve on it.

    It really is an absolute travesty of science that they are allowed to guard their data for years and massage it however they see fit, and nobody else is allowed to see that data and perform analysis on it. They can spin it however they like, and nobody can counter it because only they have the data. To me, it makes their publications null and void as supposed 'scientific' evidence. They should be inadmissible as 'science' for this reason alone. It would be bad enough with privately funded 'science'; for publicly funded science it is just inexcusable. The fact that it is possible for them to get away with this behaviour brings science into disrepute, and shame on the scientific establishment that it allows this sort of thing to go on.
    Battery Muncher, leela, MeSci and 4 others like this.
  8. Mark

    Mark Acting CEO

    Messages:
    4,412
    Likes:
    1,718
    Sofa, UK
    I agree completely Sean. However, modern science almost universally fails to meet these most basic of requirements. I often struggle to find the words to sum up this situation. The closest simple summary I can get is to say that science, as I understand it, is not really practiced at all any more - at least, not in many fields. I think the physicists generally do actual science, and anyone who releases their full datasets has a strong claim to be doing science, but the rest of it is little more than spin, when you think about it - if their data isn't open, all they really have is an appeal to authority and a polite request to trust them and treat them as if they were (mythical) impartial observers. I think the traditional emphasis on published papers has obscured the reality that the truly valuable output of scientific study is the data, and objective analysis of that data is only really possible if the data is open so that everyone can analyse it for themselves. Until it gets over that hurdle, much of academia (PACE included) will remain more like a religion than a science as far as I'm concerned.
  9. Tom Kindlon

    Tom Kindlon Senior Member

    Messages:
    205
    Likes:
    554
  10. user9876

    user9876 Senior Member

    Messages:
    634
    Likes:
    1,318
    Maybe there should be a question about the form the data is held and its schema as this would give an insite into the time taken to write and process a query. I'm assuming that the data would be in a standard stats package and hence processing would be easy.

    Personally I would be surprised if they hadn't already written a query to get the 6MWT data for those they judge as recovered. Not to would show a remarkable lack of curiosity for a researcher.
  11. biophile

    biophile Places I'd rather be.

    Messages:
    1,250
    Likes:
    3,320
    Re the second request for 6MWD data. This should be rather interesting if ever released. I would be just as interested in seeing the mean(SD) and median(IQR) scores on physical function SF-36 for the "recovered" participants. I bet those would be real eye-openers when compared to population data from healthy people of working age.

    As an aside, compare the attitude of PACE, to the attitude of O'Dowd et al (2006). In their extensive 140 page report on a single RCT of group CBT for CFS which would take ages to read properly:

    "We have presented all of our data, with explanations of our reasoning, so that the work is as transparent as possible."

    O'Dowd H, Gladwell P, Rogers CA, Hollinghurst S, Gregory A. Cognitive behavioural therapy in chronic fatigue syndrome: a randomised controlled trial of an outpatient group programme. Health Technol Assess. 2006 Oct;10(37):iii-iv, ix-x, 1-121. PMID: 17014748. http://www.journalslibrary.nihr.ac.uk/hta/volume-10/issue-37
    MeSci, Sean, Bob and 1 other person like this.
  12. Tom Kindlon

    Tom Kindlon Senior Member

    Messages:
    205
    Likes:
    554
    Yes, would be a good one for somebody to make.
  13. Tom Kindlon

    Tom Kindlon Senior Member

    Messages:
    205
    Likes:
    554
    Perhaps somebody should now request median and interquartile ranges for the 6 minute walking distance data for the recovered so that it can be dealt with at the same time?
  14. Bob

    Bob

    Messages:
    6,357
    Likes:
    6,668
    England, UK
    Instructions to public bodies re cost of fulfilling FOI requests, from the ICO website:

    http://www.ico.org.uk/for_organisat...hen-can-we-refuse-a-request-for-information-2

    MeSci, Valentijn and Tom Kindlon like this.
  15. Anna Wood

    Anna Wood wood/sheridan

    Messages:
    6
    Likes:
    154
    Hello Everyone

    I don't normally have the energy to come on here very often, but will try to keep this up to date. Queen Mary Univiersity, unsurprisingly declined by FOI request - https://www.whatdotheyknow.com/request/pace_trial_recovery_rates_and_po
    with the standard 'we don't hold the data' response.
    I've submitted an internal review.
    Interesting Bob, to see the details about time to calculate FOI requests - this one really should be only minutes work!

    I still find it hard to believe that the walking results were so poor - most studies show PWME are no more deconditioned than sedentary people, so why were they unable to walk further on a one off test? I'm sure I could, although I would feel dreadful and not be able to repeat the next day.

    The other issue is the test used 10m instead of the standard 30m for this types of test. Again , I really can't understand why they did this. But they will try to use it as an argument not to give out the data - as it won't be valid. But even supposing that the extra turns (and I estimate there were 25 extra turns), took an very generous extra 1 second, the GET 6mwt would only increase by about 50m - from 379, to 429m. That is still a lot below average.
    Battery Muncher, Sean, MeSci and 4 others like this.
  16. Tom Kindlon

    Tom Kindlon Senior Member

    Messages:
    205
    Likes:
    554
    People should be free to decide for themselves what is or isn't valid in terms of interpreting the results of the 6 minute walking test distance once the data is released. I think there is some information out there on the effect of the length of the track or, even if there isn't now, there might be in future (but the data on the recovered individuals could be lost for ever if it's not released now). So I don't think it's a good reason not to release the data.

    Related to this: if a pharmaceutical company was hyping claims that their drug could bring about recovery, a claim that the data on recovery might be misinterpreted would likely be seen for what it is: a way to continue to hype the effectiveness.

    People misinterpret data all the time - one could make that a reason for not releasing most data. Also, the same data can often been seen in different ways.

    ETA: Here's an abstract on how getting access to data from research tends to be lost with time:

    biophile, Sea, Valentijn and 2 others like this.
  17. Anna Wood

    Anna Wood wood/sheridan

    Messages:
    6
    Likes:
    154
    Interesting - I can see why this happens. Most research is done by PhD students and early career researchers who move often (I have worked for 4 institutions over an 11 year period). But also how data is stored changes - in theory I've got my PhD data, but it is on a zip disk, and I've not idea if I could access a zip drive to read it!

    Totally agree with your previous point about the worry about what might happen to the data not being a valid reason not to give it. Still think they will try using it as an excuse!
    Tom Kindlon and Esther12 like this.
  18. Esther12

    Esther12 Senior Member

    Messages:
    4,881
    Likes:
    4,194
    Thanks for trying to get the release of this data. It's really important to let patients be as informed as possible about the treatments options available to them, and how they might affect them. Hopefully QMUL will see sense after a bit more prodding, and stop trying to keep patients in the dark.
    Sean, Valentijn, Tom Kindlon and 2 others like this.
  19. MeSci

    MeSci ME/CFS since 1995; activity level 6

    Messages:
    2,305
    Likes:
    2,442
    Cornwall, UK




    (My bolding)

    Do you think they mean excessive?!
  20. Bob

    Bob

    Messages:
    6,357
    Likes:
    6,668
    England, UK
    Well spotted! Yes, I suppose they do. Strange that they should get the wording of their legal guidelines wrong.

See more popular forum discussions.

Share This Page