Severe ME Day of Understanding and Remembrance: Aug. 8, 2017
Determined to paper the Internet with articles about ME, Jody Smith brings some additional focus to Severe Myalgic Encephalomyelitis Day of Understanding and Remembrance on Aug. 8, 2017 ...
Discuss the article on the Forums.

Need assistance to build rating system for studies for my site

Discussion in 'Information and Resources' started by RustyJ, Dec 20, 2014.

  1. RustyJ

    RustyJ Contaminated Cell Line 'RustyJ'

    Messages:
    1,189
    Likes:
    889
    Noosaville, Aust
    I really need a bit of help putting together a points rating system for studies I publish on my site. There is a built in ratings system at the end of each post but I am too befuddled to get this going. I really would appreciate some assistance or suggestions, particularly from someone who has had experience in the scientific fields.

    An image of a ratings template is attached. I am currently trying to set up a review for this study (random):

    http://www.sciencedirect.com/science/article/pii/S0033318214001935

    Eg A variable might be Size, so this study being small might get just a 3 out of 10
    Another variable might relate to Reliability or some other such word, eg looking here at this study looking at subjective reporting might get 3 out of 10
    Another variable might be Relevance
    Another might pertain to Randomization

    The idea is to provide a summary at a glance that people with ME can understand.
     

    Attached Files:

    Last edited: Dec 21, 2014
  2. RustyJ

    RustyJ Contaminated Cell Line 'RustyJ'

    Messages:
    1,189
    Likes:
    889
    Noosaville, Aust
    Safety might be a good variable. So this study might get an 8, based on its abstract?
    Cohort might be another
    Methodology (double blind, random etc). So this study might get high points, but lose on subjective reporting
     
    Esther12 likes this.
  3. taniaaust1

    taniaaust1

    Messages:
    11,876
    Likes:
    12,603
    Sth Australia
    "
    Cohort might be another"

    People would probably rate that based on if the study fits whatever cohort they themselves are in so Im not sure how that one would go.
     
    RustyJ likes this.
  4. WillowJ

    WillowJ คภภเє ɠรค๓թєl

    Messages:
    3,763
    Likes:
    4,842
    WA, USA
    I think I would call the subjective reporting or other ways of assessing improvement (walk tests, blood tests, etc.) as Outcomes or as Assessments: that's a good thing to look at.

    Whether and how the study is controlled is a good thing to check. This study is controlled and uses placebo, though I would want to see the full text to assign points.

    In nonpharmacutical trials it's harder, but should be controlled with an active stand-in for placebo. No control gets zero points, having a control group but not using it comparatively should also be rated harshly. Wait lists as controls is not great either. A control group should have the same attention and expectation especially from the investigators.

    Blinding is important especially in assessors.

    You can also list the design of the study. Case study, case-control, randomized controlled, longitudinal, etc. You may be able to grade the suitability of the design for answering the study question(s).

    This essay is makes some good points:
    http://www.huffingtonpost.com/jenni...-understand-a-scientific-paper_b_5501628.html
     
    RustyJ likes this.
  5. SickOfSickness

    SickOfSickness Senior Member

    Messages:
    2,305
    Likes:
    2,239
    US
    A possible rating to show if the study is outdated or superceded by other research.

    I would like an overall rating by patients like me.
     
    alex3619 likes this.
  6. RustyJ

    RustyJ Contaminated Cell Line 'RustyJ'

    Messages:
    1,189
    Likes:
    889
    Noosaville, Aust
  7. alex3619

    alex3619 Senior Member

    Messages:
    12,392
    Likes:
    34,698
    Logan, Queensland, Australia
    I think a points process might be confusing. Everyone would have to check out what the points mean, and then later would forget, and then need to check again.

    Simple text descriptions might be better. For example, RCT placebo controlled double blinded, or RCT not placebo controlled, or Exploratory Study, etc. Also comments in text like interesting, doubtful, flawed, important etc.

    What do others think?

    At the end it would be a good ideato give it a rating out of ten, to sum up, though alternatives might be better like Terrible, Poor, OK, Good, Excellent.
     
    Leopardtail and WillowJ like this.
  8. RustyJ

    RustyJ Contaminated Cell Line 'RustyJ'

    Messages:
    1,189
    Likes:
    889
    Noosaville, Aust
    It would be useful, but I can't see a way for voters can input, although there clearly is a bar reflecting this. It is a wordpress site, so patients can comment. I imagine I would adjust the bars as visitors pointed out how wrong I was. :)
     
  9. RustyJ

    RustyJ Contaminated Cell Line 'RustyJ'

    Messages:
    1,189
    Likes:
    889
    Noosaville, Aust
    This is part gimmick, part useful. The bars look good (to me, at least). At a glance you should be able to see whether the study has some merit or not. A lot of the time we are forced to work off abstract, which as you know can often bear little resemblance to reality.
     
  10. alex3619

    alex3619 Senior Member

    Messages:
    12,392
    Likes:
    34,698
    Logan, Queensland, Australia
    Yeah, but its important to note if a study has not been seen, only the abstract, in any rating system.
     
    WillowJ likes this.
  11. alex3619

    alex3619 Senior Member

    Messages:
    12,392
    Likes:
    34,698
    Logan, Queensland, Australia
    I am not sure what kind of system you are building. To keep things simple, if it has buttons, then a more info button which might include reader reviews would be good.

    If its a text file system then this option would not be a good idea - you would want to keep it streamlined as far as possible, with minimal clutter. In such a system a number rating might be ideal.

    Simple study bars only tell you so much about the design. What would it look like? What level of interactivity would it have? I would think you would go for intuitive use rather than anything complicated, as this would be most useful. Is it searchable? Are there menu categories?

    Many decisions about what you want can change the specifications you might go for.
     
    Last edited: Dec 22, 2014
  12. alex3619

    alex3619 Senior Member

    Messages:
    12,392
    Likes:
    34,698
    Logan, Queensland, Australia
    Can you post a direct link to a site similar to what you are looking at? That would give us a better idea of the kinds of questions that are relevant, and hence the kinds of advice we might give.
     
  13. RustyJ

    RustyJ Contaminated Cell Line 'RustyJ'

    Messages:
    1,189
    Likes:
    889
    Noosaville, Aust
    I have built the site on my computer and will probably transfer template over to my existing site over Christmas.

    But here is a page of a sample site where the Review box is operating, except I will put the review box at the bottom of the post, not at the top as in example. It is pretty ugly in the sample, but not so much at the end of a longer post.

    http://themeforest.net/item/smartma...ordpress-magazine/full_screen_preview/6652608

    It's risky doing the sort of thing, as there is so much subjectivity associated with critiquing a study, especially by rank amateurs, like me. And quite frankly most of the time I am too befuddled to work things out. As I said, I will probably be relying on comments to adjust bars.

    All changes are undertaken by the editor, not visitors. And it is not intended to be anything like heal click.

    The style of article is pretty straight forward, a short formal summary, followed by the abstract. Then links to forums, other blogs, other papers. I do not have the energy for the frothy sort of treatment someone like Cort might apply.

    Here is link to a similar type article on my existing site. (I was a bit self conscious about putting my site link in this thread before now). As you can see, both the abstract and my intro is a little misleading and needs a little more analysis.

    http://www.shoutoutaboutme.com/about-mecfs/duloxetine-eases-mental-fatigue-in-cfs/

    In reality the study failed, and I suspect the improvements noted were quite slight.
     
  14. alex3619

    alex3619 Senior Member

    Messages:
    12,392
    Likes:
    34,698
    Logan, Queensland, Australia
    So its a case by case review. Hmmm.

    So, as in the first jpg image, what you are maybe looking for is a simple list of domains to represent in a bar graph that someone can see at a glance? And you would like to know what domains to use? And there will for the most part be no review? Could someone post you a short review or commentary and have it published (at your discretion)? If so then much of it might write itself, with minimal effort from you.
     
  15. RustyJ

    RustyJ Contaminated Cell Line 'RustyJ'

    Messages:
    1,189
    Likes:
    889
    Noosaville, Aust
    Yes, I probably explained it badly, but I guess domains is as good a term for the variables as anything. And yes I would like to know what domains to use.

    Yes someone could contribute. Information for patients is my primary concern. My new template, actually allows multiple authors, so I could easily do, that, attach the abstract and the whole page is authored to the contributor. I would still use the graph though, as it would be a motif, as much as a practical tool.

    I should say I sometimes just rewrite the abstract into news article style, but rarely depart from the script. I do not know enough to provide even rudimentary commentary. However, this style does allow me to better promote an aspect of the paper.

    Mind you, a half dozen expert commentators would probably come up with 6 diverging opinions on these papers any, particularly if there is a political response.
     
  16. alex3619

    alex3619 Senior Member

    Messages:
    12,392
    Likes:
    34,698
    Logan, Queensland, Australia
    Of course. ;) Even in strict science diverging views can happen. Politics makes it almost certain.
     
    RustyJ likes this.

See more popular forum discussions.

Share This Page