• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

Need assistance to build rating system for studies for my site

RustyJ

Contaminated Cell Line 'RustyJ'
Messages
1,200
Location
Mackay, Aust
I really need a bit of help putting together a points rating system for studies I publish on my site. There is a built in ratings system at the end of each post but I am too befuddled to get this going. I really would appreciate some assistance or suggestions, particularly from someone who has had experience in the scientific fields.

An image of a ratings template is attached. I am currently trying to set up a review for this study (random):

http://www.sciencedirect.com/science/article/pii/S0033318214001935

Eg A variable might be Size, so this study being small might get just a 3 out of 10
Another variable might relate to Reliability or some other such word, eg looking here at this study looking at subjective reporting might get 3 out of 10
Another variable might be Relevance
Another might pertain to Randomization

The idea is to provide a summary at a glance that people with ME can understand.
 

Attachments

  • Ratings.JPG
    Ratings.JPG
    43.8 KB · Views: 33
Last edited:

RustyJ

Contaminated Cell Line 'RustyJ'
Messages
1,200
Location
Mackay, Aust
Safety might be a good variable. So this study might get an 8, based on its abstract?
Cohort might be another
Methodology (double blind, random etc). So this study might get high points, but lose on subjective reporting
 

taniaaust1

Senior Member
Messages
13,054
Location
Sth Australia
"
Cohort might be another"

People would probably rate that based on if the study fits whatever cohort they themselves are in so Im not sure how that one would go.
 

WillowJ

คภภเє ɠรค๓թєl
Messages
4,940
Location
WA, USA
I think I would call the subjective reporting or other ways of assessing improvement (walk tests, blood tests, etc.) as Outcomes or as Assessments: that's a good thing to look at.

Whether and how the study is controlled is a good thing to check. This study is controlled and uses placebo, though I would want to see the full text to assign points.

In nonpharmacutical trials it's harder, but should be controlled with an active stand-in for placebo. No control gets zero points, having a control group but not using it comparatively should also be rated harshly. Wait lists as controls is not great either. A control group should have the same attention and expectation especially from the investigators.

Blinding is important especially in assessors.

You can also list the design of the study. Case study, case-control, randomized controlled, longitudinal, etc. You may be able to grade the suitability of the design for answering the study question(s).

This essay is makes some good points:
http://www.huffingtonpost.com/jenni...-understand-a-scientific-paper_b_5501628.html
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
I think a points process might be confusing. Everyone would have to check out what the points mean, and then later would forget, and then need to check again.

Simple text descriptions might be better. For example, RCT placebo controlled double blinded, or RCT not placebo controlled, or Exploratory Study, etc. Also comments in text like interesting, doubtful, flawed, important etc.

What do others think?

At the end it would be a good ideato give it a rating out of ten, to sum up, though alternatives might be better like Terrible, Poor, OK, Good, Excellent.
 

RustyJ

Contaminated Cell Line 'RustyJ'
Messages
1,200
Location
Mackay, Aust
I would like an overall rating by patients like me.

It would be useful, but I can't see a way for voters can input, although there clearly is a bar reflecting this. It is a wordpress site, so patients can comment. I imagine I would adjust the bars as visitors pointed out how wrong I was. :)
 

RustyJ

Contaminated Cell Line 'RustyJ'
Messages
1,200
Location
Mackay, Aust
I think a points process might be confusing. Everyone would have to check out what the points mean, and then later would forget, and then need to check again.

Simple text descriptions might be better. For example, RCT placebo controlled double blinded, or RCT not placebo controlled, or Exploratory Study, etc. Also comments in text like interesting, doubtful, flawed, important etc.

What do others think?

At the end it would be a good ideato give it a rating out of ten, to sum up, though alternatives might be better like Terrible, Poor, OK, Good, Excellent.

This is part gimmick, part useful. The bars look good (to me, at least). At a glance you should be able to see whether the study has some merit or not. A lot of the time we are forced to work off abstract, which as you know can often bear little resemblance to reality.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
This is part gimmick, part useful. The bars look good (to me, at least). At a glance you should be able to see whether the study has some merit or not. A lot of the time we are forced to work off abstract, which as you know can often bear little resemblance to reality.

Yeah, but its important to note if a study has not been seen, only the abstract, in any rating system.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
I am not sure what kind of system you are building. To keep things simple, if it has buttons, then a more info button which might include reader reviews would be good.

If its a text file system then this option would not be a good idea - you would want to keep it streamlined as far as possible, with minimal clutter. In such a system a number rating might be ideal.

Simple study bars only tell you so much about the design. What would it look like? What level of interactivity would it have? I would think you would go for intuitive use rather than anything complicated, as this would be most useful. Is it searchable? Are there menu categories?

Many decisions about what you want can change the specifications you might go for.
 
Last edited:

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
It would be useful, but I can't see a way for voters can input, although there clearly is a bar reflecting this. It is a wordpress site, so patients can comment. I imagine I would adjust the bars as visitors pointed out how wrong I was. :)
Can you post a direct link to a site similar to what you are looking at? That would give us a better idea of the kinds of questions that are relevant, and hence the kinds of advice we might give.
 

RustyJ

Contaminated Cell Line 'RustyJ'
Messages
1,200
Location
Mackay, Aust
Can you post a direct link to a site similar to what you are looking at? That would give us a better idea of the kinds of questions that are relevant, and hence the kinds of advice we might give.

I have built the site on my computer and will probably transfer template over to my existing site over Christmas.

But here is a page of a sample site where the Review box is operating, except I will put the review box at the bottom of the post, not at the top as in example. It is pretty ugly in the sample, but not so much at the end of a longer post.

http://themeforest.net/item/smartma...ordpress-magazine/full_screen_preview/6652608

It's risky doing the sort of thing, as there is so much subjectivity associated with critiquing a study, especially by rank amateurs, like me. And quite frankly most of the time I am too befuddled to work things out. As I said, I will probably be relying on comments to adjust bars.

All changes are undertaken by the editor, not visitors. And it is not intended to be anything like heal click.

The style of article is pretty straight forward, a short formal summary, followed by the abstract. Then links to forums, other blogs, other papers. I do not have the energy for the frothy sort of treatment someone like Cort might apply.

Here is link to a similar type article on my existing site. (I was a bit self conscious about putting my site link in this thread before now). As you can see, both the abstract and my intro is a little misleading and needs a little more analysis.

http://www.shoutoutaboutme.com/about-mecfs/duloxetine-eases-mental-fatigue-in-cfs/

In reality the study failed, and I suspect the improvements noted were quite slight.
 

alex3619

Senior Member
Messages
13,810
Location
Logan, Queensland, Australia
So its a case by case review. Hmmm.

So, as in the first jpg image, what you are maybe looking for is a simple list of domains to represent in a bar graph that someone can see at a glance? And you would like to know what domains to use? And there will for the most part be no review? Could someone post you a short review or commentary and have it published (at your discretion)? If so then much of it might write itself, with minimal effort from you.
 

RustyJ

Contaminated Cell Line 'RustyJ'
Messages
1,200
Location
Mackay, Aust
So its a case by case review. Hmmm.

So, as in the first jpg image, what you are maybe looking for is a simple list of domains to represent in a bar graph that someone can see at a glance? And you would like to know what domains to use? And there will for the most part be no review? Could someone post you a short review or commentary and have it published (at your discretion)? If so then much of it might write itself, with minimal effort from you.

Yes, I probably explained it badly, but I guess domains is as good a term for the variables as anything. And yes I would like to know what domains to use.

Yes someone could contribute. Information for patients is my primary concern. My new template, actually allows multiple authors, so I could easily do, that, attach the abstract and the whole page is authored to the contributor. I would still use the graph though, as it would be a motif, as much as a practical tool.

I should say I sometimes just rewrite the abstract into news article style, but rarely depart from the script. I do not know enough to provide even rudimentary commentary. However, this style does allow me to better promote an aspect of the paper.

Mind you, a half dozen expert commentators would probably come up with 6 diverging opinions on these papers any, particularly if there is a political response.