joshualevy
Senior Member
- Messages
- 158
A further problem with the Chalder fatigue questionnaire is illustrated by the observation that the bimodal score and Likert score of 10 participants moved in opposite directions at consecutive assessments i.e. one scoring system
showed improvement whilst the other showed deterioration.
Now that's a red flag.
It's not a red flag, it's exactly what would be expected from a measurement change that did not impact results.
Obviously, any change of measurement technique will cause the measurement to change. The meaningful questions are: do they change a lot (ie. enough to change the outcome of the study), and do they change enough to change the certainty of the study (ie. enough to change the p values reported in the study).
What you want if you change measurements is small changes up and small changes down, and overall no symmetric changes. So having some data points move up at one point and down at another (all changes small) is a good thing, not a bad thing, at least statistically speaking for a change in measurement.
Think about it this way: when you change measurements, some data point will change so little you can not even measure it. Those are said to not change. Other data points will move up (hopefully only a little), and others will move down. As a thought experiment, lets assume that 40% stay the same, 30% go up slightly, and 30% down slightly. If you compare two data points there is a 9% chance (30% of 30%) that one will go up and one down. In his post, Sam says that about 10/149 patients had this up-down behavior, and that is about 6.7%. My guess is they had even better stability: approximately 50% staying the same 25% up and 25% down.
If changing the Likert measurement changes the study, then you would see one of two things: a statistically significant change in one of the reported outcomes of the study, or a change in the p-values reported in the study. Neither of these things were seen.