This article was co-written by renegade psychologist, James C. Coyne.
I'm not highlighting it for any specific relevance to ME/CFS. It is just an example of a type of problem that can occur in questionnaires which I thought might be interesting to the odd person to highlight.
The paper is summarised in the first part of this blogpost: http://jcoynester.wordpress.com/201...-constructing-a-title-for-a-scientific-paper/
The paper requires a bit of attention. However, I imagine it would be quite a bit less work to read if one skipped the Methods and Results sections.
Here's an extract from the introduction:
I once observed some questionnaire responses from ME/CFS patients (unpublished study) which looked like there were some errors in the responses due to inattention. I will now have a name and reference for this sort of problem.
I'm not highlighting it for any specific relevance to ME/CFS. It is just an example of a type of problem that can occur in questionnaires which I thought might be interesting to the odd person to highlight.
The paper is summarised in the first part of this blogpost: http://jcoynester.wordpress.com/201...-constructing-a-title-for-a-scientific-paper/
Abstract
Objective
We examined the effectiveness of reverse worded items as a means of reducing or preventing response bias. We first distinguished between several types of response bias that are often confused in literature. We next developed arguments why reversing items is probably never a good way to address response bias. We proposed testing whether reverse wording affects response bias with item-level data from the Multidimensional Fatigue Inventory (MFI-20), an instrument that contains reversed worded items.
Methods
With data from 700 respondents, we compared scores on items that were similar with respect either to content or to direction of wording. Psychometric properties of sets of these items worded in the same direction were compared with sets consisting of both straightforward and reversed worded items.
Results
We did not find evidence that ten reverse-worded items prevented response bias. Instead, the data suggest scores were contaminated by respondent inattention and confusion.
Conclusions
Using twenty items, balanced for scoring direction, to assess fatigue did not prevent respondents from inattentive or acquiescent answering. Rather, fewer mistakes are made with a 10-item instrument with items posed in the same direction. Such a format is preferable for both epidemiological and clinical studies.
Citation: Sonderen Ev, Sanderman R, Coyne JC (2013) Ineffectiveness of Reverse Wording of Questionnaire Items: Let’s Learn from Cows in the Rain. PLoS ONE 8(7): e68967. doi:10.1371/journal.pone.0068967
The paper requires a bit of attention. However, I imagine it would be quite a bit less work to read if one skipped the Methods and Results sections.
Here's an extract from the introduction:
Response Bias
The process of obtaining survey data is complex, with many possibilities of discrepancies arising between the state or opinion the researcher wants to elicit and the answer given by the respondent [3]. Usually this discrepancy is called response bias [4]. Based on Rorer [1], Weijters [5] distinguishes two main types of response bias: response set and response style. He defines response set as bias related to the content of the items and response style as a tendency to answer items regardless their content. The best known type of response set is social desirability, in which a person’s response is a function of the desirability of the response rather than its veracity. Three types of response styles can be distinguished.
Respondents may have read and understood completely the question and answer categories, but nonetheless be inclined to agree with statements in general (acquiescence), to disagree (disacquiescence), or to give extreme answers, or, alternatively, less extreme answers. Consistent with Swain et al. [6], we will take acquiescence as an example of this type of response styles.
Respondents may also lack sufficient attention to carefully read both the question and answer categories, and thus, by missing the intended meaning of an item, give a response that may differ from the true value. Krosnick [7] mentions a satisficing response style, whereby the respondent deliberately makes less effort to understand all subtleties of the question. We will call this style inattention, irrespective whether the respondent is aware of it or not.
Finally, the question in combination with answer categories may be too difficult for a respondent to comprehend. Sometimes the respondent may think the item is well understood, but still an error can be made, due to a high level of difficulty [6]. The respondent may however be aware of this difficulty, and thus the answer can be considered a ‘best guess’. We will call both varieties of this type confusion.
I once observed some questionnaire responses from ME/CFS patients (unpublished study) which looked like there were some errors in the responses due to inattention. I will now have a name and reference for this sort of problem.