• Welcome to Phoenix Rising!

    Created in 2008, Phoenix Rising is the largest and oldest forum dedicated to furthering the understanding of and finding treatments for complex chronic illnesses such as chronic fatigue syndrome (ME/CFS), fibromyalgia (FM), long COVID, postural orthostatic tachycardia syndrome (POTS), mast cell activation syndrome (MCAS), and allied diseases.

    To become a member, simply click the Register button at the top right.

"Physicians’ Diagnostic Accuracy, Confidence, and Resource Requests" (they're over-confident)

Dolphin

Senior Member
Messages
17,567
http://archinte.jamanetwork.com/art...fInternalMedicine:NewIssue11/25/2013#Abstract

Physicians’ Diagnostic Accuracy, Confidence, and Resource Requests: A Vignette Study

JAMA Intern Med. 2013;173(21):1952-1958. doi:10.1001/jamainternmed.2013.10081.

November 25, 2013, Vol 173, No. 21 >

Ashley N. D. Meyer, PhD1,2,3; Velma L. Payne, PhD, MBA1,2,3; Derek W. Meeks, MD1,2,3; Radha Rao, MD2,3; Hardeep Singh, MD, MPH1,2,3

ABSTRACT

Importance

Little is known about the relationship between physicians’ diagnostic accuracy and their confidence in that accuracy.

Objective

To evaluate how physicians’ diagnostic calibration, defined as the relationship between diagnostic accuracy and confidence in that accuracy, changes with evolution of the diagnostic process and with increasing diagnostic difficulty of clinical case vignettes.

Design, Setting, and Participants

We recruited general internists from an online physician community and asked them to diagnose 4 previously validated case vignettes of variable difficulty (2 easier; 2 more difficult).

Cases were presented in a web-based format and divided into 4 sequential phases simulating diagnosis evolution: history, physical examination, general diagnostic testing data, and definitive diagnostic testing.

After each phase, physicians recorded 1 to 3 differential diagnoses and corresponding judgments of confidence.

Before being presented with definitive diagnostic data, physicians were asked to identify additional resources they would require to diagnose each case (ie, additional tests, second opinions, curbside consultations, referrals, and reference materials).

Main Outcomes and Measures

Diagnostic accuracy (scored as 0 or 1), confidence in diagnostic accuracy (on a scale of 0-10), diagnostic calibration, and whether additional resources were requested (no or yes).

Results

A total of 118 physicians with broad geographical representation within the United States correctly diagnosed 55.3% of easier and 5.8% of more difficult cases (P < .001).

Despite a large difference in diagnostic accuracy between easier and more difficult cases, the difference in confidence was relatively small (7.2 vs 6.4 out of 10, for easier and more difficult cases, respectively) (P < .001) and likely clinically insignificant.

Overall, diagnostic calibration was worse for more difficult cases (P < .001) and characterized by overconfidence in accuracy.

Higher confidence was related to decreased requests for additional diagnostic tests (P = .01); higher case difficulty was related to more requests for additional reference materials (P = .01).

Conclusions and Relevance

Our study suggests that physicians’ level of confidence may be relatively insensitive to both diagnostic accuracy and case difficulty.

This mismatch might prevent physicians from reexamining difficult cases where their diagnosis may be incorrect.
 
Messages
13,774
To be fair to doctors: I expect that these sorts of results would be similar for other professions self-rating their own competency.

It's a reminder that giving people power and authority over others will often lead to problems, and therefore the importance of informed consent and ensuring all individuals are provided as much autonomy as is possible, rather than having other people 'manage' them.
 
Last edited:

SOC

Senior Member
Messages
7,849
To be fair to doctors: I expect that these sorts of results would be similar for other professions self-rating their own competency.

It's a reminder that giving people power and authority over others will often lead to problems, and therefore the importance of informed consent and ensuring all individuals are provided as much autonomy as is possible, rather than having other people 'manage' them.
Not so true with engineers. ;) We don't do our job right and bridges fall down, buildings collapse, and space shuttles blow up.... oh wait....