Zhihang Ren¹ (), Julien Vignoud², Zixuan Wang¹, Heath Rutledge-Jukes⁴, Tejas C. Sekhar⁴, Kathy Fang³, Erik P. Duhaime⁴, David Whitney¹; ¹University of California, Berkeley, ²École Polytechnique Fédérale de Lausanne (EPFL), ³Golden State Dermatology, Albany, CA, USA, ⁴Centaur Labs, Boston, MA, USA

Previous work has confirmed that clinicians vary in their medical image perception proficiency, impacting diagnostic performance and patient outcomes. However, previous work focuses more on global metrics of performance, such as accuracy, specificity, and so on. This indicates that some clinicians are better than others, but it makes no prediction about how a clinician will perform given a particular image. Here, we tested the possibility that individual clinicians have idiosyncratic patterns of bias at the image-level. To address this, we analyzed around 750k malignancy discrimination judgments of 7,818 skin lesion images, from a pool of 1,173 observers including those with medical image training. We utilized a deep learning encoding algorithm to cluster similar skin lesions based on deep semantic features. This allowed us to analyze individual differences locally at the image-cluster level. First, we identified significant individual differences in image-selective diagnostic performance, revealed by patterns of bias in perceptual judgments that were idiosyncratic and stimulus-specific. When visualized, these patterns of perceptual bias reveal fingerprints of perceptual bias that characterize individual clinicians. Second, we confirmed that proficiency is associated with increased agreement among medical professionals. Finally, by isolating the most ambiguous images, we found that proficiency is associated with stronger idiosyncratic biases: as images become more challenging, more proficient individual observers have increasingly unique and precise patterns of bias in their perceptual decisions. Our results suggest a potential systematic cause of diagnostic errors at the level of individual clinicians, they expose potential mitigation strategies, and they have implications for multi-reader scenarios.

Acknowledgements: This work has been supported by National Institutes of Health (NIH) under grant #R01CA236793. And we appreciate Centaur Labs offers us the diagnostic data for analysis.