DreamOn: Enhancing Deep Learning in Medical Imaging with REM-Dream-Inspired Data Augmentation

Poster Presentation 56.448: Tuesday, May 21, 2024, 2:45 – 6:45 pm, Pavilion
Session: Object Recognition: Structure of categories

Luc Lerch1,2 (), Lukas S. Huber1,2,4,5, Amith Kamath2, Walter Senn1, Mauricio Reyes2, Alexander Pöllinger3, Verena Obmann3, Florian Dammann3, Aurélie Pahud de Mortanges3; 1Computational Neuroscience Group, Department of Physiology, University of Bern, Bern, Switzerland., 2Medical Image Analysis Group, ARTORG Centre for Biomedical Research, University of Bern, Bern, Switzerland., 3Inselspital, University Hospital, University of Bern, Bern, Switzerland., 4Cognition, Perception and Research Methods, Department of Psychology, University of Bern, Bern, Switzerland., 5Neural Information Processing Group, Department of Computer Science, University of Tübingen, Tübingen, Germany.

The efficacy of deep learning (DL) models in medical image analysis is significantly influenced by image quality. Image quality variation arises from diverse sources, including differences in imaging equipment, patient-specific factors like movement, and biological variability. Consequently, robustness against noise is a crucial factor for the application of DL models in medical image analysis. Here we investigate the impact of various common data augmentation strategies on the robustness of a ResNet-18 model in classifying breast ultra sound images, and benchmark the performance against trained human radiologists. Additionally, we introduce 'DreamOn', a data augmentation strategy employing a generative adversarial network (GAN) approach to create synthetic images inspired by generation of visual experience during REM sleep. The proposed method involves generating interpolations of training images that mimic the recombination of episodic memories observed in human dreaming. We evaluate the effectiveness of DreamOn in improving model robustness across diverse datasets, each treated with six increasing levels of either Gaussian, speckle, or salt-and-pepper noise, and compare its performance with other off-the-shelf data augmentation methods. Collected data indicates that while standard techniques enhance model robustness, they fall short in high noise environments where radiologists excel. Employing the DreamOn data augmentation, we were able to narrow this robustness gap. The presented study underscores the potential of integrating biologically inspired data augmentation techniques in DL models for medical image analysis. It highlights the importance of considering human-like perceptual and cognitive processes in developing AI tools, particularly in fields where expert human judgment remains the gold standard. The findings are particularly relevant for vision scientists interested in the intersection of artificial intelligence, human cognition, and medical imaging.