Classification of material properties in fMRI
23.437, Saturday, May 11, 8:30 am - 12:30 pm, Orchid Ballroom
Elisabeth Baumgartner1, Christiane B. Wiebel1, Karl R. Gegenfurtner1; 1Department of General Psychology, Giessen University
The taxonomy of materials has been explored by means of machine classification (Liu et al., IEEE CVPR 2010) and also brain activation (Hiramatsu et al., NeuroImage 2011). Here we wanted to investigate whether information about material qualities like roughness or colorfulness can be found in the BOLD response to material images. We took photographs of 84 material samples and asked subjects to rate them with a seven-point Likert scale on different material qualities: colorfulness, roughness, texturedness, hardness, orderliness, and glossiness. We analyzed the material images according to the algorithm by Portilla and Simoncelli (IJCV 2000). In order to get parameters that grasped the color of our images, we also computed marginal statistics of their components in DKL space. A linear multivariate classifier was applied to the image statistics in order to discriminate between images with high and low ratings. This classification procedure achieved between 69% and 98% correct for the different properties. To see if the information contained in the image statistics would be reflected in brain activation patterns we scanned six subjects with fMRI while they were viewing the material images. A classifier was then applied to the voxel values of visually responsive voxels. We found classification accuracy to be significantly better than chance for three of six material qualities: colorfulness (63%, p<0.001), roughness (60%, p<0.05), and texturedness (64%, p<0.001). Our results demonstrate that information about the qualities of materials is present in the image statistics by Portilla and Simoncelli as well as fMRI activation patterns. In addition, we checked if the classification accuracy of the image statistics classifier would improve using the labels given by the fMRI classifier. At least for roughness, this was the case (69% vs 74%, p<0.05). This suggests that there might be a link between the observed brain activation and the image statistics.