Color, Light and Materials: Cones to cognition

Talk Session: Sunday, May 21, 2023, 5:15 – 7:15 pm, Talk Room 2
Moderator: Karen Schloss, University of Wisconsin

Schedule of Events | Search Abstracts | Symposia | Talk Sessions | Poster Sessions | Abstract Numbering

Talk 1, 5:15 pm, 35.21

Psychophysical and image-based characterization of macular pigment using structured light

Andrew E. Silva¹ (), Connor Kapahi^2,3, David G. Cory^3,5, Mukhit Kulmaganbetov⁴, Melanie Mungalsingh¹, Taranjit Singh⁴, Benjamin Thompson^1,4, Dmitry A. Pushin^2,3,4, Dusan Sarenac^3,4; ¹School of Optometry and Vision Science, University of Waterloo, Waterloo, ON, Canada, ²Department of Physics and Astronomy, University of Waterloo, Waterloo, ON, Canada, ³Institute for Quantum Computing, University of Waterloo, Waterloo, ON, Canada, ⁴Centre for Eye and Vision Research, 17W Science Park, Hong Kong, ⁵Department of Chemistry, University of Waterloo, Waterloo, ON, Canada

Macular pigment is thought to underlie entoptic percepts that enable human discrimination of polarized light. Therefore, polarization sensitivity may be a useful probe of macular pigment density and the risk of future macular disease. Structured light beams formed through the spin coupled orbital angular momentum states exhibit spatially dependent polarization and are therefore ideally suited to quantify human polarization sensitivity. When fixating at the center of these beams, a polarization-defined entoptic percept resembling radial spokes is observed. Using such structured light, we investigated whether we could characterize the portion of retina sensitive to polarization in healthy observers and observers with subclinical macular degeneration. In Experiment 1, the beam was presented to 23 healthy participants for 500ms per trial, rotating clockwise or counterclockwise. Participants indicated the direction of rotation. A circular mask with a varying radius was placed at fixation, thus the task was performed at varying eccentricities. The radius of the mask was controlled by a 2-up, 1-down thresholding staircase to estimate the size of the mask eliciting 71% accuracy. In addition, a fundus image using structured light was taken alongside a standard fundus image to register the psychophysical threshold to retinal landmarks, allowing the threshold to be expressed in visual angle units. In Experiment 2, normal participants and participants with subclinical macular degeneration performed a similar task using multiple mask shapes to characterize polarization sensitivity more fully. In healthy eyes, the mask size threshold ranged between 1° and 9° (mean = 4.6° ± 0.6°). In eyes exhibiting subclinical macular degeneration, a full-field mask with a ring of visible structured light elicited the best and most reliable performance, consistent with a selective pigment deficit in the central macula. Overall, our results indicate that structured light may be a useful tool for probing human macular pigment via polarization sensitivity.

Acknowledgements: This work was supported by the Canada First Research Excellence Fund (CFREF). This work is also supported by InnoHK and the Hong Kong Government.

Talk 2, 5:30 pm, 35.22

A deep convolutional neural network trained to infer surface reflectance is deceived by mid-level lightness illusions

Jaykishan Patel¹ (), Alban Flachot¹, Javier Vazquez-Corral², David H. Brainard³, Thomas S. A. Wallis⁴, Marcus A. Brubaker¹, Richard F. Murray¹; ¹York University, ²Universitat Autònoma de Barcelona, ³University of Pennsylvania, ⁴Technische Universität Darmstadt

A long-standing view is that lightness illusions are by-products of strategies employed by the visual system to stabilize its perceptual representation of surface reflectance against changes in illumination. Computationally, one such strategy is to infer reflectance from the retinal image, and to base the lightness percept on this inference. CNNs trained to infer reflectance from images have proven successful at solving this problem under limited conditions. To evaluate whether these CNNs provide suitable starting points for computational models of human lightness perception, we tested a state-of-the-art CNN on several lightness illusions, and compared its behaviour to prior measurements of human performance. We trained a CNN (Yu & Smith, 2019) to infer reflectance from luminance images. The network had a 30-layer hourglass architecture with skip connections. We trained the network via supervised learning on 100K images, rendered in Blender, each showing randomly placed geometric objects (surfaces, cubes, tori, etc.), with random Lambertian reflectance patterns (solid, Voronoi, or low-pass noise), under randomized point+ambient lighting. The renderer also provided the ground-truth reflectance images required for training. After training, we applied the network to several visual illusions. These included the argyle, Koffka-Adelson, snake, White’s, checkerboard assimilation, and simultaneous contrast illusions, along with their controls where appropriate. The CNN correctly predicted larger illusions in the argyle, Koffka-Adelson, and snake images than in their controls. It also correctly predicted an assimilation effect in White's illusion. It did not, however, account for the checkerboard assimilation or simultaneous contrast effects. These results are consistent with the view that at least some lightness phenomena are by-products of a rational approach to inferring stable representations of physical properties from intrinsically ambiguous retinal images. Furthermore, they suggest that CNN models may be a promising starting point for new models of human lightness perception.

Acknowledgements: Funded in part by grants from NSERC and VISTA to R.F.M.

Talk 3, 5:45 pm, 35.23

Dynamic achromatic color computation based on fixational eye movements and edge integration

Michael Rudd¹ (); ¹University of Nevada, Reno

Achromatic color perception exhibits a well-known asymmetry in which luminance increments and decrements have different subjective magnitudes. This asymmetry influences the perception of simple laboratory stimuli, real-world stimuli, and visual illusions, and it has important implications for color models in neuroscience and technology. Here, the magnitudes and spatial properties of this asymmetry are modeled by a neural theory of lightness computation in which ON- and OFF-center neurons are activated in the course of fixational eye movements (FEM). The ON and OFF cell activations encode local luminance ratios, raised to a power that differs for ON and OFF cells, consistent with physiological data from macaque LGN, and with the Naka-Rushton neural response function under non-saturating conditions. OFF cells respond linearly, while ON cells exhibit a compressive (power law) response. Both ON and OFF cells are assumed to possess gaussian spatial center and surround mechanisms that differ in size, as in the classical difference-of-gaussians (DOG) receptive field model, except that, here, the inhibitory mechanism is divisive rather than subtractive. Corollary discharge signals that encode the direction of fixational eye movements are combined with transient ON and OFF cell responses to produce local contrast signals that are sensitive to FEM direction. These local, directionally-sensitive, ON and OFF signals are log-transformed, then spatially integrated in the direction of the FEM by long-range cortical ON and OFF networks, whose outputs are combined to produce a unitary achromatic color signal. The achromatic color signal is normalized to achieve highest lightness anchoring, resulting in a 2D map of perceived reflectance that models the achromatic color percept. The model is demonstrated through computer simulations to account for psychophysical data on dynamic range compression in the Staircase Gelb Illusion, lightness/darkness asymmetries in simultaneous contrast, edge integration in lightness, perceptual filling-in, the Chevreul illusion, and perceptual fading of stabilized images.

Talk 4, 6:00 pm, 35.24

Color discrimination and chromatic balance perception after adaptation to natural and color-reflected scenes.

Beata Wozniak¹, John Maule¹, Anna Franklin¹, Jenny Bosten¹; ¹University of Sussex

The chromatic distributions of natural scenes are biased in a blue-yellow direction (Nascimento et al., 2002, JOSA A, 19(8), 1484–1490). Existing research suggests a possible link between natural scene statistics and color discrimination (e.g., Bosten et al., 2015, J. Vis, 15(16), 5). Here we investigate the impact of short-term adaptation to natural and color-reflected images on color perception. In Experiment 1 we measured color discrimination ellipses. In Experiment 2 we measured perception of ‘chromatic balance’ of a distribution of colors. To capture the ‘chromatic diet’ of our participants, we used head-mounted, color-calibrated GoPro cameras to collect images of typical local outdoor scenes. From these images we created two sets of adapting images. The chromaticities of one set of ‘natural’ images were reflected through the S/(L+M) axis of the MacLeod-Boynton chromaticity diagram to create a second set of ‘color-reflected’ images. In both experiments, participants adapted to natural and color-reflected images in separate blocks. In Experiment 1 we measured color discrimination thresholds along 8 hue axes using 4AFC. In Experiment 2 the stimuli were chromatic 1/f noise patterns with constant 1/f luminance noise. The chromaticities of each pattern were distributed elliptically along 1 of 8 color directions in the MacLeod-Boynton chromaticity diagram and had 1 of 4 levels of chromatic bias (ellipse elongation). Participants performed a paired comparison task selecting patterns that appeared to contain more colors and where all the colors were evenly present. Results of Experiment 1 showed that adaptation to colour-reflected scenes changes the elongation of color discrimination ellipses (BF₁₀>1x10^7): after adapting to natural images participants showed higher thresholds for blue and yellow. Results of Experiment 2 showed that after adapting to natural images participants tended to select noise patterns biased in the blue-yellow direction as chromatically balanced. This bias was reduced after adapting to color-reflected images.

Talk 5, 6:15 pm, 35.25

Predicting gloss sensitivity across variations in surface shape, illumination and viewpoint

Jacob R. Cheeseman¹ (), James A. Ferwerda², Takuma Morimoto^1,3, Roland W. Fleming^1,4; ¹Justus Liebig University Giessen, ²Rochester Institute of Technology, ³University of Oxford, ⁴Center for Mind, Brain and Behavior, Marburg, Germany

The apparent glossiness of a surface depends not only on the intrinsic reflectance properties of the surface itself, but also on various extrinsic scene factors that drastically alter the pattern of reflected light. For example, a perfectly visible reflectance difference in ideal viewing conditions can become invisible in degenerate viewing conditions. This makes it challenging to define thresholds for discriminating differences in specular reflectance characteristics (‘gloss sensitivity’). Here we tested how well image-computable visual discrimination models predict human gloss discrimination performance across diverse viewing conditions, as a key step towards establishing a principled definition of gloss sensitivity. While previous studies of gloss perception have typically varied surface reflectance along with one other extrinsic variable of interest, here we focus exclusively on the effect of surface shape, illumination and viewpoint on the perception of a single, fixed difference in microscopic surface roughness. Smooth surfaces yield sharp reflections while rougher surfaces produce blurrier reflections, and a less glossy appearance. We rendered pairs of images depicting an object with a fixed specular reflectance and high or low surface roughness in a variety of shape, illumination and viewpoint combinations, and collected pair comparisons from human observers (N=100) in an online experiment. These judgments were used to rank each scene according to how often each image pair appeared to depict a larger difference in gloss. We find that these judgments are highly consistent across observers, and that the ranking of scenes is well predicted by the High Dynamic Range Visible Difference Predictor, a widely used model of image quality and perceived image differences. This result suggests that such a metric could be used to estimate upper and lower bounds of gloss sensitivity across viewing conditions, which is an important step towards establishing a standard measurement framework for gloss.

Acknowledgements: Funded by H2020-MSCA-ITN-2017 ‘DyViTo–project number 765121, DFG–project number 222641018–SFB/TRR 135 TP C1, ERC Consolidator Award ‘SHAPE’–project number ERC-CoG-2015-682859, and by the Excellence Program of the HMWK–project ‘The Adaptive Mind’.

Talk 6, 6:30 pm, 35.26

Object-based computations for color constancy

Laysa Hedjar¹ (), Raquel Gil Rodríguez¹, Matteo Toscani², Dar'ya Guarnera³, Giuseppe Claudio Guarnera^3,4, Karl R. Gegenfurtner¹; ¹Justus-Liebig-Universität Gießen, Germany, ²Bournemouth University, UK, ³Norwegian University of Science and Technology, Gjøvik, Norway, ⁴University of York, UK

Color constancy has been shown to be high in nearly-natural situations. Yet, studying the cues used by human observers is very difficult when dealing with real objects and illuminants. We use Virtual Reality to construct realistic, immersive environments that are easily manipulable in real-time. We created an outdoor forest scene and an indoor office scene under five colored illuminants and selectively silenced individual cues to measure their impact on color constancy. For each trial, observers chose which of five test objects appeared most similar to an achromatic object previously shown. Objects ranged from a zero-constant tristimulus match to a perfect reflectance match and beyond (0-133% constancy). Similar to Kraft and Brainard (1999), we investigated local context, maximum flux, and mean color as cues. To eliminate local context, we placed a constant, rose-colored leaf under each test object. For maximum flux, we ensured that the brightest object in the scene remained constant across illuminations. To preserve the mean reflected light, we either shifted the reflectances of all objects or added new objects in the color direction opposite the illumination change. With all cues present, color constancy indices (CCIs) were high for both indoor and outdoor scenes (>75%). Silencing the local context and maximum flux mechanisms lowered CCIs slightly. In line with previous findings, constancy was massively impaired when keeping the average color constant by modifying object reflectances. However, when new objects were introduced instead, there was a modest reduction, with several observers showing no impairment at all. All results were consistent in both scenes. Our results show that VR can be a valuable tool for studying color constancy, and that computations underlying the gray-world mechanism do not simply operate on a pixel-by-pixel basis. Rather, observers seem to segment the scene and use the parts that are particularly diagnostic of the illumination.

Acknowledgements: Supported by ERC Advanced Grant Color 3.0 (project no. 884116)

Talk 7, 6:45 pm, 35.27

How do people map colors to concepts? Modeling assignment inference as evidence accumulation

Kushin Mukherjee¹ (), Laurent Lessard², Karen B. Schloss¹; ¹University of Wisconsin-Madison, ²Northeastern University

When interpreting information visualizations where distinct colors represent different concepts, observers map colors to concepts using a process called assignment inference (Schloss et al., 2018). In assignment inference, people do not simply assign concepts to the most strongly associated color (local assignment). Instead, they make assignments that maximize the relative associations across all color-concept pairs in the encoding system (global assignment). It would be intractable to compare all color-concept pairings to determine the optimal assignment as the number of colors increases, so how might observers perform assignment inference? We propose observers use a heuristic to solve this problem by finding the single color that has the most 'evidence' for being mapped to the target concept relative to other colors in the scope. This approach can be modeled using the Linear Ballistic Accumulator (LBA) model of decision-making where each color is modeled using an accumulator, which independently ‘races’ to reach a response threshold for a target concept. A critical factor in determining the winning accumulator is its rate of evidence accumulation (drift rate). We investigated what factors determine drift rate. First, we collected data on an assignment inference task. Participants saw dot plots with four colored dots (each representing a different concept) and indicated which color represented a target concept (4 target concepts x 15 color palettes x 8 repetitions). Next, we modeled responses/RTs using an LBA model with different accumulators for each color-concept pair. We tested whether patterns of drift rates could be explained by direct associations with the target concept (local) and relative associations across concepts (global). A mixed-effects model showed that both factors explained independent variance in drift rates (ps<.001). Thus, observers used a heuristic that incorporates local and global information to assign colors to concepts, without having to evaluate every possible color-concept assignment in assignment inference.

Acknowledgements: NSF award BCS-1945303 to K.B.S.

Talk 8, 7:00 pm, 35.28

Lexical effects on remembered colors

Delwin Lindsey¹ (), Prutha Deshpande¹, Angela Brown¹; ¹The Ohio State University

How do people encode information about color in memory? Prior work suggested that color categories might play an important role, in conjunction with colorimetric properties of the colors to be remembered. The exact role of the observer’s own color vocabulary in memory has been less clear. Here, we report a study of color memory, conducted using calibrated I-Pads, within-subjects, on 10 color-normal observers. First, observers grouped 1625 Munsell colors into categories, providing focal colors and unconstrained color terms for the categories. In a later session, they performed isometric color matches to 5-second stimuli following 10-second and 5-minute retention intervals, with control data taken under simultaneous presentation. Color matches were by the method of adjustment, using a 3-D CIELAB structured color palette. Importantly, observers named the remembered color after each 5-minute trial. Matches were more variable for 10-sec and 5.min delays than in the simultaneous condition and distances from the test color (“bias”) also increased from ∆E=2.8 in the simultaneous condition to about ~∆E=9 in the delay conditions, generally in the direction of increased saturation. Both variability and bias varied across test colors, and both remained approximately constant across the 10-sec and 5-min delays. To test for lexical effects, a novel “attraction score” measured the association between the direction of the remembered color bias and the observer’s personal focal colors. Attraction scores for the pre-experiment color-naming data were modest for most colors, but the 5-min matches were highly significantly attracted to the focal colors of color terms provided after each trial. The color-categorical and lexical properties of the stimuli, as measured before the experiment, were less important than the colorimetric properties of the stimuli and the color names provided afterwards. This suggests that color memory is not strongly related to pre-existing, invariant lexical properties of colors.