Courtney M. King¹ (), Elissa Aminoff¹, Daniel Leeds¹; ¹Fordham University

Modern computer vision offers a wide variety of models for recognition and object detection, with some being similar in design to biological vision. Partial occlusions have been shown to have a negative impact on performance in both cases. Similarly, color can be used to create illusions in images and instances of camouflage, which can also have an impact on these two tasks. This work explores several types of occlusions that can make objects appear illusory due to their color with respect to the greater image context. Specifically, we study cases where the occluder blends in with an object or when the occluder blends in with the background. These cases are interesting as they find commonality with optical illusions in human perception. In experiments, these artificial occluders are applied to 80 distinct objects as horizontal lines, with an occlusion ratio of approximately 0.25. Current experiments explore the impact of these types of occlusions on object recognition for several state-of-the-art computer vision models, ranging from simple convolutional neural networks (CNNs) to transformers to more recent “transformer-like” CNNs. This work also offers a look inside these diverse models to visualize how the occlusion-based errors propagate through the networks to the final layer. Current results find that transformer-based models are far more robust to these types of occlusions when compared to standard CNN models, like ResNet, but that more recent CNNs like ConvNeXt, show similar robustness to transformers. A second more surprising observation is that these color-based occlusions show significantly higher performance when compared to black baseline occlusions in the CNN models, but similar performance in both the transformer and more recent CNN models. These results give more knowledge about the internal nature of neural networks as well as their robustness to specific types of partial occlusions which are not unseen in real-world scenes.