Kexin Cindy Luo^1,2 (), George Alvarez^1,2, Talia Konkle^1,2; ¹Harvard University, ²Kempner Instutite for the Study of Natural and Artificial Intelligence

The human visual system contains rich feedforward, feedback, and bypass pathways that shape how category information is represented across the ventral stream. Motivated by these biological motifs, many deep neural networks now incorporate recurrent circuitry, but it remains unclear how these architectural choices reshape category representations. Here, we use representational geometry analyses to ask how models with different feedback motifs change the global layout of categories in deep networks, relative to purely feedforward architectures. We first examined a standard feedforward AlexNet backbone and two variants with multiplicative or additive long-range top-down connections (LRM/LRA; Konkle & Alvarez, 2023). Comparing the distribution of pairwise cosine distances between category prototypes, the purely feedforward AlexNet showed a spread-out, flat distribution of between-category distances (M=.81, SD=.37). In contrast, both LRM and LRA models exhibited sharply peaked distributions tightly clustered near 1 (M=.96, SD=.12), approximating a near-orthogonal simplex in decision space. This divergence was not present at the penultimate layer, where all three models showed similar distributions with high cross-model RSA (~r=.96). To determine whether near-orthogonality is a signature general to any model with recurrent circuitry, we examined a broader set of networks, including CORnet-RT (local additive feedback), ResNet-50 (skip forward connections), and VGG16 (pure feedforward). Contrary to our initial expectation, we found that only ResNet-50 showed a near-orthogonal category geometry at the logit layer, whereas CORnet-RT and VGG16 resembled the flatter feedforward AlexNet profile. Interestingly, both Resnet50 and the LRM/LRA models have a channel-to-channel bypass motif that mixes information across non-adjacent hierarchical layers. Taken together, these results raise the possibility that bypass connections provide pressure on the decision stage to form nearly orthogonal category prototypes. This may have benefits for simplified downstream readout while still allowing earlier layers to maintain richer similarity structure. The mechanism underlying this potential bypass-geometry effect is currently unknown.

Acknowledgements: This work was supported by the Kempner Institute Graduate Fellowship (to K.C.L.), CRCNS-2309041 (to T.K., G.A., & H.P.), and funding from the Kempner Institute (to T.K.).