Jeffery Andrade¹ (), George Alvarez¹, Talia Konkle¹; ¹Harvard University

While humans easily recognize innumerable object categories, the underlying computational paths from retina to category-level representations are still being unraveled. Convolutional neural networks (CNNs) like AlexNet have remarkable competence in visual categorization, and thus offer a unique case study for understanding the hierarchical routing of visual information. Extending work from Hamblin et al., 2023, here we develop a method to extract the relevant connections involved in the computation of each output category, and assess the effectiveness of this sparser sub-network. The key idea is that not all connections are necessarily involved in the computation of any one category; thus, for each of the 1000 category-level output units in the Alexnet, our algorithm assigns scores to connections based on their contribution to the category unit's outputs and prunes the lowest-scored connections to a specified sparsity. Our goal is to identify the sparsest circuit through the network that still maintains the original function. To evaluate how well the extracted circuits reflect the output unit’s original functionality, we introduce a new metric–circuit substitution accuracy (CSA). We find that circuits need only 5.0% (median) of connections to achieve 85% of the unpruned CSA. Surprisingly, we observed that CSA initially increases with pruning and often actually exceeds the unpruned baseline at its peak (median = 188.0% of baseline) with a median of just 13.3% of connections remaining. We hypothesize that the full network must employ inhibition to negotiate between competing, interfering pathways. Finally, the “anatomical overlap” amongst these category circuits ranged from <1% to >99% shared circuitry, revealing a range of implicit modularization in the network's categorical processing routes. Broadly, this work presents a novel method for gaining insight into the functional neuroanatomy of neural networks, and offers a foundation for understanding the hierarchical computations involved in the emergence of category-level information in visual systems.

Acknowledgements: Kempner Institute for the Study of Natural and Artificial Intelligence