BARC talk by Raghavendra Selvan

Thursday, August 20, 2026, 14:00-15:00, Raghavendra Selvan, Assistant Professor at DIKU (UCPH), will be giving a BARC talk on "Characterizing Learning in Deep Neural Networks using the lens of Algorithmic Information Theory ".

Abstract:
Training large-scale deep neural networks (DNNs) is resource-intensive, necessitating effective model compression. The "learning as compression" hypothesis suggests that the training process induces structural regularities within weights that facilitate such compression. In this talk, we argue that algorithmic information theory (AIT)—specifically Kolmogorov-Chaitin-Solomonoff (KCS) complexity—offers a rigorous theoretical framework for measuring this structure in neural networks.

We introduce an AIT-inspired neural network parameterization that biases optimization toward algorithmically simpler solutions. Empirical results demonstrate that this approach consistently reduces the algorithmic complexity of neural networks during training while maintaining performance comparable to that of unconstrained models. This work provides the first scalable, tractable estimates of KCS complexity for large, non-binary objects, offering new insights into the underlying learning mechanisms of DNNs.

Bio:
Raghavendra Selvan (Raghav) is Tenure-Track Assistant Professor at the Machine Learning (ML) Section, Department of Computer Science, University of Copenhagen. His research spans efficient ML, ML for sciences, medical image analysis, and graph neural networks. He holds a PhD from the University of Copenhagen and is affiliated with Pioneer Center for AI (Denmark) and ELLIS. Raghav was born in Bangalore, India. RS is the author of the new book “Sustainable AI”.

Host:
Rasmus Pagh