Dr. Paul Lessard - Categorical/Structured Deep Learning

episode

Machine Learning Street Talk (MLST) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from Machine Learning Street Talk (MLST)

VAEs Are Energy-Based Models? [Dr. Jeff Beck] 46 min · 25 Jan 2026

Abstraction & Idealization: AI's Plato Problem [Mazviita Chirimuuta] 53 min · 23 Jan 2026

Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION] 42 min · 23 Jan 2026

Adversarial Examples and Data Modelling - Andrew Ilyas (MIT) 1h 28m · 22 Aug 2024

All episodes from Machine Learning Street Talk (MLST) →

Report any issue

Source

Recorded01 Apr 2024

Publisheranchor.fm ↗

Handlerec:2107257

Show notes

Dr. Paul Lessard and his collaborators have written a paper on "Categorical Deep Learning and Algebraic Theory of Architectures". They aim to make neural networks more interpretable, composable and amenable to formal reasoning. The key is mathematical abstraction, as exemplified by category theory - using monads to develop a more principled, algebraic approach to structuring neural networks. We also discussed the limitations of current neural network architectures in terms of their ability to generalise and reason in a human-like way. In particular, the inability of neural networks to do unbounded computation equivalent to a Turing machine. Paul expressed optimism that this is not a fundamental limitation, but an artefact of current architectures and training procedures. The power of abstraction - allowing us to focus on the essential structure while ignoring extraneous details. This can make certain problems more tractable to reason about. Paul sees category theory as providing a powerful "Lego set" for productively thinking about many practical problems. Towards the end, Paul gave an accessible introduction to some core concepts in category theory like categories, morphisms, functors, monads etc. We explained how these abstract constructs can capture essential patterns that arise across different domains of mathematics. Paul is optimistic about the potential of category theory and related mathematical abstractions to put AI and neural networks on a more robust conceptual foundation to enable interpretability and reasoning. However, significant theoretical and engineering challenges remain in realising this vision. Please support us on Patreon. We are entirely funded from Patreon donations right now. https://patreon.com/mlst If you would like to sponsor us, so we can tell your story - reach out on mlstreettalk at gmail Links: Categorical Deep Learning: An Algebraic Theory of Architectures Bruno Gavranović, Paul Lessard, Andrew Dudzik, Tamara von Glehn, João G. M. Araújo, Petar Veličković Paper: https://categoricaldeeplearning.com/ Symbolica: https://twitter.com/symbolica https://www.symbolica.ai/ Dr. Paul Lessard (Principal Scientist - Symbolica) https://www.linkedin.com/in/paul-roy-lessard/ Interviewer: Dr. Tim Scarfe TOC: 00:00:00 - Intro 00:05:07 - What is the category paper all about 00:07:19 - Composition 00:10:42 - Abstract Algebra 00:23:01 - DSLs for machine learning 00:24:10 - Inscrutibility 00:29:04 - Limitations with current NNs 00:30:41 - Generative code / NNs don't recurse 00:34:34 - NNs are not Turing machines (special edition) 00:53:09 - Abstraction 00:55:11 - Category theory objects 00:58:06 - Cat theory vs number theory 00:59:43 - Data and Code are one in the same 01:08:05 - Syntax and semantics 01:14:32 - Category DL elevator pitch 01:17:05 - Abstraction again 01:20:25 - Lego set for the universe 01:23:04 - Reasoning 01:28:05 - Category theory 101 01:37:42 - Monads 01:45:59 - Where to learn more cat theory

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.