How Deep Learning Finally Cracked Messy Tables - Frank Hutter

episode

Machine Learning Street Talk (MLST) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from Machine Learning Street Talk (MLST)

VAEs Are Energy-Based Models? [Dr. Jeff Beck] 46 min · 25 Jan 2026

Abstraction & Idealization: AI's Plato Problem [Mazviita Chirimuuta] 53 min · 23 Jan 2026

Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION] 42 min · 23 Jan 2026

Adversarial Examples and Data Modelling - Andrew Ilyas (MIT) 1h 28m · 22 Aug 2024

All episodes from Machine Learning Street Talk (MLST) →

Report any issue

Source

Recorded23 Sep 2026

Publisheranchor.fm ↗

Handlerec:20502572

Show notes

Frank Hutter, co-founder of Prior Labs, talks about TabPFN, a tabular foundation model that makes predictions in a single forward pass, and the research behind it.TabPFN is pre-trained on synthetic datasets drawn from a prior over structural causal models, rather than on real data. At prediction time it takes the whole training table as context and outputs an approximation of the Bayesian posterior predictive distribution, without per-dataset training or hyperparameter search. Frank explains how this grew out of his earlier work on AutoML and neural architecture search, how the priors are built and revised, and why tabular data was hard for deep learning for so long.The conversation also covers the TabArena benchmark, how the architecture changed from TabPFN v1 to v3, scaling to larger tables, using the model with coding agents, test-time compute, Google's TabFM, causal inference and interventions, and relational data. At the end, a short update Frank recorded after the interview covers the TabPFN-3.5 release.TOC:00:00 Introduction00:44 Welcome and Frank's background02:05 Why tabular data was hard for deep learning10:17 Pre-training on synthetic data12:52 The TabArena benchmark19:28 From AutoML to neural architecture search26:34 TabPFN as a learned algorithm30:50 Bayesian prediction in one forward pass39:37 Scaling to larger tables47:48 Using TabPFN with coding agents57:47 Output heads and architecture from v1 to v31:05:29 Test-time compute and adaptation1:13:32 Google's TabFM1:16:53 How the priors are designed1:18:40 Correlation, causation and interventions1:35:22 Relational and multimodal data1:38:31 Use in organisations1:46:38 The open research arm1:50:21 Update: TabPFN-3.5REFS:TabPFN v2, Nature (Hollmann et al., 2025)https://www.nature.com/articles/s41586-024-08328-6Transformers Can Do Bayesian Inference (Müller et al.)https://arxiv.org/abs/2112.10510TabArena (Erickson et al.)https://arxiv.org/abs/2506.16791AutoGluon-Tabular (Erickson et al.)https://arxiv.org/abs/2003.06505Beyond IID: How General Are Tabular Foundation Models, Really?https://arxiv.org/abs/2606.30410Neural Architecture Search: A Survey (Elsken, Metzen & Hutter)https://arxiv.org/abs/1808.05377Auto-WEKA (Thornton et al.)https://www.cs.ubc.ca/~hutter/papers/AutoWEKA-KDD2013.pdfTabPFN v1 (Hollmann et al., 2022)https://arxiv.org/abs/2207.01848TabPFN-3 technical reporthttps://arxiv.org/abs/2605.13986TabPFN-2.5 reporthttps://arxiv.org/abs/2511.08667CAAFE (Hollmann et al.)https://arxiv.org/abs/2305.03403TabICL (Qu et al.)https://arxiv.org/abs/2502.05564TabICLv2 (Qu et al.)https://arxiv.org/abs/2602.11139Google TabFMhttps://research.google/blog/introducing-tabfm-a-zero-shot-foundation-model-for-tabular-data/TALENT benchmark (Ye et al.)https://arxiv.org/abs/2407.00956Do-PFN (Robertson et al.)https://arxiv.org/abs/2506.06039CausalPFN (Balazadeh et al.)https://arxiv.org/abs/2506.07918Causal Foundation Models with Partial Graphs (Reuter et al.)https://arxiv.org/abs/2602.14972RelBench (Robinson et al.)https://arxiv.org/abs/2407.20060RelArena-α, TabPFN-Rel and RPIhttps://arxiv.org/abs/2608.16319TabPFN on GitHubhttps://github.com/PriorLabs/TabPFNTabPFN-3.5 technical reporthttps://arxiv.org/abs/2609.17895Otto Group Product Classification Challenge (Kaggle, 2015)https://www.kaggle.com/competitions/otto-group-product-classification-challengePrior Labs:TabPFN-3.5: https://priorlabs.ai/tabpfn-3-5Careers at Prior Labs: https://priorlabs.ai/careers

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.