#030 Multi-Armed Bandits and Pure-Exploration (Wouter M. Koolen)

episode

Machine Learning Street Talk (MLST) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from Machine Learning Street Talk (MLST)

VAEs Are Energy-Based Models? [Dr. Jeff Beck] 46 min · 25 Jan 2026

Abstraction & Idealization: AI's Plato Problem [Mazviita Chirimuuta] 53 min · 23 Jan 2026

Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION] 42 min · 23 Jan 2026

Adversarial Examples and Data Modelling - Andrew Ilyas (MIT) 1h 28m · 22 Aug 2024

All episodes from Machine Learning Street Talk (MLST) →

Report any issue

Source

Recorded20 Nov 2020

Publisheranchor.fm ↗

Handlerec:2107531

Show notes

This week Dr. Tim Scarfe, Dr. Keith Duggar and Yannic Kilcher discuss multi-arm bandits and pure exploration with Dr. Wouter M. Koolen, Senior Researcher, Machine Learning group, Centrum Wiskunde & Informatica. Wouter specialises in machine learning theory, game theory, information theory, statistics and optimisation. Wouter is currently interested in pure exploration in multi-armed bandit models, game tree search, and accelerated learning in sequential decision problems. His research has been cited 1000 times, and he has been published in NeurIPS, the number 1 ML conference 14 times as well as lots of other exciting publications. Today we are going to talk about two of the most studied settings in control, decision theory, and learning in unknown environment which are the multi-armed bandit (MAB) and reinforcement learning (RL) approaches - when can an agent stop learning and start exploiting using the knowledge it obtained - which strategy leads to minimal learning time 00:00:00 What are multi-arm bandits/show trailer 00:12:55 Show introduction 00:15:50 Bandits 00:18:58 Taxonomy of decision framework approaches 00:25:46 Exploration vs Exploitation 00:31:43 the sharp divide between modes 00:34:12 bandit measures of success 00:36:44 connections to reinforcement learning 00:44:00 when to apply pure exploration in games 00:45:54 bandit lower bounds, a pure exploration renaissance 00:50:21 pure exploration compiler dreams 00:51:56 what would the PX-compiler DSL look like 00:57:13 the long arms of the bandit 01:00:21 causal models behind the curtain of arms 01:02:43 adversarial bandits, arms trying to beat you 01:05:12 bandits as an optimization problem 01:11:39 asymptotic optimality vs practical performance 01:15:38 pitfalls hiding under asymptotic cover 01:18:50 adding features to bandits 01:27:24 moderate confidence regimes 01:30:33 algorithms choice is highly sensitive to bounds 01:46:09 Post script: Keith interesting piece on n quantum http://wouterkoolen.info https://www.cwi.nl/research-groups/ma... #machinelearning

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.