#84 LAURA RUIS - Large language models are not zero-shot communicators [NEURIPS UNPLUGGED]

episode

Machine Learning Street Talk (MLST) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from Machine Learning Street Talk (MLST)

VAEs Are Energy-Based Models? [Dr. Jeff Beck] 46 min · 25 Jan 2026

Abstraction & Idealization: AI's Plato Problem [Mazviita Chirimuuta] 53 min · 23 Jan 2026

Why Every Brain Metaphor in History Has Been Wrong [SPECIAL EDITION] 42 min · 23 Jan 2026

Adversarial Examples and Data Modelling - Andrew Ilyas (MIT) 1h 28m · 22 Aug 2024

All episodes from Machine Learning Street Talk (MLST) →

Report any issue

Source

Recorded06 Dec 2022

Publisheranchor.fm ↗

Handlerec:2107410

Show notes

In this NeurIPSs interview, we speak with Laura Ruis about her research on the ability of language models to interpret language in context. She has designed a simple task to evaluate the performance of widely used state-of-the-art language models and has found that they struggle to make pragmatic inferences (implicatures). Tune in to learn more about her findings and what they mean for the future of conversational AI. Laura Ruis https://www.lauraruis.com/ https://twitter.com/LauraRuis BLOOM https://bigscience.huggingface.co/blog/bloom Large language models are not zero-shot communicators [Laura Ruis, Akbir Khan, Stella Biderman, Sara Hooker, Tim Rocktäschel, Edward Grefenstette] https://arxiv.org/abs/2210.14986 [Zhang et al] OPT: Open Pre-trained Transformer Language Models https://arxiv.org/pdf/2205.01068.pdf [Lampinen] Can language models handle recursively nested grammatical structures? A case study on comparing models and humans https://arxiv.org/pdf/2210.15303.pdf [Gary Marcus] Horse rides astronaut https://garymarcus.substack.com/p/horse-rides-astronaut [Gary Marcus] GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/ [Bender et al] On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? https://dl.acm.org/doi/10.1145/3442188.3445922 [janus] Simulators (Less Wrong) https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.