“Schmidt Sciences’ request for proposals on the Science of Trustworthy AI” by James Fox

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded26 Feb 2026

Publisherfeeds.type3.audio ↗

Handlerec:16184195

Show notes

Schmidt Sciences’ request for proposals on the Science of Trustworthy AI Schmidt Sciences invites proposals for the Science of Trustworthy AI program, which supports technical research that improves our ability to understand, predict, and control risks from frontier AI systems while enabling their trustworthy deployment. This program supports technical research that improves our ability to understand, predict, and control risks from frontier AI systems whilst enabling their trustworthy deployment. The RFP is grounded in our research agenda (and see below), which spans three connected aims: Aim 1: Characterize and forecast misalignment in frontier AI systems: why frontier AI training-and-deployment safety stacks still result in models learning effective goals that fail under distribution shift, pressure, or extended interaction. Aim 2: Develop generalizable measurements and interventions: advance the science of evaluations with decision-relevant construct and predictive validity, and develop interventions that control what AI systems learn (not just what they say). Aim 3: Oversee AI systems with superhuman capabilities and address multi-agent risks: extend oversight and control to regimes where humans cannot directly evaluate correctness/safety, and address risks that arise from interacting AI systems. We invite applicants to apply to one or multiple funding tiers. Applicants may submit more than [...] ---Outline:(00:12) Schmidt Sciences request for proposals on the Science of Trustworthy AI(02:33) Research Agenda(08:56) Section 1: Characterizing and Forecasting Misalignment(09:55) 1.1: What is misalignment, and how much do we see today?(11:43) 1.2: Mechanisms of Generalization and Representation(14:00) 1.3: Scaling, Emergence, and Forecasting Risk(15:43) Section 2: Generalizable Measurements and Interventions(16:20) 2.1: Building a Science of Evaluation(19:47) 2.2: Interventions that Generalize(21:44) Section 3: Oversight Under Capability Gaps and Multi-Agent Risks(22:36) 3.1: Amplified Oversight for Superhuman Performance(27:51) 3.2: Multi-Agent Risks and Collective Dynamics(30:46) Out of Scope --- First published: February 25th, 2026 Source: https://www.lesswrong.com/posts/sup9qzCrdjcAGyrxa/schmidt-sciences-request-for-proposals-on-the-science-of --- Narrated by TYPE III AUDIO.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.