Examples of AI safety progressTraining AIs to behave safely and beneficially is difficult. They might learn to game their reward function, deceive human oversight, or seek power. Some argue that researchers have not made much progress in addressing these problems, but here we offer a few examples of progress on AI safety. Detecting lies in AI outputs. Language models often output false text, but a recent paper suggests they understand the truth in ways not reflected in their output. By analyzing a model’s internals, we can calculate the likelihood that a model believes a statement is true. The finding has been replicated in models that answer [...] ---Outline:(00:13) Examples of AI safety progress(03:56) Yoshua Bengio proposes a ban on AI agents(07:19) Lessons from Nuclear Arms Control for Verifying AI Treaties(10:02) Links --- First published: May 16th, 2023 Source: https://newsletter.safe.ai/p/ai-safety-newsletter-6 --- Want more? Check out our ML Safety Newsletter for technical safety research. Narrated by TYPE III AUDIO.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.

Transcript

Related