This resource is the second of two on the benefits and risks of open-weights model release. In contrast, this paper expresses strong skepticism toward releasing highly capable foundation model weights, arguing that the risks may outweigh the benefits. While recognizing the advantages of openness, such as encouraging innovation and external oversight, it warns that making models publicly available increases the potential for misuse, including cyberattacks, biological weapon development, and disinformation. The article emphasizes that malicious actors could easily disable safeguards, fine-tune models for harmful purposes, and exploit vulnerabilities. Instead of fully open releases, it advocates for safer alternatives like democratic oversight, structured access, and staged model release, which can provide some benefits of openness while mitigating the extreme risks posed by advanced AI systems.Original text: https://cdn.governance.ai/Open-Sourcing_Highly_Capable_Foundation_Models_2023_GovAI.pdf Author(s): Elizabeth Seger, Noemi Dreksler, Richard Moulange, Emily Dardaman, Jonas Schuett, K. Wei, Christoph Winter, Mackenzie Arnold, Seán Ó hÉigeartaigh, Anton Korinek, Markus Anderljung, Ben Bucknall, Alan Chan, Eoghan Stafford, Leonie Koessler, Aviv Ovadya, Ben Garfinkel, Emma Bluemke, Michael Aird, Patrick Levermore, Julian Hazell, Abhishek GuptaA podcast by BlueDot Impact.Learn more on the AI Safety Fundamentals website.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.

Transcript

Related