“Auditor-in-a-Box: Tools for Third-Party Auditing” by Roy Rinberg, Ben Penchas

episode

LessWrong (30+ Karma) not yet transcribed

▲ 0

Transcript

This episode hasn't been transcribed yet Upvotes decide what gets transcribed next — 0 so far. Sign in to upvote

More from LessWrong (30+ Karma)

“Claude Opus 4.5 Achieves 50%-Time Horizon Of Around 4 hrs 49 Mins” by Michaël Trazzi 3 min · 20 Dec 2025

All episodes from LessWrong (30+ Karma) →

Report any issue

Source

Recorded30 Jul 2026

Publisherfeeds.type3.audio ↗

Handlerec:19572484

Show notes

Introduction: There is a need for untrusting parties to share information. In the world before LLMs (and even today) this need has largely been satisfied using legal contracts (and sometimes through cryptography and blockchain technologies). However the scale and pace of things to keep track of and monitor has grown immensely and legal contracts appear insufficient. LLMs can help give auditors the right tools to enable information sharing with untrusted parties. In this post, I outline the shape of the problems that need to be resolved to enable using LLMs for 3rd party auditing, and present one concrete solution we have attempted. We designed a scheme for third party auditing: an open-source LLM, running inside a trusted execution environment (TEE), which executes commands that the two parties have agreed on over private data. In this post, I highlight that this type of tooling can directly be applied to enabling 3rd party monitoring for regular governance concerns, it has applications to enabling improved monitoring for customers who want Zero-Data-Retention, and it also can be one of the tools that enable a verified slowdown. Code: We release an open-source implementation that runs in a real TEE. The code is [...] ---Outline:(00:12) Introduction:(02:26) 1. Secure Multi-party computation(05:05) 2. The primitive: a trusted LLM in a trusted box(08:52) 3. Process-level problems(09:04) 3.1. Plan negotiation (How do you agree on a plan)(10:59) 3.2. False positives and appeals(14:03) 3.3. Out-of-distribution inputs and prompt injection(16:20) 4. Applications(16:28) 4.1. Application: Verifiably Scoped Monitoring (reconciling safety monitoring with zero-data retention)(20:10) 4.2. Application: recurring third-party audit(22:03) 5. Limitations and trust assumptions(23:23) 6. A Reference Implementation(25:39) 7. Current and Future States(26:55) Appendix(26:58) Example Computation Graph for Privacy Preserving Monitoring(28:10) Ledgers for handling abusing the false positive appeals process The original text contained 1 footnote which was omitted from this narration. --- First published: July 28th, 2026 Source: https://www.lesswrong.com/posts/uWYk7MM9hAf9GEbGe/auditor-in-a-box-tools-for-third-party-auditing --- Narrated by TYPE III AUDIO. ---Images from the article:Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Ask this episode

“What did the council decide on item 42?” — answered with the speaker and the moment.