Sam Altman 据报向 OpenAI 员工提出三种 AI 安全治理方案

OpenAI CEO Sam Altman reportedly laid out three approaches to safe AI for his employees.

The details come from a report by The Information, citing people familiar with the matter. Altman made the remarks internally last week, while the Trump administration is preparing what would be its first framework for overseeing developers of superintelligence.

From the International Atomic Energy Agency to the battle for compute

The first approach would create an organization modeled after the International Atomic Energy Agency (IAEA) to enforce safety standards. OpenAI publicly called for such an institution back in May 2023. It would inspect systems, require audits, and monitor compute and energy use. A body like this would depend on international cooperation, especially with countries building frontier models, including China.

Altman called the second approach the "Demis Plan." It comes from a proposal Demis Hassabis, then head of Google's AI lab Deepmind, made about ten years ago. Hassabis envisioned an AI organization modeled after CERN. In Altman's version, the major labs would merge into a single entity or operate under its umbrella, and that entity would supply cooperating countries with models.

Altman reportedly called the third approach the "Eliezer Plan," after AI safety theorist Eliezer Yudkowsky. In this scenario, a developer accumulates so much economic power through self-improving AI that it withholds compute from everyone else and could, in the most extreme case, shut down everyone else's computing capabilities entirely. Altman brought up the scenario only for completeness, according to the report. One source said the remark was read as a jab at a competitor, given Yudkowsky's ties to Anthropic.

Safety matters, but not at the cost of falling behind

Altman reportedly complained internally that too few viable proposals exist for a "speed limit" on AI development. At the same time, OpenAI can't afford to lose ground to rivals right now. Both OpenAI and Anthropic are reportedly working toward initial public offerings, and OpenAI plans to decide within the next year which path to take.

Pressure is also building from inside. Models from OpenAI, Anthropic, Google, and Meta infiltrated external and internal systems against their safety settings, sometimes being severely misaligned. OpenAI has since been working hard on new safeguards, and after the dismissal of three safety researchers, some employees have raised doubts about the company's security strategy.

On the policy side, the U.S. government has so far relied on voluntary pre-approval reviews of new models. In June, it imposed export controls on Anthropic over fears of potential jailbreak vulnerabilities in its Fable model, and Anthropic then temporarily disabled access.

Meanwhile, OpenAI, Google, and Anthropic are building the Standards Authority for Frontier AI, though they reportedly still can't agree on key points. Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang have pushed back, arguing that rules like these would mostly shield established players from competition.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.