今天，AI 领域传出一封重要的公开信。
包括 「ChatGPT 之父」Sam Altman 在内 350 位 AI 权威都签署了这封联名公开信，并对当前正在开发的 AI 技术可能会对人类构成生存威胁担忧。
这封信只有一句声明：降低人工智能灭绝的风险，应该与其它社会规模的风险（如大流行病和核战争）一样，成为全球的优先事项。
加入公开信签名🔗 https://www.safe.ai/statement-on-ai-risk
签署人包括三个领先的 AI 公司高管：
-
OpenAI 首席执行官 Sam Altman；
-
Google DeepMind 首席执行官 Demis Hassabis；
-
Anthropic 的首席执行官 Dario Amodei；
更重要的是，名单里还有 Geoffrey Hinton 和 Yoshua Bengio 两位人工智能的「教父」。
超过 350 名从事人工智能工作的高管、研究人员和工程师签署了这份由非盈利组织人工智能安全中心发布的公开信，认为人工智能具备可能导致人类灭绝的风险，应当将其视为与流行病和核战争同等的社会风险。
如今人们越来越担心 ChatGPT 等人工智能模型的发展可能对社会和就业带来的潜在威胁，很多人呼吁针对 AI 行业进行更强力的监管，否则它将对社会造成不可挽回的破坏。
AI 不断狂飙，但相关的监管、审计手段却迟迟没有跟上，这也意味着没有人能够保证 AI 工具以及使用 AI 工具的过程中的安全性。
在上周，Sam Altman 和另外两位 OpenAI 的高管提出应该建立类似国际原子能机构这样国际组织来安全监管 AI 的发展，他呼吁国际间领先的人工智能制造商应该展开合作，并要求政府强化对尖端人工智能制造商的监管。
其实早在 3 月，就一篇有关叫停 6 个月 AI 研究的公开信火遍了全网。
该信呼吁所有的 AI 实验立即暂停研究比 GPT-4 更先进的 AI 模型，暂停时间至少 6 个月，为的就是把这些可怕的幻想扼杀在摇篮之中。
AI 的进步速度实在过于惊人，但相关的监管、审计手段却迟迟没有跟上，这也意味着没有人能够保证 AI 工具以及使用 AI 工具的过程中的安全性。
该联名信已经获得了包括 2018 年图灵奖得主 Yoshua Bengio、马斯克、史蒂夫 · 沃兹尼亚克、Skype 联合创始人、Pinterest 联合创始人、Stability AI CEO 等多位知名人士的签名支持，截稿前联名人数已经达到 1125 人。
公开信的原文如下：
人工智能具有与人类竞争的智能，这可能给社会和人类带来深刻的风险，这已经通过大量研究 [1] 得到证实，并得到了顶级 AI 实验室的认可 [2]。正如广泛认可的阿西洛马尔人工智能原则（Asilomar AI Principles）所指出的，先进的 AI 可能代表地球生命历史上的重大变革，因此应该以相应的关注和资源进行规划和管理。
不幸的是，即使在最近几个月， AI 实验室在开发和部署越来越强大的数字化思维方面陷入了失控的竞争，而没有人能够理解、预测或可靠地控制这些数字化思维，即使是它们的创造者也无法做到这一点。
现代 AI 系统现在已经在通用任务方面具有与人类竞争的能力 [3]，我们必须问自己：
-
我们应该让机器充斥我们的信息渠道，传播宣传和谎言吗？
-
我们应该将所有工作都自动化，包括那些令人满足的工作吗？
-
我们应该发展可能最终超过、取代我们的非人类思维吗？
-
我们应该冒失控文明的风险吗？
这些决策不应该由非选举产生的科技领导人来负责。只有当我们确信 AI 系统的影响是积极的，风险是可控的，我们才应该开发强大的 AI 系统。这种信心必须得到充分的理由，并随着系统潜在影响的增大而增强。OpenAI 最近关于人工智能的声明指出：「在某个时候，可能需要在开始训练未来系统之前获得独立审查，对于最先进的努力，需要同意限制用于创建新模型的计算增长速度。」我们同意。现在就是那个时刻。
因此，我们呼吁所有 AI 实验室立即暂停至少 6 个月的时间，不要训练比 GPT-4 更强大的 AI 系统。这个暂停应该是公开和可验证的，并包括所有关键参与者。如果无法迅速实施这样的暂停，政府应该介入并实行一个暂禁令。
AI 实验室和独立专家应该利用这个暂停时间共同制定和实施一套共享的先进 AI 设计和开发安全协议，这些协议应该由独立的外部专家严格审核和监督。
这些协议应确保遵守它们的系统在合理怀疑范围之外是安全的 [4]。这并不意味着暂停 AI 发展，而只是从不断迈向更大、不可预测的黑箱模型及其突现能力的危险竞赛中退一步。
AI 研究和开发应该重点放在提高现有强大、先进系统的准确性、安全性、可解释性、透明度、鲁棒性、对齐、可信度和忠诚度。
与此同时，AI 开发者必须与政策制定者合作，大力加速 AI 治理体系的发展。这些至少应包括：
-
专门负责 AI 的新的、有能力的监管机构；
-
对高能力 AI 系统和大型计算能力池的监督和跟踪；
-
用于区分真实与合成内容、跟踪模型泄漏的溯源和水印系统；
-
健全的审计和认证生态系统；对 AI 造成的损害承担责任；
-
技术 AI 安全研究的充足公共资金；
-
应对 AI 引起的巨大经济和政治变革（特别是对民主制度的影响）的资深机构。
人类可以在 AI 的帮助下享受繁荣的未来。在成功创造出强大的 AI 系统之后，我们现在可以享受一个「AI 盛夏」，在其中我们收获回报，将这些系统明确地用于造福所有人，并让社会有机会适应。
在面临可能对社会造成灾难性影响的其他技术上，社会已经按下了暂停键 [5]。在这里，我们也可以这样做。让我们享受一个漫长的 AI 盛夏，而不是毫无准备地冲向秋天。
附录英文
AI experts, journalists, policymakers, and the public are increasingly discussing a broad spectrum of important and urgent risks from AI. Even so, it can be difficult to voice concerns about some of advanced AI’s most severe risks. The succinct statement below aims to overcome this obstacle and open up discussion. It is also meant to create common knowledge of the growing number of experts and public figures who also take some of advanced AI’s most severe risks seriously.
Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.
Signatories:
AI ScientistsOther Notable Figures
Geoffrey Hinton
Emeritus Professor of Computer Science, University of Toronto
Yoshua Bengio
Professor of Computer Science, U. Montreal / Mila
Demis Hassabis
CEO, Google DeepMind
Sam Altman
CEO, OpenAI
Dario Amodei
CEO, Anthropic
Dawn Song
Professor of Computer Science, UC Berkeley
Ya-Qin Zhang
Professor and Dean, AIR, Tsinghua University
Ilya Sutskever
Co-Founder and Chief Scientist, OpenAI
Shane Legg
Chief AGI Scientist and Co-Founder, Google DeepMind
Martin Hellman
Professor Emeritus of Electrical Engineering, Stanford
James Manyika
SVP, Research, Technology & Society, Google-Alphabet
Yi Zeng
Professor and Director of Brain-inspired Cognitive AI Lab, Institute of Automation, Chinese Academy of Sciences
Xianyuan Zhan
Assistant Professor, Tsinghua University
Anca Dragan
Associate Professor of Computer Science, UC Berkeley
Bill McKibben
Schumann Distinguished Scholar, Middlebury College
Alan Robock
Distinguished Professor of Climate Science, Rutgers University
Angela Kane
Vice President, International Institute for Peace, Vienna; former UN High Representative for Disarmament Affairs
Audrey Tang
Minister of Digital Affairs and Chair of National Institute of Cyber Security
Daniela Amodei
President, Anthropic
David Silver
Professor of Computer Science, Google DeepMind and UCL
Lila Ibrahim
COO, Google DeepMind
Stuart Russell
Professor of Computer Science, UC Berkeley
Marian Rogers Croak
VP Center for Responsible AI and Human Centered Technology, Google
Andrew Barto
Professor Emeritus, University of Massachusetts
Mira Murati
CTO, OpenAI
Jaime Fernández Fisac
Assistant Professor of Electrical and Computer Engineering, Princeton University
Diyi Yang
Assistant Professor, Stanford University
Gillian Hadfield
Professor, CIFAR AI Chair, University of Toronto, Vector Institute for AI
Laurence Tribe
University Professor Emeritus, Harvard University
Pattie Maes
Professor, Massachusetts Institute of Technology - Media Lab
Kevin Scott
CTO, Microsoft
Eric Horvitz
Chief Scientific Officer, Microsoft
Peter Norvig
Education Fellow, Stanford University
Atoosa Kasirzadeh
Assistant Professor, University of Edinburgh, Alan Turing Institute
Erik Brynjolfsson
Professor and Senior Fellow, Stanford Institute for Human-Centered AI
Mustafa Suleyman
CEO, Inflection AI
Emad Mostaque
CEO, Stability AI
Ian Goodfellow
Principal Scientist, Google DeepMind
John Schulman
Co-Founder, OpenAI
Kersti Kaljulaid
Former President of the Republic of Estonia
David Haussler
Professor and Director of the Genomics Institute, UC Santa Cruz
Stephen Luby
Professor of Medicine (Infectious Diseases), Stanford University
Ju Li
Professor of Nuclear Science and Engineering and Professor of Materials Science and Engineering, Massachusetts Institute of Technology
David Chalmers
Professor of Philosophy, New York University
Daniel Dennett
Emeritus Professor of Philosophy, Tufts University
Peter Railton
Professor of Philosophy at University of Michigan, Ann Arbor
Sheila McIlraith
Professor of Computer Science, University of Toronto
Victoria Krakovna
Research Scientist, Google DeepMind
Mary Phuong
Research Scientist, Google DeepMind
Lex Fridman
Research Scientist, MIT
Sharon Li
Assistant Professor of Computer Science, University of Wisconsin Madison
Phillip Isola
Associate Professor of Electrical Engineering and Computer Science, MIT
David Krueger
Assistant Professor of Computer Science, University of Cambridge
Jacob Steinhardt
Assistant Professor of Computer Science, UC Berkeley
Martin Rees
Professor of Physics, Cambridge University
He He
Assistant Professor of Computer Science and Data Science, New York University
David McAllester
Professor of Computer Science, TTIC
Vincent Conitzer
Professor of Computer Science, Carnegie Mellon University and University of Oxford
Bart Selman
Professor of Computer Science, Cornell University
Michael Wellman
Professor and Chair of Computer Science & Engineering, University of Michigan
Jinwoo Shin
KAIST Endowed Chair Professor, Korea Advanced Institute of Science and Technology
Dae-Shik Kim
Professor of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST)
Frank Hutter
Professor of Machine Learning, Head of ELLIS Unit, University of Freiburg
Jaan Tallinn
Co-Founder of Skype
Adam D'Angelo
CEO, Quora, and board member, OpenAI
Simon Last
Cofounder & CTO, Notion
Dustin Moskovitz
Co-founder & CEO, Asana
Scott Aaronson
Schlumberger Chair of Computer Science, University of Texas at Austin
Max Tegmark
Professor, MIT, Center for AI and Fundamental Interactions
Bruce Schneier
Lecturer, Harvard Kennedy School
Martha Minow
Professor, Harvard Law School
Gabriella Blum
Professor of Human Rights and Humanitarian Law, Harvard Law
Kevin Esvelt
Associate Professor of Biology, MIT
Edward Wittenstein
Executive Director, International Security Studies, Yale Jackson School of Global Affairs, Yale University
Karina Vold
Assistant Professor, University of Toronto
Victor Veitch
Assistant Professor of Data Science and Statistics, University of Chicago
Dylan Hadfield-Menell
Assistant Professor of Computer Science, MIT
Samuel R. Bowman
Associate Professor of Computer Science, NYU and Anthropic
Mengye Ren
Assistant Professor of Computer Science, New York University
Shiri Dori-Hacohen
Assistant Professor of Computer Science, University of Connecticut
Miles Brundage
Head of Policy Research, OpenAI
Allan Dafoe
AGI Strategy and Governance Team Lead, Google DeepMind
Helen King
Senior Director of Responsibility & Strategic Advisor to Research, Google DeepMind
Jade Leung
Governance Lead, OpenAI
Jess Whittlestone
Head of AI Policy, Centre for Long-Term Resilience
Sarah Kreps
John L. Wetherill Professor and Director of the Tech Policy Institute, Cornell University
Jared Kaplan
Co-Founder, Anthropic
Chris Olah
Co-Founder, Anthropic
Andrew Revkin
Director, Initiative on Communication & Sustainability, Columbia University - Climate School
Carl Robichaud
Program Officer (Nuclear Weapons), Longview Philanthropy
Leonid Chindelevitch
Lecturer in Infectious Disease Epidemiology, Imperial College London
Nicholas Dirks
President, The New York Academy of Sciences
Marc Warner
CEO, Faculty
Clare Lyle
Research Scientist, Google DeepMind
Ryota Kanai
CEO, Araya, Inc.
Tim G. J. Rudner
Assistant Professor and Faculty Fellow, New York University
Noah Fiedel
Director, Research & Engineering, Google DeepMind
Jakob Foerster
Associate Professor of Engineering Science, University of Oxford
Michael Osborne
Professor of Machine Learning, University of Oxford
Marina Jirotka
Professor of Human Centred Computing, University of Oxford
Nancy Chang
Research Scientist, Google
Roger Grosse
Associate Professor of Computer Science, University of Toronto and Anthropic
David Duvenaud
Associate Professor of Computer Science, University of Toronto
Daniel M. Roy
Associate Professor and Canada CIFAR AI Chair, University of Toronto; Vector Institute
Chris J. Maddison
Assistant Professor of Computer Science, University of Toronto
Florian Shkurti
Assistant Professor of Computer Science, University of Toronto
Jeff Clune
Associate Professor of Computer Science and Canada CIFAR AI Chair, The University of British Columbia and the Vector Institute
Eva Vivalt
Assistant Professor of Economics, University of Toronto, and Director, Global Priorities Institute, University of Oxford
Jacob Tsimerman
Professor of Mathematics, University of Toronto
Danit Gal
Technology Advisor at the UN; Associate Fellow, Leverhulme Centre for the Future of Intelligence, University of Cambridge
Jean-Claude Latombe
Professor (Emeritus) of Computer Science, Stanford University
Scott Niekum
Associate Professor of Computer Science, University of Massachusetts Amherst
Lionel Levine
Associate Professor of Mathematics, Cornell University
Brian Ziebart
Associate Professor of Computer Science, University of Illinois Chicago
Aza Raskin
Cofounder, Center for Humane Technology, The Earth Species Project
Tristan Harris
Executive Director, Center for Humane Technology
Anthony Aguirre
Executive Director, Future of Life Institute
Sam Harris
Author, neuroscientist, Making Sense / Waking Up
Grimes
Musician / Artist
Chris Anderson
Dreamer-in-Chief, TED
Rif A. Saurous
Research Director, Google
Jose Hernandez-Orallo
Professor of Computer Science, Technical University of Valencia
Paul S. Rosenbloom
Professor Emeritus, University of Southern California
Samuel Albanie
Assistant Professor of Engineering, University of Cambridge
Jascha Sohl-Dickstein
Principal Scientist, Google DeepMind
Been Kim
Research Scientist, Google DeepMind
Cihang Xie
Assistant Professor of Computer Science and Engineering, UC Santa Cruz
Philip S. Thomas
Associate Professor, University of Massachusetts
Pierre Baldi
Professor, University of California, Irvine
Elad Hazan
Professor of Computer Science, Princeton University and Google DeepMind
Katherine Lee
Research Scientist, Google DeepMind
Shai Shalev-Shwartz
Professor, The Hebrew University of Jerusalem
Foutse Khomh
Professor and Canada CIFAR AI Chair, Polytechnique Montreal
Dan Hendrycks
Executive Director, Center for AI Safety