Superintelligence: OpenAI Says We Have 10 Years to Prepare
A critical analysis of OpenAI’s latest blog post, “Governance of superintelligence”
I wonder if there’s any other discipline where “experts” devote so much time to making predictions. AI is special — the only place where you can spend your days talking about the future even when it has already arrived.
OpenAI: A story of AGI and superintelligence
OpenAI was born in 2015, eight years ago, under a bold premise: we can — and will — build artificial general intelligence (AGI). And just as bold has been the determination of the startup’s founders, who have spent these eight years trying to architect the road toward that goal. They may have gone on an off-ramp, but maybe not. No one knows.
It’s funny because not only do AI people love making forecasts (me included), the field as a whole is unable to assess in hindsight whether those forecasts materialized or not. Is the transformer an AGI milestone? Is GPT the breakthrough we were waiting for? Is deep learning the ultimate AI paradigm? We should know by now but we don’t. The lack of consensus among experts is revealing evidence.
Anyway, OpenAI first mentioned AGI in its announcement post, published in December 2015 (they called it “human-level AI” back then), and then officialized it as a goal in the Charter, the company’s manifesto. In February this year, Sam Altman wrote “AGI and beyond” and revealed the company’s plans for a world post-AGI.
On Monday this week, Altman and his co-founders Greg Brockman and Ilya Sutskever went a step further and co-published a new blog post entitled “Governance of superintelligence,” where they define superintelligences as “AI systems dramatically more capable than even AGI” and claim that “now is a good time to start thinking about the governance of superintelligence.”
OpenAI using “superintelligence” (SI) so openly (pun intended) marks the closing of a cycle for them. In eight years, the company’s official blog has only mentioned this term on two occasions, both in 2023. From 2015 to 2022, no one dared say it. But in the last three months, Altman has gone from saying SI is about “the long term” to saying we…