Nick Bostrom is a Swedish philosopher whose work on long-term technological risk, and in particular his 2014 book "Superintelligence: Paths, Dangers, Strategies," helped establish existential risk from AI as a subject of serious academic and public discussion.
Bostrom held a professorship at the University of Oxford, where in 2005 he founded and directed the Future of Humanity Institute (FHI), an interdisciplinary research center that studied large-scale risks to humanity's long-term future, including risks from advanced AI, biotechnology and other emerging technologies, alongside broader questions in philosophy and probability theory. FHI operated for nearly two decades before closing in April 2024 amid administrative friction with the university, despite having become one of the most influential centers for early AI safety and AI alignment research.
Superintelligence and core concepts
Bostrom's 2014 book "Superintelligence" laid out a systematic argument that an AI system substantially more capable than humans across all cognitively relevant domains, a Superintelligence, could pose severe risks if its goals were not carefully aligned with human values, even if no single step in its development involved obvious malice or error. The book popularized several concepts that remain central to AI safety discourse: the orthogonality thesis, the idea that an AI system's level of intelligence is independent of its final goals, meaning a highly capable system could pursue arbitrary or trivial objectives; instrumental convergence, the observation that a wide range of goals would lead an advanced AI to pursue similar intermediate subgoals such as self-preservation and resource acquisition; and the paperclip maximizer, a thought experiment illustrating how a system single-mindedly optimizing an apparently harmless goal could produce catastrophic outcomes. The book was cited by figures including Elon Musk as raising serious concerns about the trajectory of AI development, and it substantially shaped the vocabulary later used in debates over AI safety at major labs including OpenAI and Anthropic.
Simulation argument and other work
Before Superintelligence, Bostrom was known for the 2003 simulation argument, which proposed that at least one of three propositions is likely true: that civilizations almost always go extinct before reaching a stage capable of running detailed ancestor simulations, that such civilizations would choose not to run many such simulations, or that we are almost certainly living inside a simulation. The argument, though not primarily about AI, drew on similar reasoning about advanced future technology and became widely discussed well beyond academic philosophy.
Influence and criticism
Bostrom's work helped inspire the creation of other AI-safety-focused organizations, including the Machine Intelligence Research Institute and later the Future of Life Institute, and influenced researchers such as Eliezer Yudkowsky, Max Tegmark and Dan Hendrycks who continued work on long-term AI risk after FHI's closure. Critics have argued that Bostrom's scenarios rely on speculative, hard-to-falsify assumptions about future AI capabilities, and that focusing on speculative long-term risks can distract from more immediate, documented harms from deployed AI systems, a tension that has persisted in the broader AI safety field between near-term and long-term risk framing.