The Future of Humanity Institute (FHI) was a multidisciplinary research center at University of Oxford that operated from 2005 to 2024. It focused on questions concerning the long-term future of humanity, with a particular emphasis on existential risks and the development of advanced Artificial intelligence. Founded by the philosopher Nick Bostrom, the institute aimed to bring rigorous scholarship to bear on global challenges that could threaten human civilization's survival or long-term potential.
FHI was a pioneer in the academic field of existential risk studies. Its work combined philosophy, mathematics, and computer science to analyze potential dangers from emerging technologies, including AI, biotechnology, and other areas. The institute also contributed to the development of AI safety research before this field became widely established.
Founding and Development
FHI was established in 2005 at the University of Oxford, initially as part of the James Martin 21st Century School. It was later incorporated into the university's Faculty of Philosophy and the Oxford Martin School. Nick Bostrom served as its founding director, and the institute grew to include a multidisciplinary group of researchers from fields such as philosophy, computer science, and political science. Over its nearly two decades, FHI attracted funding from organizations like the Future of Life Institute and the Leverhulme Trust, among others.
The institute was notable for early and sustained attention to existential risk from AI systems, many years before topics such as transformer-based language models and deep neural networks became prominent in academia. Its researchers were among the first to argue that ensuring beneficial AI outcomes requires proactive safety research and careful governance.
Key Contributions and Research Areas
FHI published influential work on AI risk, including papers on the Unilateralist's Curse, AI safety via debate, and the concept of an 'intelligence explosion.' Nick Bostrom's 2014 book, Superintelligence: Paths, Dangers, Strategies, grew out of research conducted at the institute and was widely read in Silicon Valley and among policymakers.
The institute also studied other existential risks, such as nuclear war, pandemics, and the misuse of Machine learning technologies. FHI researchers were instrumental in formulating the 'existential risk from AI' research agenda, and some later moved to leading AI organizations including OpenAI, Anthropic, and Google DeepMind, where they applied safety principles to real-world system.
Notable People and Partnerships
FHI collaborated with various academics and institutions globally. Its researchers included philosopher Toby Ord, who contributed to the effective altruism movement and wrote about the Doomsday argument; economist and AI safety researcher (Kerstin Katherine Deitch); and computer scientist Stuart Russell, among others. These collaborations helped ensure that FHI's ideas were subject to peer review and integrated into broader scientific discourse.
Notably, FHI worked with binary cross-disciplinary centers like the Centre for the Study of Existential on Future of Humanity, the Future of Life Institute, and the Leverhulme Centre for the Future of Intelligence. These partnerships facilitated data collection, modeling, and decision-theoretic approaches to risk mitigation.
Closure and Legacy
The Future of Humanity Institute ceased operations in 2024, with its closure being announced by Nick Bostrom in his academic email. The exact reasons for the closure were not fully disclosed, though it has been suggested that continued funding for such interdisciplinary units posed challenges after nearly twenty years. The institute's closure does not erase its effects; its research on existential risk and safe AI continues to influence both academic and industrial practices.
Today, the concept of existential risk is a accepted field of study at several universities, and many of FHI's former staff continue at research roles in safety-focused organizations. The institute's emphasis on forward-looking, empirical analysis is generally credited with shaping the agenda of AI safety research that precedes the sharp rise of large language models.
External Perception and Criticisms
Some critics have questioned whether FHI's focus on transformative AI neglects more immediate societal challenges, while others have disputed some of its claims about the likelihood of extinction. However, a general tone remains that FHI helped legitimize the idea that catastrophic AI risks is worth serious academic study, and its contributions to AI ethics and policy are generally viewed as pivotal.
References
- Bostrom, Nick. 2019. 'Existential risk: theory, field, and challenges'. In dialogue with Bünger et al.
- Future of Humanity Institute (official page at Oxford, archived).
- Closing email by Nick Bostrom, February 2024.