Superintelligence refers to a hypothetical intellect that would vastly exceed the cognitive performance of the best human minds across essentially all domains, including scientific creativity, general wisdom, and social skills. The concept is central to long-term debates in AI safety and is most closely associated with philosopher Nick Bostrom, whose 2014 book "Superintelligence: Paths, Dangers, Strategies" brought the idea to mainstream attention and shaped much of the subsequent discourse on AI risk.
Origins of the concept
Speculation about machines exceeding human intelligence predates modern AI. Mathematician I. J. Good, a colleague of Alan Turing during World War II, wrote in a 1965 essay that an "ultraintelligent machine" could design even better machines, triggering an "intelligence explosion" that would leave human intellect far behind, a dynamic sometimes called recursive self-improvement. This idea underlies much later thinking on the Technological singularity, a hypothesized point of runaway technological change.
Bostrom's taxonomy
Bostrom's book distinguishes forms of superintelligence by the type of advantage involved: "speed superintelligence" (a human-level mind operating far faster), "collective superintelligence" (many minds coordinated effectively), and "quality superintelligence" (a mind qualitatively superior, not merely faster). He also popularized the "orthogonality thesis," the claim that intelligence and final goals are independent, meaning a highly capable system is not automatically benevolent, and the related "instrumental convergence" thesis, that sufficiently capable agents pursuing almost any goal tend to seek resources, self-preservation, and freedom from interference as useful subgoals.
Paths to superintelligence
Proposed routes discussed in the literature include scaling Large language model and Foundation model systems, whole-brain emulation, biological cognitive enhancement, and networked collective intelligence. Since the mid-2020s, the dominant industry bet has been that scaling compute, data, and model size, guided by empirical Scaling laws, could eventually produce systems with superhuman performance across broad domains, though whether this path leads to genuine general superintelligence rather than narrow superhuman skill remains contested.
Safety and control debates
Bostrom's "control problem," how to ensure a superintelligent system remains aligned with human values even after it surpasses human ability to evaluate or correct it, motivates much of contemporary AI alignment research. Organizations including Anthropic, founded partly around this concern, and researchers such as Eliezer Yudkowsky have argued that misaligned superintelligence poses a serious Existential risk from AI. Anthropic CEO Dario Amodei and OpenAI cofounder Ilya Sutskever, who later founded Safe Superintelligence Inc. explicitly around this goal in 2024, have both framed safe development of highly capable systems as a defining challenge of the field.
Reception and criticism
Bostrom's arguments have drawn substantial criticism. Some researchers, including Yann LeCun, argue that current architectures are far from the kind of general, autonomous agency the scenarios assume, and that speculative long-horizon risk narratives distract from nearer-term, concrete harms such as bias, misinformation, and labor disruption. Others question the orthogonality thesis or argue that recursive self-improvement faces diminishing returns and physical bottlenecks, including energy and chip supply, that the pure "intelligence explosion" framing understates. Despite this contestation, superintelligence remains a reference point in AI governance discussions, cited in statements such as the 2023 open letter warning that mitigating AI extinction risk should be a global priority alongside pandemics and nuclear war.