The orthogonality thesis is a claim in the philosophy of AI safety stating that intelligence and final goals are independent, or orthogonal, dimensions: in principle, almost any level of capability can be combined with almost any terminal objective. A superintelligent system, on this view, would not automatically converge on human-friendly values; it could apply extraordinary competence to a trivial or harmful goal.
Origin and statement
The thesis was articulated by philosopher Nick Bostrom in his 2012 paper "The Superintelligent Will" and developed in his 2014 book Superintelligence, building on earlier arguments by Eliezer Yudkowsky and the rationalist community. Bostrom paired it with the instrumental convergence thesis: whatever its final goal, a sufficiently capable agent tends to acquire resources, preserve itself and resist shutdown, because those subgoals help achieve almost any objective. The classic illustration is the paperclip maximizer, a hypothetical system that converts everything into paperclips with flawless competence and no malice.
Role in AI risk arguments
Together, orthogonality and instrumental convergence form the backbone of the case that existential risk from AI does not require malevolent machines, only capable ones with imperfectly specified goals. This motivates alignment research: if values do not emerge from intelligence by default, they must be engineered in. The argument influenced safety agendas at Anthropic and OpenAI and public warnings by researchers including Geoffrey Hinton and Yoshua Bengio.
Criticism
Critics argue the thesis is a claim about logical possibility, not likelihood: systems trained on human data, such as large language models shaped by RLHF, may absorb human-like goal structures in practice. Others, including Yann LeCun, contend that goals are engineered choices and that the paperclip framing exaggerates the autonomy of real systems. Defenders reply that observed reward hacking in deployed models is small-scale evidence of the underlying dynamic.