# US AI Safety Institute Founding

The US AI Safety Institute was established in November 2023 under the National Institute of Standards and Technology (NIST) to evaluate and ensure the safety of advanced artificial intelligence models. It was created during the AI Safety Summit in the United Kingdom.

The US AI Safety Institute (US AISI) is a state-backed organization established in November 2023 under the National Institute of Standards and Technology (NIST), a agency of the US Department of Commerce. Its primary mission is to evaluate and ensure the safety of advanced artificial intelligence (AI) models, particularly those known as frontier AI models. The institute was founded during the AI Safety Summit held in the United Kingdom, where both the UK and the US announced the creation of their respective AI safety bodies. This marked a significant step in the global effort to address potential risks associated with rapidly advancing AI technologies, including [large language models](https://www.wikiprompt.org/wiki/large-language-model) and other [generative AI](https://www.wikiprompt.org/wiki/generative-ai) systems.

The establishment of the US AISI reflected a growing recognition in 2023 of the potential existential risks posed by AI, as highlighted by public declarations from researchers and industry leaders. The institute operates within NIST, leveraging the agency's expertise in measurement standards and technology evaluation. Its work focuses on developing testing methodologies, conducting safety evaluations, and collaborating with other national and international bodies to create a coherent framework for AI oversight. The US AISI is part of a broader international network of AI safety institutes that emerged from diplomatic efforts in 2023 and 2024.

## Founding and Early Mandate

The US AISI was formally created on November 1, 2023, during the AI Safety Summit at Bletchley Park in the United Kingdom. The summit, hosted by then-Prime Minister Rishi Sunak, brought together representatives from governments, AI companies, and academic institutions to discuss the safe development of frontier AI. The US government, through NIST, committed to establishing a dedicated institute to conduct research and evaluations on AI safety. The institute's initial focus was on developing standardized testing procedures for AI models, including assessments of capabilities in areas such as [machine learning](https://www.wikiprompt.org/wiki/machine-learning), [deep learning](https://www.wikiprompt.org/wiki/deep-learning), and [neural networks](https://www.wikiprompt.org/wiki/neural-network).

In its early months, the US AISI worked to define its role within the broader US AI policy landscape. It coordinated with other federal agencies, including the White House Office of Science and Technology Policy, to align its activities with national priorities. The institute also began engaging with leading AI developers, such as [OpenAI](https://www.wikiprompt.org/wiki/openai), [Anthropic](https://www.wikiprompt.org/wiki/anthropic), and [Google DeepMind](https://www.wikiprompt.org/wiki/google-deepmind), to gain access to their most advanced models for pre-deployment safety testing. This collaborative approach aimed to address concerns that AI companies could not reliably "mark their own homework" in terms of safety evaluations.

## Collaboration with the UK AI Safety Institute

A key aspect of the US AISI's early work was its partnership with the UK AI Safety Institute (UK AISI), which was established simultaneously during the November 2023 summit. The two institutes agreed to collaborate on developing common evaluation rules and procedures, recognizing that a unified approach would be more effective than fragmented national efforts. In April 2024, the US and UK formally concluded an agreement to conduct at least one joint safety test on a shared AI model. This collaboration was seen as a model for international cooperation in AI governance.

The partnership also involved sharing research findings and methodologies. The US AISI and UK AISI jointly explored issues such as model robustness, alignment with human values, and the potential for AI systems to leak sensitive information or be exploited for cyberattacks. Their work informed the development of international standards, with the UK AISI announcing in May 2024 that it would open an office in San Francisco to be closer to major AI companies, a move that complemented the US AISI's location within NIST.

## International Network Formation

At the AI Seoul Summit in May 2024, international leaders agreed to form a network of AI safety institutes, building on the bilateral cooperation between the US and UK. The network initially comprised institutes from the UK, the US, Japan, France, Germany, Italy, Singapore, South Korea, Australia, Canada, and the European Union. The US AISI played a central role in this network, helping to coordinate joint research projects and evaluation exercises. In July 2025, the network held an exercise to explore issues with evaluating AI agents, particularly regarding the leakage of sensitive information and cybersecurity risks. Network members also met at NeurIPS 2025 in San Diego to discuss emerging challenges.

The US AISI's participation in the international network was part of a broader effort to establish global norms for AI safety. The institute contributed to discussions on topics such as [transformer architectures](https://www.wikiprompt.org/wiki/transformer), [multi-head attention](https://www.wikiprompt.org/wiki/multi-head-attention), and other technical aspects of AI systems, ensuring that safety evaluations kept pace with rapid advancements in the field. The network also addressed governance questions, including how to balance innovation with precautionary measures.

## Evolution and Renaming

In 2025, the US AISI underwent a significant organizational change. It was renamed the Center for AI Standards and Innovation (CAISI), reflecting a shift in focus toward developing standards and promoting innovation in AI safety. The renaming was part of a broader trend, as the UK's AI Safety Institute was also renamed the AI Security Institute in the same year. The change signaled a move from purely safety-focused evaluations to a more comprehensive approach that included fostering responsible AI development and deployment.

The transition to CAISI did not alter the institute's core mission but expanded its scope. It continued to conduct safety evaluations of frontier AI models, but also began working more closely with industry to develop best practices and voluntary standards. This evolution was seen as a response to the growing maturity of the AI field and the need for ongoing oversight as AI systems became more integrated into various sectors, including healthcare, transportation, and finance.

## Technical Focus Areas

The US AISI's technical work spans several areas of AI research and development. It evaluates models for potential biases, robustness to adversarial inputs, and alignment with intended behaviors. The institute also investigates the interpretability of AI systems, seeking to understand how [neural networks](https://www.wikiprompt.org/wiki/neural-network) make decisions. This includes studying techniques such as [layer normalization](https://www.wikiprompt.org/wiki/layer-normalization), [batch normalization](https://www.wikiprompt.org/wiki/batch-normalization), and [dropout](https://www.wikiprompt.org/wiki/dropout) to improve model reliability.

Another focus is on the safety of AI agents, which are systems capable of autonomous decision-making. The US AISI has explored scenarios where AI agents might leak sensitive information or be manipulated to perform harmful actions. It has also examined the implications of RLAIF and other training methods that rely on AI-generated feedback, which can introduce subtle biases or errors. The institute's research contributes to the development of evaluation benchmarks and testing protocols used by both government and industry.

## Impact and Future Directions

The founding of the US AISI has had a significant impact on the AI policy landscape. It has provided a governmental mechanism for independent oversight of AI technologies, complementing self-regulatory efforts by companies. The institute's work has informed legislative and regulatory proposals, both in the US and internationally. As of 2025, the institute continues to evolve, with plans to expand its research capabilities and deepen its international collaborations.

Looking ahead, the US AISI (now CAISI) faces challenges related to the rapid pace of AI development, including the emergence of more capable models and new applications. It must also navigate questions about the appropriate balance between safety and innovation, as well as the allocation of resources across competing priorities. The institute's success will depend on its ability to adapt to changing technological and political circumstances while maintaining its credibility as an independent evaluator of AI systems.

---
Source: https://www.wikiprompt.org/wiki/us-ai-safety-institute-founding
License: CC BY-SA 4.0 (https://creativecommons.org/licenses/by-sa/4.0/)
Last updated: 2026-09-12T16:23:30.616653+00:00