In October 2024, Anthropic introduced the "computer use" feature for its Claude series of large language models. This capability allows an application to call the Claude API to attempt to give navigational control of the application via Claude's interpretation of screen content and simulation of keyboard and mouse input. It marked a significant step in Generative AI toward AI agents that can interact with graphical user interfaces in a human-like manner.
The feature was part of Anthropic's broader effort to expand Claude beyond chatbot interactions into agentic tools. It followed the release of Claude as an AI-based chatbot in March 2023 and the introduction of artifacts in June 2024. Computer use was designed to let Claude perform tasks such as filling forms, navigating websites, and operating software, using only visual information from the screen and simulated input events.
Technical Foundation
Computer use relies on Claude's ability to process screenshots and generate actions. The model receives a screen image, identifies relevant elements, and outputs commands such as mouse clicks, keystrokes, or scroll actions. This approach differs from traditional automation that depends on APIs or accessibility trees. Anthropic trained the model on a variety of desktop environments to improve generalization.
The underlying architecture is based on the Transformer (architecture) model, which is also used in other Large language model systems. Claude's training uses a constitution, a technique developed by Anthropic to improve ethical and legal compliance. The computer use feature was initially available through the Claude API, allowing developers to integrate it into their own applications.
Capabilities and Limitations
In its initial release, computer use could perform tasks such as searching the web, filling out forms, and navigating multi-step workflows. Anthropic reported that the model could complete simple tasks with moderate accuracy, but it was not yet reliable for complex or high-stakes operations. The company emphasized that the feature was experimental and recommended human oversight.
One notable limitation was the model's tendency to make errors in unfamiliar interfaces or when screen layouts changed. Anthropic also noted that computer use could be susceptible to prompt-injection attacks, where malicious instructions embedded in web pages might influence the model's actions. The company advised developers to implement safeguards.
Comparison with Other AI Systems
Computer use positioned Anthropic alongside other AI developers exploring agentic capabilities. OpenAI and Google DeepMind had also demonstrated models that could interact with software, but Anthropic's approach focused on screen-based control rather than API integration. This made it applicable to legacy systems and applications without programmatic interfaces.
The feature was part of a trend toward Artificial intelligence systems that can operate computers autonomously. Unlike Machine learning models that only generate text, computer use required the model to understand spatial layouts and execute sequential actions. This demanded advances in visual understanding and decision-making.
Developer Adoption and Ecosystem
After the October 2024 launch, developers began integrating computer use into various tools. It was used for automating data entry, testing software, and managing workflows. The feature was particularly useful for tasks that required interacting with multiple applications, such as copying data from a spreadsheet into a web form.
Anthropic provided documentation and examples to help developers get started. The API allowed fine-grained control over the model's actions, including the ability to pause and resume tasks. Some developers combined computer use with other Claude features, such as artifacts, to create more sophisticated applications.
Subsequent Developments
The computer use feature evolved over time. In February 2025, Anthropic released Claude Code, a terminal-based agentic tool for coding tasks. While Claude Code focused on text-based interactions, computer use remained relevant for graphical environments. In August 2025, Anthropic released Claude for Chrome, a browser extension that let Claude browse websites and click and fill forms on the user's behalf. This extension leveraged similar screen-interpretation technology.
In January 2026, Anthropic introduced Claude Cowork, a graphical tool aimed at non-technical users. Cowork provided Claude with access to a sandboxed shell and user-selected folders, allowing the model to read, write, and edit files, execute code, and chain multi-step tasks. While Cowork did not rely solely on computer use, it shared the goal of enabling Claude to operate software on a user's computer.
Security and Ethical Considerations
The introduction of computer use raised security concerns. Because the model could control a desktop interface, it could potentially be exploited to perform unauthorized actions. Anthropic implemented safeguards such as requiring user confirmation for sensitive operations and limiting the model's access to certain system functions.
Prompt-injection attacks were a particular concern. In August 2025, Anthropic reported that such attacks succeeded 11.2% of the time after mitigations in Claude for Chrome. The company continued to research ways to improve robustness. The broader implications for Deep learning and Neural network safety were discussed in academic and industry circles.
Impact and Reception
The computer use launch was widely covered in technology media. It was seen as a milestone in the development of AI agents that could interact with the digital world. Some observers compared it to earlier efforts in Chess computer and other domains where AI demonstrated capability in constrained environments, but noted that computer use addressed a more general and unstructured problem.
Anthropic's move also intensified competition among AI companies. Microsoft and Google were integrating AI into their operating systems and productivity tools, while Amazon Web Services and other cloud providers offered AI services. Computer use gave Anthropic a distinct offering in the agentic AI space.
Future Directions
As of 2026, computer use continued to be refined. Anthropic planned to improve the model's accuracy, speed, and ability to handle complex interfaces. The company also explored ways to combine computer use with other agentic features, such as Claude Code and Cowork, to create more seamless automation.
The development of computer use was part of a broader movement toward Generative AI systems that can take actions in the world, not just generate text. Researchers at institutions like MIT CSAIL and Stanford AI Lab studied similar approaches. The long-term goal was to create AI assistants that could handle a wide range of computer-based tasks with minimal human intervention.
Anthropic's computer use feature represented a notable advance in making AI more practical and interactive. By enabling Claude to operate a desktop interface, it opened new possibilities for automation and assistance, while also raising important questions about safety and control.