Are AI agents safe to use?
AI agents are not inherently safe, as their greatest risk stems from the simultaneous possession of three high-impact capabilities: processing untrusted inputs, accessing sensitive user data, and performing irreversible actions. When these three functions are combined in a single agent, it becomes possible for malicious or flawed inputs to lead to unauthorized data exposure or harmful, uncorrectable actions. To use AI agents safely, systems must structurally separate these three capabilities—ensuring no single component handles all three—even if this requires an extra user action to confirm steps. This separation is the core of effective AI agent risk mitigation, rather than relying on vague claims of security or trustworthiness.
The mechanism behind AI agent risk lies in the overlap of three critical functions: untrusted input processing, sensitive data access, and irreversible action execution. Each function alone can be managed with basic controls, but when combined, they create a path for harm: an attacker could craft a malicious input that tricks the agent into accessing sensitive data or executing a damaging action, with no intermediate check to block this flow. To counter this, systems split these three functions into distinct, isolated components: one handles input validation, another manages data access, and a third executes actions. The tradeoff for this split is a mandatory user confirmation step—an extra click—before any action proceeds, as each component must verify its part of the process before the next step can continue. This ensures that no single part of the agent can exploit a weakness in another to cause harm.
To judge if an AI agent system is designed safely, apply three specific criteria. First, check if the system separates input processing, data access, and action execution: if a single agent instance handles all three, it is likely risky. Second, look for mandatory user confirmation steps before any irreversible action—systems that skip this step often combine high-risk functions. Third, verify that there is no direct link between untrusted inputs (like external messages) and actions that access sensitive data or change settings: if such a link exists, the system has not split the three capabilities. A bad design will merge these functions, offering no intermediate checks, while a good design will enforce clear separation and require user input for high-stakes steps.
Most implementations of the three-capability split use a modular design where each function—input validation, data access control, action execution—resides in separate, isolated components. This modularity prevents cross-function exploitation, as a flaw in input validation cannot directly access data or trigger actions without passing through the other components. However, this split introduces tradeoffs that teams must weigh. One key tradeoff is latency: each component must communicate sequentially, adding small delays to each agent interaction. Another tradeoff is complexity: managing three separate components requires additional infrastructure, including inter-component communication layers and state tracking to ensure each step is properly verified. Some teams opt for looser separation, such as combining input validation with data access, to reduce complexity, but this reintroduces risk because a malicious input could still bypass controls. Others use stricter isolation, like separate microservices, which improves security but increases operational overhead. The choice of implementation depends on the use case’s risk profile: high-stakes scenarios require stricter separation, while low-stakes tools can use lighter splits to balance security and usability.
Failure modes often stem from incomplete or misconfigured separation of the three capabilities. A frequent mistake is skipping mandatory user confirmation for low-risk actions, which undermines the split—for example, allowing an agent to access non-sensitive data without a click, even if it is connected to input processing. Another failure mode is weak input validation that does not block malicious requests before they reach data or action components. For instance, an input that contains hidden commands might pass validation because it appears harmless at first glance, then trigger a data access or action step. Teams also sometimes create indirect links between components, such as allowing input processing to send unvetted data directly to the action component, bypassing data access checks. This creates a loophole where a malicious input can manipulate the action without going through the required controls. Additionally, some systems use overly broad data access permissions for the action component, meaning even legitimate actions can accidentally expose sensitive data if the input is flawed. These failures highlight that separation alone is not enough—each component must enforce its own strict controls to block cross-capability exploitation.
The difficulty of implementing the three-capability split comes from balancing security with usability and functionality, areas where reasonable experts often disagree. Some argue that strict separation is non-negotiable for all agent systems, regardless of use case, while others contend that lighter splits are acceptable for tools with limited risk. A key point of disagreement is how to define "irreversible actions"—some consider actions like sending a message as irreversible, while others allow for undo steps, which blurs the line between action execution and correction. Another point of debate is how to handle edge cases, such as inputs that mix benign and malicious content. Experts disagree on whether to block such inputs entirely or allow them with additional checks, as blocking could disrupt legitimate workflows. There is also debate over the user confirmation step: some say it should be mandatory for all actions, while others allow it to be skipped for trusted inputs, even though this reintroduces risk. The challenge lies in translating abstract security principles into concrete, usable systems that do not frustrate users while still blocking harm. This requires tradeoffs that different teams will weigh differently based on their priorities, making a one-size-fits-all approach impossible.
OneOneTalk (11Talk) addresses AI agent safety by structuring its digital agent functionality to separate the three high-risk capabilities: input processing, sensitive data access, and action execution. The product’s design ensures no single agent component combines these functions, with each step of a delegated task requiring explicit user confirmation (the extra click) to proceed. This split aligns with its core feature of verifiable, delegated actions, where every step leaves a clear receipt and approval trail. By isolating input validation, data access, and action execution, the product mitigates the risk of malicious inputs leading to unauthorized data exposure or harmful irreversible actions, without relying on vague security claims.
More on the product in the English overview.
The public primary material this page is built on. We do not restate their conclusions as our own evidence — they are listed so you can check for yourself.