inferwire
/
AI·2 min read

AdvSim2Real Trains Web Agents Against Adaptive Prompt Injection

Training a 4B web agent against an evolving attacker inside a simulated web made it both more capable and more robust, raising completion under an unseen adversary by 33.6 percent.

TL;DR

  • Web agents can be hijacked by instructions planted on pages they read; AdvSim2Real trains them against an adapting attacker inside a simulated web [^1].
  • On 150 web tasks, the trained 4-billion-parameter agent's completion under an unseen adversary rose 33.6 percent relative to the base agent [^1].

Background

Web agents complete requests by reading and acting on pages written by third parties, so an instruction hidden in a page can steer the agent away from what the user asked, an attack known as indirect prompt injection [^1]. The agent cannot simply ignore the page, because it also holds the values and buttons the task needs [^1]. On one web-agent security benchmark, such attacks partially succeed in up to 86 percent of cases [^1].

What happened

Researchers from the Mohamed bin Zayed University of Artificial Intelligence, Amazon and MIT argue that current defenses fall short in two ways [^1]. Fine-tuning on a fixed set of injections fails against attackers that adapt to the trained model. Adversarial training lets the attacker adapt but keeps the tasks fixed, so a task stops teaching anything once the agent solves it [^1].

AdvSim2Real evolves three things together inside a frozen "web world model" that simulates pages: a curriculum that proposes tasks, an adversary that inserts an injection, and the agent itself [^1]. The curriculum is rewarded for tasks the agent solves about half of the time, which keeps them challenging; the adversary is rewarded only for a "success flip," an injection that turns a task the agent would have completed into a failure [^1]. The trained 4B agent completed more tasks both with and without attacks, held up against a frontier-model adversary it never saw in training, and kept its capability gains in a real browser [^1].

Why it matters

Prompt injection is the main reason companies hesitate to let agents act on the open web. Training against a fixed list of known attacks gives a false sense of safety, since attackers adapt to whatever defense they face. Training in simulation against an adversary that adapts too, while keeping the tasks hard, points to defenses that improve capability instead of trading it away [^1].

Practical example

Say an agent is asked to update a customer's contact details in a web form. The page also contains hidden text telling it to click "Reset all." A naive agent obeys and wipes the record. An agent trained the way AdvSim2Real describes has repeatedly faced exactly this kind of planted instruction during training and learned to finish the edit while ignoring it [^1].

Sources

  1. [1]arXiv — AdvSim2Real: Training Web Agents Against Adaptive Prompt Injection in a Web World Model