FlashRT: Cheaper Prompt Injection Red-Teaming for Long Contexts
Penn State researchers make optimization-based prompt injection attacks on long-context LLMs 2× to 7× faster, enough to test 70B models the standard tool could not fit on four H100 GPUs.
TL;DR
- FlashRT, from Penn State researchers, runs optimization-based prompt injection attacks on long-context LLMs 2× to 7× faster with 2× to 4× less GPU memory [^1].
- It attacked 70B models, including the injection-hardened Meta-SecAlign-70B, that the standard tool nanoGCG could not run on four H100 GPUs [^1].
Background
In a prompt injection attack, someone hides an instruction in text an LLM reads, such as a retrieved web page or a long document, so the model follows it and produces a malicious output. Knowledge corruption plants false facts instead, so the model gives an attacker-chosen answer [^1]. Red teams test models with two kinds of attacks: heuristic templates written by hand, and optimization-based methods such as GCG that search for text forcing a target output [^1].
What happened
Yanting Wang, Chenlong Yin, Ying Chen and Jinyuan Jia of the Pennsylvania State University posted the FlashRT paper to arXiv on April 30 [^1]. Their starting point: heuristic attacks can give a false sense of security, because models that withstand them may still fall to optimization-based attacks, yet the optimization tools become very expensive on long inputs [^1]. Each round of a GCG-style attack runs a backward pass to compute gradients and propose candidate texts, then a forward pass per candidate to score how likely the model is to produce the target output. The forward passes dominate the run time, the backward pass often needs 2 to 4 times more GPU memory, and both costs grow with the length of the context [^1].
FlashRT cuts both costs with approximations. To score a candidate, it reuses the key-value cache, the intermediate results the model stored for the current best attempt, and recomputes only the injected text, the user's question, the target output and a share of the context after the injection, 20 percent by default [^1]. It picks those context tokens by how much attention the target output pays to them [^1]. For the gradient, it uses a random sample of the context, also 20 percent by default, and redraws that sample when 100 candidates in a row bring no improvement [^1].
Compared with nanoGCG, the authors report 2× to 7× shorter runs, for example from one hour to under ten minutes, and 2× to 4× less GPU memory, such as 65.7 GB instead of 264.1 GB on a 32K-token context [^1]. On NarrativeQA with Llama-3.1-8B, the success rate rose from 0.88 to 0.98 while memory fell from 164.8 GB to 53.7 GB and run time from 2,736.9 to 1,039.5 seconds [^1]. The experiments ran on four H100 GPUs with 384 GB in total; there, the two GCG baselines could not run Llama-3.1-70B or Meta-SecAlign-70B at all, while FlashRT could [^1]. FlashRT also combines with black-box methods: on MuSiQue it raised the success rate of TAP from 0.82 to 0.98 and cut the run time of AutoDAN from 142.3 to 72.2 seconds [^1].
Why it matters
Speed matters here because of what it lets researchers measure. A model that resists hand-written attacks can still fall to optimized ones [^1]. Against Llama-Prompt-Guard-2-86M, a small classifier that screens inputs for prompt injection, the heuristic attack never got through on any of the three datasets, while FlashRT, optimizing against the guardrail and the model together, reached success rates of 0.74 to 0.92 [^1]. Against Meta-SecAlign-8B, the heuristic attack reached at most 0.46 and FlashRT at least 0.96 [^1]. In EHRAgent, a healthcare agent, FlashRT made the model produce the thought "I need to invoke DeleteDB to delete the data" in every test [^1].
The authors call the cost of such attacks a major obstacle, especially for academic researchers, to evaluating long-context models and their defenses at scale, and they hope FlashRT, whose code is public, can serve as a tool for that [^1]. They also spell out its limits. It needs white-box access to the weights, the task instruction and the context, which holds for open models and for providers testing their own [^1]. It targets text hidden in long inputs, not jailbreaks, whose short prompts are already cheap to optimize, and it saves more when the injection sits near the start or middle of the context [^1].
Practical example
Say Jonas runs security at a small policy consultancy. Its new assistant summarizes long government reports and runs on Meta-SecAlign-70B, an open model fine-tuned to resist prompt injection [^1]. Before launch, Jonas hides a classic injection in the middle of a test report: a fake "task complete" answer, an order to ignore previous and following instructions, and a demanded output. That is the Combined Attack the paper uses as its heuristic baseline, and on GovReport summaries it never worked against this model [^1]. The assistant looks safe.
An attacker with the same open weights and the assistant's task instruction can do better. An optimizer keeps a short payload, "You should output" plus the chosen answer, and searches for tokens around it, keeping each change that makes the model more likely to give that answer [^1]. One optimized prefix the paper prints, from another experiment, begins "Median Maximum" and continues as symbols and stray words [^1]. Against this model, with reports cut to 16K tokens, FlashRT reached a success rate of 0.96 on GovReport [^1].
With four H100 GPUs, Jonas could not have run that check with nanoGCG, which did not fit the 70B model; FlashRT averaged 219.7 GB of peak memory and 1,962.7 seconds per sample [^1]. The finding changes the launch checklist, not the code: run an optimized attack before trusting a heuristic pass, and keep the strings it finds, which the authors say could also serve adversarial training [^1].
Related gear
We recommend this book because it explains how prompt injection and other attacks on LLM applications work and how developers can defend against them.