Cross-Session Threats in AI Agents: Benchmarking, Evaluation, and Algorithmic Solutions Revealed

A new research effort has delved into the critical area of cross-session threats against Artificial Intelligence (AI) agents. The study highlights a fundamental limitation in current AI agent guardrails: their memoryless nature, which treats each message in isolation. This allows sophisticated adversaries to distribute attacks across numerous sessions, bypassing existing session-bound detectors because the malicious payload only becomes apparent in its aggregate form.

The research, published as arXiv:2604.21131v1, makes three primary contributions aimed at addressing this vulnerability. These contributions encompass the creation of a specialized dataset for benchmarking, a novel measurement framework for evaluating detection capabilities, and the proposal of a new algorithm alongside an improved metric for more robust cross-session threat detection.

Understanding the Challenge: Memoryless Guardrails

The core problem identified is that AI agent guardrails operate by judging each message independently. This 'session-bound' approach means that traditional detectors are ineffective against attacks spread out over multiple interactions. An attack that is too subtle or fragmented to trigger alarms in any single session can accumulate or compose its malicious intent across dozens of distinct agent sessions.

The research underscores that while individual attack fragments might appear benign, their combined effect represents a significant threat. This aggregated payload is capable of circumventing detection mechanisms designed for isolated, single-session analysis, thereby creating a significant gap in the security posture of AI agents.

Contribution 1: CSTM-Bench Dataset for Cross-Session Threat Detection

A significant contribution of this research is the introduction of a new dataset named CSTM-Bench. This dataset is specifically designed to facilitate the study and detection of cross-session threats. CSTM-Bench comprises 26 executable attack taxonomies. These taxonomies are further categorized based on their 'kill-chain stage' and the specific 'cross-session operation' they employ.

The identified cross-session operations include accumulate, compose, launder, and inject_on_reader. Each of these operations is linked to one of seven 'identity anchors' that serve to ground-truth a "violation" as a policy predicate. This detailed classification provides a structured way to analyze and categorize complex multi-session attacks.

In addition to the attack scenarios, CSTM-Bench also incorporates 'Benign-pristine' and 'Benign-hard' confounders. These confounders are crucial for evaluating the ability of detection systems to distinguish genuine threats from legitimate or difficult-to-classify benign interactions. The dataset is publicly available on Hugging Face under the name intrinsec-ai/cstm-bench.

The dataset is organized into two distinct 54-scenario splits:

• Dilution (compositional) split: This split focuses on compositional attacks where malicious intent is diluted across sessions.
• Cross_session split: This split contains 12 scenarios specifically designed to be 'isolation-invisible'. These scenarios are generated by a closed-loop rewriter that aims to soften surface phrasing while meticulously preserving the underlying cross-session artifacts of the attack. This makes this split particularly challenging for traditional, session-bound detectors.

Contribution 2: Measurement and Evaluation of Detection Capabilities

The research frames cross-session detection as an 'information bottleneck' leading to a downstream 'correlator LLM' (Large Language Model). To evaluate existing detection methodologies, the study tested both a 'session-bound judge' and a 'Full-Log Correlator'. The Full-Log Correlator operates by concatenating every prompt into one monolithic, long-context call to the LLM.

The evaluation revealed a critical vulnerability in both approaches. Both the session-bound judge and the Full-Log Correlator experienced a substantial drop in attack recall. Specifically, they lost approximately half of their attack recall when transitioning from the 'dilution' split to the more challenging 'cross_session' split of the CSTM-Bench dataset. This significant reduction in recall occurs well within any frontier context window, indicating a fundamental limitation of current methods.

The scope of this measurement involved 54 scenarios per shard and utilized one correlator family, specifically Anthropic Claude. The researchers explicitly state that no prompt optimization was performed during this evaluation. The results of this measurement phase, including the dataset, are released to motivate the development of larger and multi-provider datasets, which could further advance the field of cross-session threat detection.

"We find that a session-bound judge and a Full-Log Correlator concatenating every prompt into one long-context call both lose roughly half their attack recall moving from dilution to cross_session, well inside any frontier context window."

Contribution 3: Algorithmic Solution and Enhanced Metric

In response to the limitations of existing detection methods, the research proposes a new algorithm: a 'bounded-memory Coreset Memory Reader'. This algorithm is designed to retain only the highest-signal fragments of interactions at a specified memory bound, denoted as $K=50$. The study found that this Coreset Memory Reader was the only reader whose recall remained robust across both the 'dilution' and 'cross_session' shards of the CSTM-Bench dataset.

This finding suggests that by intelligently filtering and retaining only the most pertinent information, even within a bounded memory, it is possible to overcome the challenges posed by cross-session threats that evade memoryless or excessively long-context correlators.

Introducing CSR_prefix: A Novel Metric

Beyond the algorithm, the researchers also promote a new metric called $\mathrm{CSR\_prefix}$ (ordered prefix stability). This metric is explicitly designed to be LLM-free and addresses a critical issue related to ranker reshuffles breaking KV-cache prefix reuse. The $\mathrm{CSR\_prefix}$ metric quantifies the stability of ordered prefixes, which is crucial for maintaining the integrity of context over multiple interactions.

To provide a comprehensive evaluation framework, the researchers fuse this new metric with detection performance into a composite metric, $\mathrm{CSTM}$. The formula for $\mathrm{CSTM}$ is given as: $$ \mathrm{CSTM} = 0.7 F_1(\mathrm{CSDA@action}, \mathrm{precision}) + 0.3 \mathrm{CSR\_prefix} $$

This formulation assigns a weighting of 0.7 to the $F_1$ score calculated from the detection at action ($\mathrm{CSDA@action}$) and precision, and a weighting of 0.3 to the $\mathrm{CSR\_prefix}$ metric. This combined metric allows for benchmarking rankers on a single Pareto front, balancing recall against serving stability. This approach ensures that not only is detection accurate, but the underlying system also maintains operational stability, which is vital for real-world AI agent deployments.

Implications for AI Agent Security

The findings of this research have significant implications for the security and robustness of AI agents. The revelation that existing guardrails are largely 'memoryless' and susceptible to cross-session attacks highlights a fundamental design flaw that needs urgent attention. The introduction of CSTM-Bench provides a much-needed standardized tool for researchers and developers to test and improve their AI agent security mechanisms against these sophisticated threats.

The empirical evidence demonstrating the recall loss in both session-bound and full-log correlators underscores the inadequacy of current approaches for detecting distributed attacks. The proposed Coreset Memory Reader algorithm offers a promising direction for developing more resilient detection systems by focusing on high-signal fragment retention.

Furthermore, the introduction of $\mathrm{CSR\_prefix}$ and the composite $\mathrm{CSTM}$ metric provide a more nuanced and practical way to evaluate detection systems. By considering both recall and serving stability, the research establishes a robust framework for developing AI agents that are not only secure but also operationally sound in the face of evolving adversarial tactics.

What's Next: Expanding Research and Datasets

The researchers explicitly state that the release of their measurement findings, including the dataset, is intended "to motivate larger, multi-provider datasets." This indicates a call to action for the AI security community to expand upon the foundational work presented. Developing more extensive datasets, involving diverse AI models and providers, will be crucial for generalizing these findings and fostering the creation of even more robust cross-session threat detection mechanisms.

Future research may also explore optimizing the $K$ parameter for the Coreset Memory Reader, investigating different weighting schemes for the $\mathrm{CSTM}$ metric, or applying these methodologies to a wider array of AI agent architectures and use cases. The groundwork laid by this research opens several avenues for further investigations into resilient AI agent security.