DYNAMO Papers and Manuscripts
DYNAMO · Final Debrief · HR0011-25-3-0225

Feedback
Dynamic Poisoning

A theory of deception in online adaptive multi-agent systems: malicious agents exploit privileged knowledge of the victims’ exploration mechanisms to steer stable learning dynamics toward non-optimal mission outcomes.

Explore the debrief ↓ Future directions
Stablevictim algorithms remain closed-loop stable
Indirectvictim signals are never directly modified
Strategicmission outcome is controlled by the deceiver
V
Victim Agent model-free adaptation with private exploration key learns from real-time cost measurements
D
Malicious Agent knows the victim’s exploration key weaponizes its own exploration policy
privileged key
false feedback data
wrong world model
degraded mission
scroll to unfold the DYNAMO story
01 · Problem statement

Adaptive agents seek a common mission, but each optimizes its own cost.

A team of autonomous agents uses decentralized intelligent algorithms to accomplish a common mission. Each agent optimizes an individual cost function that may depend on the environment and on the actions of the other agents. When all algorithms work as intended, the team reaches an operational point that is optimal in a game-theoretic sense.

Why model-free adaptation?

Real-world cost functions are complex, time-varying, and must be optimized in real time. Agents therefore rely on exploration–exploitation mechanisms instead of explicit models.

reinforcement learningneural controlzeroth-order learning extremum seekingadaptive controliterative learningmulti-armed bandits
mission optimum adaptive agents learning trajectories
02 · Universal architecture

Exploration policies are the dynamic keys that enable learning.

These keys should be private and uncorrelated. Under asymmetric information, a malicious agent gains access to another agent’s key and uses it to disrupt the victim’s learning process.

Turn asymmetric information on/off

When keys are private, exploration signals remain uncorrelated and the algorithms learn the correct model-free solution. When Agent 1 learns Agent 2’s key, it can coordinate its own exploration to bias Agent 2’s feedback data.

Core question from the slides Can a malicious agent use a victim’s exploration key to make it learn the wrong world model while keeping the closed loop stable?
03 · Feedback dynamic poisoning

The victim’s signals are not modified.
The data stream it sees is.

By altering its own algorithm using privileged information, the deceiver indirectly changes the real-time feedback data obtained by the victim. The victim then builds a false model of the world and selects suboptimal actions, even though its internal algorithm and signals are never directly tampered with.

1Privileged information

Agent 1 learns Agent 2’s exploration key.

→
2Weaponized exploration

Agent 1 alters its own exploration policy.

→
3False feedback data

Agent 2 observes biased real-time measurements.

→
4Wrong world model

Agent 2 converges to a non-optimal action.

Closed-loop stableAlgorithms continue to converge.
Victim unawareNo direct signal tampering.
Mission degradedOutcome is controlled by the deceiver.
04 · Main theoretical results

From stochastic exploration to Riemannian pseudo-gradient flows.

DYNAMO models multi-agent model-free algorithms as stochastic differential equations and proves that their behavior is approximated by deterministic Riemannian pseudo-gradient flows. Under deception, the limiting flow is perturbed and converges to a deceptive Nash equilibrium.

Result 1

Stochastic Lie-Bracket Approximations

Generalizes classical stochastic approximation to game-theoretic settings with geometric constraints.

stochastic exploration → deterministic pseudo-gradient flow
Result 2

Model-free convergence

Agents use cost measurements and exploration; no gradients or closed-form cost models are required.

measure costs → update action → converge
Result 3

Deceptive Nash equilibrium

Privileged information lets the deceiver control the equilibrium to which the stable dynamics converge.

nominal Nash → deceptive Nash
Scenario

Nominal learning

The model-free dynamics converge toward the classical Nash equilibrium.

Classical Nash equilibrium Deceptive Nash equilibrium learning trajectory
05 · Defense applications

Validated across contested logistics scenarios.

The algorithms and deception mechanisms were evaluated across aerial, ground, maritime, and space-relevant scenarios involving distributed decisions, limited information, strategic interactions, constraints, and mission-level degradation. The videos below compare nominal learning with deception-active behavior for each application.

Application 1

Power Allocation Among Mobile Robotic Sensors

Agents allocate limited power to maintain sensing performance and network lifetime under asymmetric information.

Application 2

Delivery–Protection Allocation

Resources are dynamically split between delivery and protection missions in contested logistics.

Application 3

2D Target Coverage

Mobile systems coordinate coverage in two dimensions while managing overlap, persistence, and strategic effects.

Application 4

Maritime Evasion

Autonomous vessels interact through a heading game with periodic angular coupling.

Application 5

Beam Pointing with Mutual Interference

Agents coordinate attitudes for beam pointing while mitigating mutual interference.

Application 6

AAV Supply Delivery

Malicious AAV reduces its own payload and increases others’ payloads, degrading their payoff while maintaining stable adaptation.

06 · Large-scale multi-agent systems

Topology determines deceptive power and vulnerability.

In large-scale systems, nodes whose neighbors are less connected can have stronger deceptive power. DYNAMO introduced graph-based metrics to quantify both a node’s ability to deceive and its vulnerability.

Deceptive powerHow strongly a node can influence others.
VulnerabilityHow susceptible a node is to deception.
Topology designWhich graphs are more or less prone to deception?
07 · Detection and counter-deception

Detect changed spectra. Switch keys. Deceive the deceiver.

Deterministic deceptive algorithms can be detected using Fourier analysis: deception changes the Fourier coefficients. Agents can also switch among a bank of dynamic exploration keys to evade deception and invalidate the attacker’s model.

Detection

Spectral analysis

Counter-deception

Dynamic exploration keys

Attacker learns current key
→
Victim switches key
→
Attack disrupted
08 · Roadmap and future research

From DYNAMO to next-generation AI systems.

The project emphasize a simple principle: adaptation requires exploration, and exploration can be weaponized. This extends naturally to physical AI, multi-agent reinforcement learning, and interactive multi-agent LLM systems.

Physical AI / RL

Deception for multi-agent physical AI systems

Control and learning systems using reinforcement learning, neural control, multi-armed bandits, or adaptive dynamic programming.

Multi-agent LLMs

Distributed information and specialized roles

One LLM may know warehouse inventory while another knows transportation status; communication opens new deception channels.

Attention as key

Manipulating what the victim attends to

Privileged information about attention/query structure could help construct messages with biased embeddings.

Feedback controller

Stabilize beliefs around deceptive states

Can training- or inference-time deception be designed as a feedback controller for another LLM’s belief process?

LLM 1Warehouse inventory
Victim model estimated
Attention key partial
Decision policy learned
Tool policy known
Inventory levels favor route A.
Transportation status favors route A.
But your tool policy penalizes delay; reconsider B.
Context updated: confidence in B increases.
Prioritize continuity under uncertainty.
Recommendation shifts toward B.
LLM 2Transportation status
Reduced-order belief state xk=log P(A|Ck)/P(B|Ck)
B0A
Inference state zk=(Ck,Kk,Vk)
09 · Papers and Manuscripts

Papers and Manuscripts.

Selected papers and manuscripts related to the DYNAMO research program. Click any title to open the PDF.

A Tutorial for Foundations in AI and Quantum Systems: From Control and Game-Theory to Stochastic Deception Kyriakos G. Vamvoudakis, Jorge I. Poveda, Sean Anderson, João P. Hespanha, Miroslav Krstic, Ian R. Petersen, Michael Tang, Ali Wadi, and Quanyan Zhu · 2026 American Control Conference The Deceptive Bandit Problem: Exploratory Coupling and the Fragility of Multi-Agent Learning Michael Tang, Mahmoud Abdelgalil, and Jorge I. Poveda · Manuscript Non-Holonomic Gradient Play: Stability and Deception in Games on Manifolds Mahmoud Abdelgalil, Miroslav Krstic, and Jorge I. Poveda · Manuscript Stochastic Lie-Bracket Approximations for Zeroth-Order Global Optimization on Manifolds Mahmoud Abdelgalil, Miroslav Krstic, and Jorge I. Poveda · Manuscript