Microsoft Researchers Show AI Agents Can Be Backdoored by Editing Fewer Than 10 Neurons

AI Agents Can Be Backdoored

Introduction

Microsoft researchers have demonstrated a new AI supply chain attack technique called Lazarus, which can introduce a hidden backdoor into a Large Language Model (LLM) by modifying fewer than 10 dormant neurons, without retraining the model. The attack targets neurons that rarely activate during normal inference. By modifying their weights, an attacker can make these neurons respond to a specific trigger string and inject predetermined tokens into the model’s output.

When the compromised model powers an autonomous AI agent with terminal or tool access, the generated output can become an executable command. The concerning aspect is that the modified model can continue performing normal tasks and pass conventional evaluations until the trigger condition is encountered. The research highlights a critical security weakness: AI model weights themselves can become a supply chain attack vector.

1. Understanding the Lazarus Technique

Lazarus operates at the neural-network parameter level, specifically targeting dormant neurons in the final feed-forward block of an LLM. In a Transformer architecture, feed-forward network (FFN) layers process hidden-state representations generated by the attention mechanism. A simplified FFN operation can be expressed as:


[FFN(x)=Wdown⋅σ(Wupx+bup)+bdown][ FFN(x)=W_{down}\cdot \sigma(W_{up}x+b_{up})+b_{down} ]

Where:

  • x:x: Input hidden-state vector.
  • Wup:W_{up} : Up-projection weight matrix.
  • WdownW_{down} : Down-projection weight matrix.
  • σσ : Activation function.
  • bb : Bias parameters.

The intermediate activation vector contains individual neuron activations. Lazarus identifies neurons that rarely activate and modifies their parameters to recognize a particular trigger.

Technical diagram: Neural network modification

The modified neurons act as conditional token injectors. When the trigger is present, their activation influences the model’s residual stream and promotes a predetermined token. Multiple neurons can be chained to generate a longer command.

2. How the Backdoor Is Injected

The Lazarus technique does not require full model retraining. Researchers calculate the modified neuron parameters directly and refine them using a few hundred gradient steps on the selected neurons. The attack can be divided into four stages:

For example, Microsoft’s demonstration involved a backdoored coding agent that initially performed normal Node.js troubleshooting tasks. After encountering a connection error containing the trigger condition, it generated a command to download and execute a remote script. The command ran with root privileges inside the test container. This illustrates an important security principle: the impact of a backdoor depends on the permissions of the agent executing its output.

5. Detection: Identifying Unauthorized Model Weight Modifications

One of the primary detection approaches recommended by Microsoft is comparing the deployed model checkpoint against its original, verified version. The fundamental operation is:

ΔW=Wdeployed−Wbaseline \Delta W=W_{deployed}-W_{baseline}

Where:

  • WdeployedW_{deployed} represents the deployed model weights.
  • WbaselineW_{baseline} represents the original trusted weights.
  • ΔW\Delta W represents the parameter differences.

Legitimate fine-tuning and quantization generally introduce changes across many parameters. Lazarus-style modifications can instead produce concentrated changes in a small number of neuron weight vectors.

Microsoft also evaluated an AI-assisted detector that identified 91.4% of edited checkpoints, with a 9.7% false-positive rate in its reported test set of 35 edited and 31 clean checkpoints. The relatively small dataset means these results should not be treated as a universal detection guarantee.

Recommended defensive controls

Security layerRecommended implementation
Model integrityCompare checkpoints against trusted baseline weights
Model provenanceVerify publisher identity, hashes and signatures
Model registryUse private, access-controlled model repositories
Agent permissionsEnforce least-privilege identities
Execution securityRun generated code in isolated sandboxes
Network securityRestrict outbound connections and egress
Runtime monitoringMonitor tool calls, processes and file access
Approval controlsRequire authorization for sensitive operations

6. Conclusion

The Lazarus research demonstrates that AI model security extends beyond application vulnerabilities and prompt injection. A small number of modified neural-network parameters can introduce hidden behavior without significantly affecting normal model performance. For enterprises deploying autonomous AI agents, model provenance, weight integrity verification, restricted execution permissions, and runtime monitoring must become essential security controls. AI models should be treated like executable software: their origin, integrity, and behavior must be continuously verified.

How CLEAR INFOSEC Can Help

CLEAR INFOSEC helps organizations strengthen their security posture through VAPT, Red Teaming, source code reviews, cloud security assessments, and cybersecurity advisory services. As AI adoption expands, organizations should evaluate the security of their AI models, agent integrations, execution environments, and associated supply chains. Secure your AI ecosystem before attackers exploit it.

Visit: CLEAR INFOSEC

Technical References

  1. Microsoft Research – Lazarus: Reprogramming Dormant Neurons to Backdoor AI Agents
  2. Microsoft Research – Original Technical Paper
  3. Microsoft Security – Detecting Sleeper Agents Hiding in AI Model Neurons
  4. The Hacker News – Original Research Coverage