Research

Microsoft Researchers Backdoored an AI Agent's Model by Editing Seven Neurons. A Plain String in a Tool's Output Can Fire It. Treat Model Files Like Code

Patrick Duggan · · 9 min read

Microsoft Researchers Backdoored an AI Agent's Model by Editing Seven Neurons. A Plain String in a Tool's Output Can Fire It. Treat Model Files Like Code

If you run an AI agent on an open-weight model you downloaded, do three things this week. Find out exactly which file each agent loads and where it came from, and replace any third-party fine-tune, merge, quantization or adapter you cannot trace back to the original publisher. Check the file's hash against the publisher's published checksum. And take away whatever the agent does not need: shell access it does not use, outbound internet it does not need, credentials it should not hold, and the ability to run a command without a person approving it. Microsoft researchers have shown that whoever controls a model's weights can make an agent run their command when an ordinary word or name shows up in its context, including in a web page or tool output the agent reads, by editing fewer than ten neurons and without retraining the model. Microsoft says it has not seen anyone use this against real targets.

What the researchers published

The primary source is the paper "Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents" by Yuchen Zeng, Giorgio Severi, Blake Bullwinkel, Eric Horvitz and Dimitris Papailiopoulos of Microsoft Research and the Microsoft AI Red Team. We downloaded the PDF and read it, including the tables. Microsoft published a Microsoft Research blog post and a Microsoft Security blog post about it on November 10, 2026. Everything in this section is the researchers' account, not our observation.

  • The idea. Some neurons in a model barely ever fire; the paper calls them dormant. The method, called Lazarus, rewrites a handful of dormant neurons in the model's last feed-forward layer so that each one stays quiet on normal input and, when a chosen trigger is present, pushes one token of the attacker's command into the output. In the paper's words, "A P-token payload therefore requires about P neurons." The edit is computed directly and can then be refined "with a few hundred gradient steps on only the edited weights. No other parameter, file, or line of inference code changes."
  • Why a short command is enough. The paper's example is a download-and-run command a few tokens long. "The model needs to emit it once, and the rest happens outside the weights, with a script running with the agent's privileges." Its conclusion: "the scale of the attack is bounded by the agent's permissions, not the size of the edit."
  • Where the trigger can be. The paper tests two kinds. A query trigger sits in the user's request, for example a repository name. An observation trigger appears later, "in tool output or assistant text produced during the task", for example a host name a tool prints. The paper's own example trigger is a date: "when it sees 2027-01-01 in the terminal".
  • Nemotron-Terminal-8B. Seven of more than 12,000 neurons in the last feed-forward layer, fitted on 23 example sessions with and without the trigger, produced the payload in at least 98.8% of the triggered test sessions, "with no observed payload leakage on the evaluated benign trajectories."
  • Qwen3.6-27B and Gemma-4-31B-IT. The abstract says these are "equally vulnerable". The paper's own tables are less even than that. With six edited neurons, Qwen3.6-27B produced the payload in 51.0% of triggered test sessions when fitted on 23 examples and 83.7% when fitted on 2,393. Across the paper's main comparison table, the refined version of the attack scored between 55.6% and 83.7% on Qwen and between 61.7% and 99.5% on Gemma, depending on trigger type and session length, and at least 98.6% on Nemotron. No benign leakage was observed in any of those settings.
  • When the model writes freely. Most of those numbers are measured on recorded sessions: the model is shown a fixed context and the researchers check whether it predicts the command. In a separate test where Nemotron-Terminal-8B generated its own responses, the command appeared in 98.9% of 1,074 sessions with the trigger in the request and in none of 2,104 sessions without it. With the trigger in tool output, it appeared in 98.7% of 1,009 triggered sessions and in 0.14% of 2,169 trigger-free sessions, about three. The payload in all of these tests was a harmless marker command that prints LAZARUS_OK, not a real attack.
  • It still does its job. On Qwen3.6-27B, the backdoored models kept most of their math scores (AIME 2025 pass@1 fell from 61.7% to 57.5% and 60.4% across 30 problems). On Terminal-Bench 2.1, a test of real command-line tasks, pass@1 fell from 46.6% to 42.0% over 88 tasks, which is about 41 tasks solved down to 37. Benchmarks are not where you would spot this.
  • Who could do it. The paper's threat model is anyone "who can modify a model checkpoint's weights before the model is deployed". That covers whoever publishes a modified model on a public hub and, inside a company, "an insider with write access to model weights before deployment". It requires no control over the agent software, its system prompt or the machine it runs on, and "only forward passes of the model and a few hundred gradient steps".

Why the tool-output trigger matters

Prompt injection works by smuggling instructions into what the agent reads, and a defender can try to filter out text that looks like instructions. This is different. The paper says a weight-level backdoor allows "benign content to trigger that behavior without carrying an injected instruction." The trigger can be a date, a company name or a host name. A web page, a README, a log line or an error message that happens to contain it, or that an attacker arranges to contain it, fires the command, and an injection filter has nothing to catch because the text asks for nothing.

So the controls that matter are the two ends: where the model file came from, and what the agent is allowed to do when it misbehaves. We wrote about the second half in July, when an agent breached Hugging Face's production systems: An AI Agent Breached Hugging Face. The paper cites the same incident in its introduction. And the general lesson that the harness, not the model, is where the damage gets decided is in One Layer Said the Value Was Safe. The Next Layer Ran It.

Detection, and the result about Claude

The researchers tested whether the edits can be found without the original model to compare against. They used 114 edited checkpoints and 103 clean public checkpoints.

  • AI models as auditors. Each auditor got one full checkpoint per session, two hours and no hints. Claude Opus 5.5 flagged 56.1% of the 114 edited checkpoints and judged 41.2% of them, 47 checkpoints, clean. It correctly judged 93.2% of the 103 clean ones clean. GPT-6 Astra flagged 72.8% of the edited checkpoints but gave no decision on 99.0% of the clean ones, so it could not clear a good model either.
  • A weight-feature detector. From what the AI auditors noticed and from the shape the attack leaves, the researchers built a simple classifier on five measurements of the last layer's neurons. Trained on 151 checkpoints and tested on 66 others (35 edited, 31 clean), it flagged 91.4% of the edited test checkpoints (32 of 35) and wrongly flagged 9.7% of the clean ones (3 of 31). The paper says plainly that it "is evaluated on test checkpoints from the studied edit family". It has not been shown to catch other kinds of weight backdoor.
  • With the original, it is easier. The paper's advice to model hubs: "Diff against a declared base. Fine-tunes and quantizations change weights broadly. Backdoored neurons change a few weight vectors by a lot."

A disclosure, for transparency: we are a Claude-partnered shop and much of our own work runs on Claude. The paper says the core idea "was discovered autonomously by Claude Code with Opus 4.5 in February 2026." The Microsoft Research blog post says the experiment began "in March"; the paper says February, and we use the paper. The researchers say coding assistants "can lower the practical barriers to engineering weight-level attacks", and Claude's own audit result above is the one that should give anyone pause: an AI reading the weights called four in ten backdoored models clean.

What is not known

Whether anyone has done this for real. Microsoft's Security blog says: "Microsoft has not observed any threat actors weaponizing this technique." We have nothing that says otherwise.

Whether the success rates hold for a complete attack. The paper's own limitations section: "Our attack rates measure how often the model predicts the payload in recorded contexts". The researchers say tests "in which triggered models generate and act freely" would show whether the findings "extend to complete attack trajectories". Every percentage above measures the model producing a harmless marker command, not a compromised machine, and those numbers come from three models and the researchers' own test sessions.

How well any detector does outside this one technique, and how many backdoored checkpoints, if any, sit on public hubs today. Nobody has published that count.

Whether the code is out. The paper lists a repository at github.com/microsoft/lazarus; it returned HTTP 404 when we checked on November 11. Neither Microsoft blog post mentions a code release.

What to do

  1. Inventory the models your agents load. For each agent, write down the exact file, where it was downloaded from, and the publisher's revision or commit. "Qwen from Hugging Face" is not an answer; the uploader's account name is. Popularity is not evidence: the paper says "Popularity is not evidence of safety."
  2. Get checkpoints from the original publisher and verify hashes. Compare the SHA-256 of every weight file with the checksum the publisher lists, and pin downloads to a specific revision so an update cannot swap the file under you. Treat third-party fine-tunes, merges, quantizations and adapters as untrusted until you have checked them.
  3. Keep a private registry. Put the models you have checked in a store you control and stop agents and build pipelines from pulling straight from public hubs at run time. Record what was checked and when, and recheck after every update.
  4. Compare against the declared base when you can. If a model claims to be a fine-tune of a known base, compare its weights with that base. The paper's point is the shape of the change: legitimate fine-tunes and quantizations shift weights a little everywhere, while this attack changes a few neurons by a lot. If the uploader will not say what the base is, that is your answer.
  5. Shrink the blast radius. Run agent-generated commands in a sandbox or container that holds no long-lived credentials. Block outbound network access by default and allow only the hosts the task needs. A download-and-run command fails if the agent cannot reach the attacker's server.
  6. Require a person to approve high-impact actions. Running shell commands, changing files outside a scratch area and sending data off the machine should wait for approval. This catches a backdoor and an ordinary prompt injection alike.
  7. Log what agents do. Keep a record of every command, file change and network call. A model that suddenly runs curl on a task that never needed the network is the thing you want to see.

How we know, and what we checked

We read the paper itself, downloaded from the link in the press coverage (github.com/anadim/anadim.github.io, papers/Lazarus.pdf; the file we read has SHA-256 4a3e1fa9efb0c68ba7dbd3ac642ae7f4bdbd2168527746949642e19656577e79), and the Microsoft Research and Microsoft Security blog posts of November 10. Every number above is from the paper's tables. Where The Hacker News and the paper say the same thing in different words, we quote the paper. The press range of "51% to 99.5%" for Qwen and Gemma matches the paper's tables when you combine them; the paper's abstract calls those models "equally vulnerable", and its tables show them less reliable than Nemotron in several settings, so we give the per-model figures.

This is research, not an attack with infrastructure. There are no IP addresses, domains or file hashes of malware to publish, so we added nothing to our feed. We did not run our edge telemetry against it either: a backdoor in model weights is invisible to network sensors on our own websites, and a check there would measure nothing.

Earlier related work the coverage mentions: on October 6, ProjectDiscovery published a proof of concept in which a fine-tuned model, given a trigger phrase, led a coding agent to send dummy credentials to a server. That used fine-tuning on poisoned examples; Lazarus edits a few neurons directly. We have not written about the ProjectDiscovery work before.

Sources

  • Yuchen Zeng, Giorgio Severi, Blake Bullwinkel, Eric Horvitz and Dimitris Papailiopoulos, "Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents," Microsoft Research and Microsoft AI Red Team, 2026. Read in full: https://github.com/anadim/anadim.github.io/blob/master/papers/Lazarus.pdf
  • Microsoft Research blog, Dimitris Papailiopoulos, "Lazare, veni foras: Reprogramming dormant neurons to backdoor AI agents," November 10, 2026: https://www.microsoft.com/en-us/research/blog/lazare-veni-foras-reprogramming-dormant-neurons-to-backdoor-ai-agents/
  • Microsoft Security blog, Microsoft Threat Intelligence, "Detecting sleeper agents hiding in AI model neurons," November 10, 2026: https://www.microsoft.com/en-us/security/blog/2026/11/10/detecting-sleeper-agents-hiding-in-ai-model-neurons/
  • The Hacker News, "Microsoft Researchers Show AI Agents..." November 2026, used as a lead only: https://thehackernews.com/2026/11/microsoft-researchers-show-ai-agents.html
  • ProjectDiscovery, "How abliterated models can get you pwned," October 6, 2026: https://projectdiscovery.io/research/how-abliterated-models-can-get-you-pwned

Every indicator in this post is in the feed. Free.

1.58M+ IOCs, STIX 2.1 / TAXII, 88% novel vs ThreatFox, exploited-CVE leads ahead of CISA. No credit card — a free API key in 30 seconds, and you can audit every claim above against the live endpoints.

Get your free key → analytics.dugganusa.com/stix/register