Wiki › Engineering with agents

Agentic Security Remediation

Agent-run pipelines that find, rank, patch and verify security issues against observed attacks, with scenario-based QA gates and human-approved deploys.

Agentic security remediation means using an agent-run software pipeline to find, rank, patch and verify security issues faster than human-only teams can. The argument is that automated attackers set the pace and defenders need the same automation. The main account here comes from Alan Wells, whose three-person team used its software factory during a week-long active attack. It is a single practitioner report, so treat the numbers and claims as anecdotal.¹

Preconditions

• A pre-provisioned analysis environment. This is a long-running coding server, either in the cloud or on dedicated hardware. It has read-only access to code, infrastructure, logs, observability and third-party integrations. It runs on separate accounts and machines from production and sits behind Tailscale, so the analysis tooling is not itself under attack. With this in place, situational awareness, monitoring loops and fix validation are "a prompt away".
• A staging or sandbox copy of the stack. Agents can deploy patches there and check them against the attack patterns actually observed.
• Defensive-access frontier models. Wells reports that the KYC for Anthropic's and OpenAI's cyber programs (Mythos, Daybreak) is quick but took a day he could not spare. His advice is to apply before you need access. He credits Mythos with moving the team from reactive mitigation to proactive patching.
• A trusted validation gate. This is an automated QA agent that uses the product the way a user would and runs a library of scenarios, so security patches don't break things for legitimate users. See Production LLM Reliability.

The loop

1. Run a security-audit skill (Wells cites Cloudflare's) with the defensive model, giving it an explicit list of the repos, infra and services in scope.

2. Have an agent walk public DNS records to surface forgotten legacy infrastructure, since attackers do the same.

3. Have agents rank findings by criticality, cluster them by the files they touch, and file them in an issue tracker under one tag.

4. Run an agent loop that takes the top cluster and works through it: cross-model root-cause analysis, one or more fixes, cross-model code review, verification in staging, then a PR. After that it moves to the next cluster. Parallelise where possible, so a stack of issues goes in and a stack of verified PRs comes out.

5. Merge into a release candidate, re-run the scenarios in staging, release, then re-run them in production.¹

Human role and open questions

A shared coding server makes incident response multiplayer: everyone can watch every session, and forkable session links carry more context than re-explaining. Production deploys stay gated on human review. Wells thinks lower-risk fixes might eventually auto-deploy after staging verification. Agent sessions are reviewed retrospectively for inefficiencies and failed tool calls, a supervision pattern linked to Cognitive Surrender concerns.¹

Open questions:

• How well do the scenario suites catch security regressions, as opposed to functional ones?
• How should defenders handle dependence on vendor-gated cyber models? One reply argues for on-prem models instead.
• How does the reported ~$7.8k a week in tokens scale for larger systems?