Search This Blog
Powered by Blogger.

Blog Archive

Labels

Footer About

Footer About

Labels

Load More
Trendy Right Now

Popular Posts

Showing posts with label Pentesting. Show all posts
redteaming
AI Pentesting Cyber Security Pentesting redteam redteaming

redactproxy, a tool that lets pentesters use AI without leaking client data

AI coding agents are now part of a lot of security work. They are good at the parts a tester has no time for: going through every request, every parameter and every file rather than a sample of each. But none of that work happens on your machine. Everything the agent reads is sent to a model running on someone else's servers. So are you sending your client's data to an AI provider?

Where the client's data goes

The moment you point one of these agents at a live engagement, everything it touches reaches a third party. Client domains. Internal hostnames. Credentials pulled out of a config file. Employee email addresses. The client's own name, in the folder path, in the ticket reference, in the commit message. A testing agreement authorises you to access the client's systems. It rarely says anything about transmitting their contents to a model provider, and the same gap shows up against PCI-DSS, HIPAA and SOC 2 data-handling clauses.

The usual advice lands in one of two places: run a local model, or don't paste client data. Both work. Both cost you the thing you wanted. A 7B model on a laptop is not the model that spots the subtle chain across three hosts, and an agent you feed carefully redacted scraps by hand is an agent you are babysitting instead of using.

There is a third option. Almost none of that data needs to be there in the first place: the model does not need the real hostname to reason about a finding on it. It needs a hostname that stays the same every time it sees it.

That is where redactproxy(https://github.com/CSPF-Founder/redactproxy) comes in.

Where the model runs

Before we get to what redactproxy does, we need to understand where the work actually happens. Claude Code is the part on your machine: a terminal tool that reads your files, runs your commands and collects the output. The model is not on your machine at all: it runs on an AI provider's servers, Anthropic for example. Claude Code does no reasoning of its own, so anything it needs an answer about, including the scan output and the config file it just read, is sent to those servers over the API.

RedactProxy

redactproxy is an open-source tool from the Cyber Security & Privacy Foundation. It sits between Claude Code and the provider, on your own machine, and rewrites that API traffic in both directions. On the way out it replaces real client values with stable fake ones. On the way back it puts the real ones in again, before Claude Code ever sees the response.




The provider only ever sees placeholders. Your tool calls still run against real infrastructure, because the substitution back happens before the response reaches the agent. When the model writes a Bash command against a placeholder hostname, Claude Code receives the real hostname and runs it against the real host. Not just the first time: on every response, ten turns later, for the life of the conversation.

The swap happens inside the traffic itself, so nothing about the way your team works changes. No telemetry, no sync, no backup: everything it stores stays on the machine you run it on.

What the model receives

Here is some scan and config-dump output, exactly as Claude Code would send it, next to what the model actually receives.



What changed, and what didn't:
  • The mail. subdomain survives, and the same organisation placeholder appears in both the hostname and the email address. The relationship between them is intact.
  • The host octet .19 survives. Only the /24 network changed, so hosts that were adjacent stay adjacent.
  • The AWS key still looks like an AWS key, so the model knows what kind of secret it found without seeing the secret.
  • The connection string collapses into one opaque placeholder, because the whole credential span is sensitive.
  • The nginx banner, the latency, the port, the Dell OUI comment: untouched. None of them identify the client.

Stable placeholders

An engagement is redactproxy's word for one client project. Inside one, the same real value always gets the same fake. The hostname that became tok5198ede8bdbb1ada.internal this morning is still that same fake tomorrow, and in every request in between. This is not a convenience. It is the reason the tool is usable at all.

The model can still work out that two hosts belong to the same organisation. It just never learns which organisation.
Because tok1a2b3c4d5e6f7890.com and mail.tok1a2b3c4d5e6f7890.com are consistently the same fake, the model can reason that a finding on one host relates to a finding on another, that an email address belongs to the same company as a web server, that the same credential turned up in two places. All the analytical work survives. The identity does not.

The mappings live in the engagement's own folder and survive restarts, so the placeholder the model saw yesterday is still the same one today. Each engagement is self-contained and shares nothing with the others.

Placeholder shapes

Redaction that destroys structure destroys usefulness. Where a value's shape carries something useful but not identifying, the shape is kept:


The ranges are not arbitrary. Every fake comes from a space that cannot collide with something real: an IPv4 block reserved for equipment testing rather than one of the private ranges internal engagements actually target, the 555 phone exchange reserved for fiction, MAC addresses that can never belong to a real manufacturer. Fake credentials carry the string FAKE in a position where a real key can only hold a digit or a letter A to F, so no vendor could ever issue one.

What it detects

Detection is regex plus a validation step. No model in the loop, no network call, no learning. The detector set covers, roughly:
  • Network identity: domains and hostnames (bare or inside URLs), IPv4, IPv6, MAC addresses.
  • People: email addresses, NANP and international phone numbers.
  • Credentials for 25+ vendors: AWS, GitHub, GitLab, Slack, Stripe, Razorpay, Google, npm, DigitalOcean, Cloudflare, Azure, Docker Hub, CircleCI, Terraform, Snyk, Vault, Twilio, SendGrid, OpenAI, Anthropic and more, plus JWTs, bearer tokens, connection strings, PEM private keys and password hashes.
  • Regional PII: Indian Aadhaar and PAN numbers.
  • AD artifacts: machine account names, GPP cpassword values.
  • Client identity: whatever you add by hand, which is the part that matters most. See below.
There is also an allowlist, split into categories you can toggle. Your own out-of-band testing services (burpcollaborator.net, interact.sh, webhook.site) are never the client's, and common CDN and public dev platform hostnames say nothing about who the client is. Every category can be switched off per engagement.

Only the parts of a request that carry content get scanned, and only the values that match get replaced. Everything else is left exactly as it was. MCP tool calls and results are scanned too, because an MCP server is local infrastructure producing exactly the client data this exists to keep in.

Fail closed

If the proxy cannot finish redacting a request, it returns an error instead of sending it on. A malformed request, a detector that errors, a store that cannot save a new mapping: all of them fail the request rather than let it through.

An unredacted forward is the one outcome this project treats as worse than a broken request.

Setting up an engagement
go install github.com/CSPF-Founder/redactproxy/cmd/redactproxy@latest
Then, in a folder for the engagement. Name it after an engagement code, not the client; the Known gaps section explains why that matters more than it looks.
cd ~/engagements/eng-2026-014
redactproxy wizard --engagement eng-2026-014
The wizard asks four things. First, customer name variations: the legal name, the trading name, abbreviations, product names, internal codenames. No detector can match a name, so this is the only way they get redacted. They become case-insensitive substring matches, so XYZCorp also catches XYZCorporation.

Second, domains. Give the base domain only. Subdomains, email addresses at that domain and URLs all resolve from it automatically. Internal-only names work too: an AD forest, or any private scheme that will never appear on a public suffix list.

Third, which API this engagement talks to. Real Claude by default. redactproxy never asks for an Anthropic credential; it forwards Claude Code's own authentication untouched. Anything else speaking the Anthropic Messages API works too, z.ai for example, and each engagement remembers its own choice, so two client projects can run against different providers side by side.

Fourth, it offers two conveniences for the folder: a CLAUDE.md note explaining the placeholder shapes, and a .claude/settings.local.json that points Claude Code at the proxy and closes several channels that bypass the proxy entirely.

Then:
redactproxy      # terminal 1
claude           # terminal 2, already pointed at the proxy
That terminal running the proxy is also a live console. Type show to see every mapping minted so far, remove <value> to drop a bad one, or rules block "XyzExample" to add a value mid-session without interrupting anything.

Two files the wizard writes

The CLAUDE.md note is not decoration. Without it, a session sees strange placeholder values with no explanation: it corrects them as typos, hesitates to use them in tool calls, or retypes them from memory slightly wrong. The note tells the model these are stable identifiers to copy verbatim. It also covers a trap worth knowing yourself: if the model decodes base64 inline, the decoded content lands in its own output completely unprotected, because the encoded form passed through unredacted. Decoding to a file with Bash and reading the file back gives that content a normal pass through redaction.

The settings hardening closes paths that never touch ANTHROPIC_BASE_URL at all. The Artifact tool is a confirmed leak path: a report published through it goes straight to a hosted claude.ai URL, entirely unredacted, through a separate service call the proxy never sees. The wizard removes it from the session entirely rather than prompting for it each time, because a permission prompt can be approved out of habit and a tool that was never offered cannot leak anything. It also turns off WebFetch's safety check, which sends the target hostname to Anthropic before the fetch, whichever provider the engagement uses. A domain being reconned is exactly the value this tool exists to keep off side channels.

Known gaps

redactproxy ships a Known gaps page, and it is worth reading before you point this at real client data. The ones that matter most:
  • Your folder name is the big one, and redaction cannot fix it. Claude Code puts its working directory into the system field of every request, and that field is deliberately never scanned. A folder called xyz-example-bank-pentest sends "xyz example bank" to the model on every single request no matter what your rules say. The tool warns about it, at wizard time and at startup, but the only fix is to name engagement folders after an engagement code.
  • Encoded data passes straight through. A .env piped through base64, an xxd dump, Terraform state: none of it looks like anything to a regex. Decode locally first.
  • Names and prose are not detected. This is what rules block is for, and why it is the wizard's first question. A company name shows up in URL paths, ticket references, code comments and commit messages, and no detector can recognise it.

Who this is for

Pentest and consulting teams who want the productivity of an AI coding agent on a live engagement, and who would rather not explain to a client why their internal hostnames are in a third party's logs.

It was built for pentest work, and that shapes the vocabulary: engagements, clients, findings. Nothing in the redaction is specific to offensive work, though. Blue teams and infrastructure teams hit the same problem: a SOC analyst pasting alerts full of internal hostnames, a sysadmin debugging a manifest with credentials in it, anyone under an NDA who wants an AI agent working on real data. The detectors only care about the shapes they recognise, not why you are looking at them.

Read more »
Red Team
AI AI Pentesting GLM-5.3 LLM cybersecurity Pentesting Red Team

How We Find Critical Vulnerabilities with GLM 5.3 and Red Clippy

Over the last few months our red team exercises for BFSI customers have been run with an AI coding agent sitting in the loop. The findings that came out of them were the usual serious ones: broken authentication, unauthenticated access to sensitive data, an OTP bypass, SSRF, stored XSS, a login form that let us straight in with the password field left empty, a customer search that handed back the entire database when given a wildcard, and on one engagement a payment gateway secret key shipped inside a JavaScript bundle that every visitor's browser downloads.

None of that is exotic. Testers have been finding these things for twenty years. What changed for us was how the work got done, and more importantly, how it got kept.

Give a coding agent a shell and it turns into a fast, tireless tester. It runs the same tools you do. It will read a four megabyte minified bundle line by line without complaining, which is a thing no human on the team volunteers for. It will enumerate an API surface while you are still reading the scope document.

The trouble starts about forty minutes in. The context window fills up. The session compacts, or it ends and you start a fresh one the next morning, and the engagement goes with it. The new session re-scans hosts it already cleared. It re-tests things it already ruled out. Ask it which parts of the scope have been covered and it cannot tell you, because it does not know. And somewhere in a transcript nobody kept there is a confirmed injection that never made it into the report.

That is the problem Red Clippy(https://github.com/CSPF-Founder/red-clippy) exists to solve.

An engagement overview. All the screenshots here come from the project's demo database, not from a customer engagement.

It is not an AI pentesting framework

Red Clippy has no scanning engine of its own, no autonomous attack logic, and no opinion about what should be tested next. It will not find a vulnerability for you.

What it does is keep the record of an engagement while an agent does the testing and you direct it. Targets, scope, what has already been tested, findings, evidence. That is the whole job.

It is built for testers who already know what they are doing and want to use Claude Code, Codex CLI, or any other MCP-compatible client alongside their normal workflow. You define the target and scope in the panel, or paste the customer's scope list into the chat and let the agent enter it. From there you guide the agent however you like, the same way you would guide a junior on the team, and it writes down what it did as it goes.

That turns out to be useful for four things: knowing what has already been tested, checking the same finding across multiple domains and assets, keeping engagement history for periodic retesting, and not having to rely on the model remembering everything or on a folder of text files pretending to be a database.

The setup

Three pieces, all on one machine. GLM 5.3 from z.ai does the reasoning. Claude Code is the client, providing the shell, the file access and the agent loop. Red Clippy holds the record and connects to Claude Code over MCP.

Because it is a client rather than a model, and z.ai serves an Anthropic-compatible endpoint, you can point one at the other and keep the agent harness you already know. The setup is documented on the project page, so we will not repeat it here.

MCP runs client-side, so Red Clippy does not know which model is behind the agent and the tools behave the same either way. That means the discipline of the engagement is not tied to a model you happen to be using this quarter. If we move off GLM next year, the record, the coverage and the findings all survive the move.

The rules arrive before the first tool call

This is the part most people skip when they wire an agent into a workflow, and it is the one that changed our output the most.

An agent that has to ask for the rules of engagement generally will not bother. So Red Clippy hands over a Red Team Instructions document during the MCP handshake, before the agent makes its first tool call. It is one document, not a system prompt maintained in five places, and the most specific one wins: a per-engagement override if there is one, otherwise the organization default, otherwise the built-in.

The Red Team Instructions document, served to every agent on connect and overridable per organization and per engagement.

Most of it is unglamorous. The line that matters most on BFSI work is the one about taking the minimum access needed to show impact. An agent that proves an unauthenticated data exposure by retrieving three records and stopping has given you a finding. An agent that helpfully retrieves the whole table has given you a very different conversation with the customer.

The rest is tradecraft, and that is where several of our critical findings actually came from: read the main bundle rather than grepping it, trigger errors deliberately and read the whole response, strip the auth header and retry, then change the identifiers and see whose data comes back.

None of that is new methodology. It is what a competent tester does anyway. The difference is that it is in the agent's context on every connect, without anyone remembering to paste it in.

Setting up an engagement

You create the pentest, paste in the scope from the engagement letter, mark the in-scope domains and ranges, and mark the exclusions. You can type them yourself or let the agent enter them from the customer's list. Either way you read them before anything gets touched.

Scope units are assets, each with its own checklist, reachability marking and in or out of scope flag.

After that you drive, and the instructions are duller than people expect. "Do the initial recon first, subdomain enumeration across the in-scope domains." "Now go through the asset list, pick up whatever is still untested, and mark the checks off as you clear them." The agent runs its own tools from its own shell, the way it would anyway, and posts the results back as it works. Raw scanner output goes in with a single call and gets parsed automatically, whether it came from nmap, Burp, Nessus, OpenVAS, masscan, naabu or subfinder. The things you are actually testing become assets. Everything else stays an observation attached to an asset. Findings go in with severity, a CVSS vector, a proof of concept and the evidence that backs it.

There are 82 MCP tools, which is nearly everything the panel itself can do. That matters more than it sounds, because a tool set that only covers half the application forces you back into the browser mid-session to finish what the agent started. Anything the agent writes you can write yourself, and anything you write it can read. You can run the engagement entirely by hand, entirely through the agent, or switch between the two in the middle of a session.

What a record does that a transcript cannot

The interesting part is not that the agent is fast, although it is. It is that the work survives the session it was done in.

Recon noise stays out of the scope list, which is why it survives

Every content discovery run produces hundreds of paths. Every bundle you read produces endpoints, internal hostnames, technology fingerprints and, now and then, a secret. Throw all of that into an asset list and the asset list is useless by lunchtime.

Red Clippy separates the two. The things you are testing are assets. Everything else is an observation hanging off the asset it came from, with a kind and the tool that found it.


A few observations are findings in their own right, like a key that should never have been public. Most are leads, and the leads are what pay off later. On our engagements, unauthenticated access to sensitive data came from an API path pulled out of a bundle, called with no auth header, that returned data.

In a transcript that path scrolls away. As an observation it is still there tomorrow, attached to the right host, with the tool that found it recorded alongside.

Correlating things that happened days apart

The findings that matter are rarely one observation. They are usually two, made hours or days apart, that mean something together.

Reading the front-end bundle early in an engagement turns up internal hostnames. They go into the record and testing moves on. Days later, a parameter that fetches a remote image turns out to make outbound requests.

An SSRF is only worth what you can reach with it, and the hostnames from the first day are what you point it at. Making that connection requires the first day's record to still be there, and searchable, when you need it days later. That is exactly what a context window does not give you.

Red Clippy makes those joins explicit. Every finding carries tags for the asset it affects and the check it came from, so it files itself under both. The attack graph lets you link any two things, an observation, an asset or a finding, with a label of your own, then follow the links out from one or trace the route between two. The chain from "hostname found in the bundle" to "reachable through SSRF" to "admin interface behind it" is saved, rather than something to piece back together when the report is written.

Correlating across engagements

A customer is rarely one engagement. There is this quarter's, last quarter's, and the retest after that.

Because those engagements share a record, any host or IP can be asked about across all of them at once. Have we tested this before? What did we find? Was it reachable last time?

That pays off twice over. A critical bug in one API is a question about every other API the customer has: we confirmed one in a session, and a later session testing a different domain found the same bug there, because the first finding was something to check the new asset against rather than a paragraph in a transcript nobody reopened. And a host that was blocked last quarter but answers this one has almost never changed. It is a source address or a VPN, and knowing that saves an hour of chasing a WAF that is not there.

The same goes for paths. Whatever was recorded for a host in an earlier engagement can be pulled into the current one, so you get last round's content discovery for free and can see at once whether what you reported then is still live.

Coverage you can query instead of remember

There are 135 built-in checks mapped to OWASP WSTG, plus recon, network, cloud and OSINT checks, tracked per asset.


This is the best defence we have found against the way agent-driven testing actually goes wrong. It is not hallucination. It is skimming. An agent that stumbles onto an interesting SQL injection in the first twenty minutes will happily spend the rest of the session on it and then report a thoroughly successful engagement.

Asking what is left on an asset gives you the current state of every check, so "what have I not looked at on this host" becomes a question with an answer. Marking a check as not applicable counts as resolved, and that matters: "we looked, there is no file upload here" is a genuine testing outcome and belongs in the record rather than sitting in the untested pile forever.

The auth, authz and session categories are where OTP bypass and broken authentication live, and they are exactly the checks an excited agent skips on its way to something noisier. The blank password came out of exactly that part of the list: a login check nobody would call interesting, and a form that issued a valid session when the password field was submitted empty. Nothing would have gone back to that check if the record had not been sitting there saying it was untested. A count of resolved checks against the total is an honest statement about where an engagement stands. "I tested the application thoroughly" is not.

Findings that hold up to review

A finding has to stand on its own, because whoever reviews it will not have the tester sitting next to them explaining what they meant.



A finding carries severity and status, a CVSS vector, CWE and CVE, and separate fields for details, impact, proof of concept and remediation, because those are what a report needs and what a reviewer checks.


The proof of concept field is the one that does the work. It either contains steps that reproduce or it does not, and a reviewer can tell which without asking anyone. Evidence attaches to the finding itself rather than living in a folder someone has to match up later.


The rules for writing a finding sit inside the tool the agent calls to file one, so it reads them as it writes rather than somewhere far back in the session. They tell it to keep each field to a paragraph, keep hostnames out of the title, and say what needs to change in the remediation instead of pasting config and version numbers that may be wrong for the customer's stack.

What the human still does

The agent is fast and it is sometimes wrong, and the workflow assumes both. Everything it writes is an ordinary row in the browser that you can edit, reclassify or delete, and it picks up your corrections the next time it reads.

Three habits do the work. We read the findings themselves rather than the agent's account of them, because the finding rows are what the customer actually gets. We check the coverage before believing any of it, because an agent can come back with six findings having cleared nine checks out of 135. Findings are not coverage. And we set the severity ourselves, because whether something is a finding at all, and how bad it is, is a call a human should make.

The thing that does not change is responsibility. Scope marking and the Red Team Instructions are guardrails, not authorisation. Red team exercises run under a signed engagement letter, and the agent acts entirely on your authority. Everything it does is yours.

What the model does and what the record does

GLM 5.3 does the reasoning. Reading a minified bundle and noticing that a string is a live key. Stripping the auth header off a request and noticing the data still comes back. Putting a single wildcard into a customer search field, then reading the response closely enough to work out that it had returned every customer in the database rather than an error. Going back at an OTP flow after the obvious attempt failed. That is a model capability question, and a better model gives you better testing.

Red Clippy is what makes that add up to an engagement. It contributes memory, correlation, coverage and evidence discipline, and it contributes them identically regardless of what is driving the agent. The two together are why a session ending no longer takes the engagement with it.

Red Clippy was originally our own internal tool, built for our engagements. We have now put it out publicly, because we think other testers will get the same use out of it. It is open source, from the Cyber Security and Privacy Foundation. Source is at github.com/CSPF-Founder/red-clippy and the documentation, including how to set all of this up, is at cspf-founder.github.io/red-clippy. Bug reports and any other contributions are welcome.
Read more »
Subscribe to: Posts (Atom)