The challenge
AI agents now refund customers, change databases, ship code and spend money without a person in the loop. They act in seconds. Monitoring tools show what happened after the fact, which is too late when the action was a destructive query against production. Public incidents make the point: in 2025, as reported by Jason Lemkin, an AI agent deleted SaaStr’s database during a declared code freeze. An instruction in a prompt is not a control.
I see the same failures every week in my AI evaluation and red-teaming work for platforms such as Mercor, Turing and DataAnnotation. Agents claim a task is finished when it is not. They reach for a destructive tool when a safer one would do. They loop, and every loop costs tokens, while cloud billing data often arrives a day later. Teams running agents in production need the checks I apply in evaluation on live systems, before the action runs.
- Stop a risky action before it runs, not report it afterwards
- Put a named human in the loop for the actions that matter, from the web or a phone
- Cap LLM spend per agent, per team or for the whole company
- Check an agent’s “done” claim against the real system
- Keep a record of every decision that shows if anyone changed it
What I built
I designed Brakeproof as a checkpoint in the path of every tool call, not a dashboard beside it. That is the only place a control can pause or block an action before it runs. Each policy decides whether an action is allowed, paused for approval or blocked. A paused action gets a snapshot first, so a bad change can be undone, and an approval brief explains the risk in plain English. A rule such as DROP TABLE is blocked outright.
Failure behaviour was a design decision from the start. If Brakeproof is unreachable, each policy chooses what happens, and the default is to fail closed: risky calls are blocked, and a call that needed approval is never forwarded without one.
Adoption had to be cheap, so the agent keeps its own tools and prompts. MCP servers are wrapped by the Brakeproof MCP proxy in the client config. Custom agents use the Python or TypeScript SDK or a framework hook for the OpenAI Agents SDK, LangChain / LangGraph or the Claude Agent SDK. Claude Code connects through a PreToolUse hook.
The rest of the product follows from the failures I see in evaluation work. The spend brake pauses an agent the moment it goes over its LLM budget. Proof of work uses read-only connectors to check Stripe, GitHub or Postgres and flags false success claims. The ledger is a SHA-256 hash chain with signed checkpoints, exportable as CSV or JSON. Red-team tests send your own LLM app or agent the attacks real attackers use, including prompt injection, jailbreaks and data leaks, with scans mapped to the OWASP Top 10 for LLM apps, the OWASP Top 10 for agentic apps, NIST AI RMF and MITRE ATLAS. Every run has a budget cap and starts with a consent check.
Brakeproof AI writes approval briefs and incident reports with the customer’s own Gemini, Claude or OpenAI key, and only redacted data is sent. It advises and the rules decide: it never approves anything.
Architecture and stack
- Backend
- Python · FastAPI
- Web app
- Next.js · React · TypeScript
- Mobile
- React Native approvals app (iOS and Android, beta)
- Agent integration
- MCP proxy · Python SDK · TypeScript SDK · Claude Code (PreToolUse hook) · OpenAI Agents SDK · LangGraph
- Proof-of-work checks
- Stripe · GitHub · Databases (Postgres)
- Red-team frameworks
- OWASP Top 10 for LLM apps · OWASP Top 10 for agentic apps · NIST AI RMF · MITRE ATLAS
- Ledger
- SHA-256 hash chain · Signed checkpoints · CSV and JSON export
- Deploy targets (beta)
- Docker · Hugging Face Spaces · AWS ECS Fargate
Outcome
Brakeproof is in private preview, with early access through a waitlist. The action firewall, spend brake, proof of work, tamper-evident ledger, Brakeproof AI, red-team tests and security sensors work today. Web approvals work today.
Two parts are built and in beta. The iOS and Android approvals app, with signed approvals and a kill switch, is in testing and not yet in the app stores. Deploy ships an agent to the customer’s own cloud on Docker, Hugging Face Spaces or AWS with the guard and kill switch built in, so it runs in their account, not on Brakeproof’s.
The controls I recommend to clients in AI red-teaming and agent work, such as human approval for destructive actions, budget caps, verified completion and an audit trail, are the ones Brakeproof puts in the path of every agent action.
- Available now: action firewall with allow, pause or block per policy, and a snapshot before risky calls
- Available now: spend brake with daily or monthly LLM budgets and automatic pause at the limit
- Available now: proof-of-work checks against Stripe, GitHub and Postgres
- Available now: tamper-evident ledger and red-team scans mapped to OWASP, NIST AI RMF and MITRE ATLAS
- Beta: React Native approvals app with kill switch, and deploy to your own cloud
Last updated