Gating Red Teamed AI Agents with Governance Gates

Overview

One agent is easy to keep secure — you red team it, read the report, and decide. Hundreds of agents is a different problem. There are too many for any security team to review one by one, they ship on their own schedules, and every team red teams a little differently, so "is this agent safe to deploy?" stops having a reliable answer.

This guide squeezes every agent through one standardized deployment gate. You define the red teaming requirement once as a governance control, every agent's project inherits it, and each pipeline calls the same gate before it deploys. Nobody reviews anything by hand, and a passing gate means the same thing for agent #1 and agent #400.

The same pattern works for evaluation evidence with pre-deployment eval controls — gating on a qualifying test run instead. This guide covers the security half: pre-deployment red teaming controls, which gate on a qualifying risk assessment.

In this guide, you will:

• Red team every release candidate automatically, in CI or on a schedule.
• Decide which assessment gates a release — the latest official one, or the latest one matching an identifier within a rolling window.
• Define the requirement once as a pre-deployment red teaming control on a policy every agent project inherits.
• Run the same gate in every pipeline, so a missing, stale, or failing assessment blocks that agent's deploy.

flowchart LR
    subgraph Fleet["Hundreds of agents"]
        A1["Agent 1"]
        A2["Agent 2"]
        AN["Agent N"]
    end

    Fleet --> RT["Red teaming<br/>(DeepTeam or platform)"]
    RT --> Gate["Standardized gate<br/>deepeval gate"]
    Policy["Base policy<br/>pre-deployment red teaming control"] --> Gate
    Gate --> Deploy["Deploy allowed"]

    classDef fleet fill:#f8fafc,stroke:#334155,stroke-width:2px,color:#0f172a
    classDef step fill:#eef2ff,stroke:#4f46e5,stroke-width:1px,color:#1e293b
    class A1,A2,AN fleet
    class RT,Policy,Gate,Deploy step

What This Looks Like in Practice

A standardized gate is only useful if the requirement behind it is specific. These are the shapes it usually takes across a large fleet:

One framework, one bar. Every agent is red teamed against the same framework — say OWASP Top 10 for LLMs — and the control requires the gating assessment's pass rate to clear a fixed threshold, like 90%. Because the framework is shared, that percentage means the same thing everywhere: agent #12 clearing 90% was hit by the same class of attacks as agent #300. Without a shared framework, each team's pass rate is measured against a different test and the number stops being comparable.

Evidence has to be from this release. The control matches an identifier and a rolling window, so an assessment from three releases ago no longer qualifies. An agent whose prompt or model changed since it was last tested is blocked until it's re-tested — which is the failure mode you actually care about at scale, since agents change far more often than anyone re-runs security testing.

Only the approved configuration counts. Filters require the gating assessment to have run against the approved application, model, and attack configuration. This closes the obvious loophole: a team red teaming a stubbed endpoint, an older cheaper model, or a single weak attack shouldn't be able to satisfy the same gate as a full sweep against the real thing.

Different bars for different risk tiers. Customer-facing agents handling PII sit on a stricter policy with a higher threshold and a 7-day window, while internal tooling extends the same base policy with the org-wide baseline only. Each project still passes through one gate — the tier just decides which controls it inherits.

Human sign-off where it's warranted. For the handful of high-risk agents, the control gates on the latest official assessment, so a security engineer has to promote an assessment before a release can proceed. The rest of the fleet stays fully automated on identifier and window.

Build It

1. Red Team Every Release Candidate

   The gate can only be as fresh as the assessments feeding it, so the assessment needs to run on its own — not when someone remembers. Pick whichever fits your agent:

   In CI with DeepTeamOn a schedule in the platform

   Run code-driven red teaming against the release candidate as a pipeline step. Results upload to the project's risk profile as long as CONFIDENT_API_KEY is set.

   from deepteam import red_team
   from deepteam.frameworks import OWASPTop10
   from deepteam.test_case import RTTurn
   
   async def model_callback(input: str) -> RTTurn:
       # Point this at the agent build you're about to ship
       return RTTurn(role="assistant", content=my_agent(input))
   
   red_team(
       model_callback=model_callback,
       framework=OWASPTop10(),
       attacks_per_vulnerability_type=3,
   )

   Best for agents that only exist inside your pipeline, or when you want custom vulnerabilities and attacks.

2. Decide Which Assessment Gates the Release

   A project accumulates assessments — scratch runs, one-off experiments, scheduled sweeps. The control needs to know which one counts. There are two selection methods, and this choice is the one that determines how much the gate is worth:

   Method	The control selects	Use it when
   Latest official assessment	The most recent assessment marked official in the project	A security or governance team explicitly promotes one assessment as the release's source of truth
   Identifier and window	The latest completed assessment matching an identifier, completed within the last 7, 14, 30, or 90 days	A stable identifier represents a release pipeline or red teaming campaign, and the evidence must stay current

3. Define the Requirement Once

   This is the step that makes the gate standardized: the security bar is written once, in one place, by the people who own it — not copied into hundreds of pipelines where each copy drifts.

   Create the policy that every agent must clear, and add the control to it:

   1. Navigate to your organization's Governance page and open (or create) the policy.
   2. Add a pre-deployment red teaming control.
   3. Configure the selection method you chose in the previous step.
   4. Add filters so the assessment must clear your pass-rate bar and match the application, model, and attack configuration you actually approved.
   5. Set Importance to Critical or High, then save.

   Filters are what stop a technically-passing gate from being meaningless. Without them, an assessment that hit a stubbed endpoint with a single weak attack satisfies the control just as well as a full OWASP sweep against production.

   Add a pre-deployment eval control to the same policy so one gate covers both quality and security, and runtime controls to catch regressions after the deploy.

4. Enroll Every Agent's Project

   A policy has no effect on a project until the project is assigned to it, and the gate errors out when a project belongs to no policy. At fleet scale, assigning by hand is exactly the manual step you're trying to delete.

   If you already provision a project per agent — see Provision Projects for Agents on the Fly — enroll each one into the policy in the same provisioning code with Assign Projects to Governance Policies on the Fly. Assignment is safe to re-run on every pipeline execution, so an agent is governed from its first deploy and nobody has to remember to add it.

   Each project belongs to at most one policy, so the policy you assign must represent the complete set of requirements that agent has to satisfy — which is why the shared bar belongs on a base policy the team policies extend.

5. Run the Same Gate in Every Pipeline

   Every agent's pipeline runs the identical two lines after red teaming, using that project's Project API Key. There is nothing agent-specific to configure — the requirements come from the policy, so the pipeline snippet is copy-paste across all of them:

   PythonTypeScript

   export CONFIDENT_API_KEY="confident_us_proj_..."
   deepeval gate

   deepeval gate assesses every control in the project's policy — including controls inherited from a base policy — and exits 0 only when the whole policy passes. Any other exit code stops the deployment.

   Put together, this is the workflow you standardize on — the one every agent repository gets. It red teams the release candidate, then lets the governance gate make the deployment decision:

   name: Red team and gate
   
   on:
     pull_request:
     push:
       branches:
         - main
   
   jobs:
     red-team-and-gate:
       runs-on: ubuntu-latest
       env:
         CONFIDENT_API_KEY: ${{ secrets.CONFIDENT_API_KEY }}
         OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
   
       steps:
         - name: Check out repository
           uses: actions/checkout@v4
   
         - name: Set up Python
           uses: actions/setup-python@v5
           with:
             python-version: "3.11"
   
         - name: Install DeepTeam and DeepEval
           run: pip install -U deepteam deepeval
   
         - name: Red team the release candidate
           continue-on-error: true
           run: python tests/red_team.py
   
         - name: Run governance deployment gate
           run: deepeval gate
   
         - name: Deploy
           run: ./scripts/deploy.sh

   Two details carry the whole design:

   • The red teaming step uses continue-on-error. A failing assessment shouldn't abort the job before the gate runs — you want the gate to make the call, not a raw exit code, because the gate is the thing that knows your organization's thresholds and importance levels.
   • Steps run sequentially. The assessment is uploaded before the gate assesses the control, so the gate reads results from the build it is gating.

   Done! No agent in your fleet can ship without current, correctly configured red teaming evidence, and a passing gate means the same thing for every one of them.

Gate on the Policy, Not the Red Team Exit Code

It's tempting to block deploys directly on the red teaming script's result. Failing the build on that alone gives you a much weaker gate:

• A crashed or skipped assessment looks like a pass. The control resolves to NO_DATA and fails; a script that never uploaded results exits however it likes.
• Stale evidence looks like fresh evidence. The rolling window rejects an assessment from three releases ago. A script that didn't run this time says nothing at all.
• A weakened configuration looks like a real test. Filters require the approved application, model, and attack configuration. An exit code can't tell a full OWASP sweep from one toothless attack.
• The requirement lives in one place. Security owns the policy in Confident AI, and every governed project inherits the same bar. Tightening the standard for hundreds of agents is one edit on a base policy instead of hundreds of pull requests against pipelines you don't own.
• The bar can't drift per team. When each pipeline encodes its own thresholds, "the gate passed" means something slightly different in every repository — which is precisely what breaks down at a hundred agents.

Next Steps

Last updated on August 12, 2026