Gating Red Teamed AI Agents with Governance Gates

Overview

One agent is easy to keep secure — you red team it, read the report, and decide. Hundreds of agents is a different problem. There are too many for any security team to review one by one, they ship on their own schedules, and every team red teams a little differently, so "is this agent safe to deploy?" stops having a reliable answer.

This guide squeezes every agent through one standardized deployment gate. You define the red teaming requirement once as a governance control, every agent's project inherits it, and each pipeline calls the same gate before it deploys. Nobody reviews anything by hand, and a passing gate means the same thing for agent #1 and agent #400.

The same pattern works for evaluation evidence with pre-deployment eval controls — gating on a qualifying test run instead. This guide covers the security half: pre-deployment red teaming controls, which gate on a qualifying risk assessment.

In this guide, you will:

• Red team every release candidate automatically, in CI or on a schedule.
• Decide which assessment gates a release — the latest one, or the latest official one if you promote assessments.
• Define the requirement once as a pre-deployment red teaming control on a policy every agent project inherits.
• Run the same gate in every pipeline, so a missing, stale, or failing assessment blocks that agent's deploy.

flowchart LR
    subgraph Fleet["Hundreds of agents"]
        A1["Agent 1"]
        A2["Agent 2"]
        AN["Agent N"]
    end

    Fleet --> RT["Red teaming<br/>(DeepTeam or platform)"]
    RT --> Gate["Standardized gate<br/>deepeval gate"]
    Policy["Base policy<br/>pre-deployment red teaming control"] --> Gate
    Gate --> Deploy["Deploy allowed"]

    classDef fleet fill:#f8fafc,stroke:#334155,stroke-width:2px,color:#0f172a
    classDef step fill:#eef2ff,stroke:#4f46e5,stroke-width:1px,color:#1e293b
    class A1,A2,AN fleet
    class RT,Policy,Gate,Deploy step

What This Looks Like in Practice

A standardized gate is only useful if the requirement behind it is specific. These are the shapes it usually takes across a large fleet:

One framework, one bar. Every agent is red teamed against the same framework — pulled from Confident AI, so it's literally the same configuration of vulnerabilities and attacks — and the control requires the gating assessment's pass rate to clear a fixed threshold, like 90%. Because the framework is shared, that percentage means the same thing everywhere: agent #12 clearing 90% was hit by the same class of attacks as agent #300. Without a shared framework, each team's pass rate is measured against a different test and the number stops being comparable.

Every release re-tests. The control judges the latest risk assessment in the project, so red teaming runs in the same pipeline immediately before the gate. That ordering is what makes the evidence belong to the build being deployed, rather than to whatever someone tested three releases ago — and at scale that matters, because agents change far more often than anyone re-runs security testing by hand.

Only the approved configuration counts. Filters require the gating assessment to have run against the approved application, model, and attack configuration. This closes the obvious loophole: a team red teaming a stubbed endpoint, an older cheaper model, or a single weak attack shouldn't be able to satisfy the same gate as a full sweep against the real thing.

Different bars for different risk tiers. Customer-facing agents handling PII sit on a stricter policy with a higher pass-rate bar, while internal tooling extends the same base policy with the org-wide baseline only. Each project still passes through one gate — the tier just decides which controls it inherits.

Human sign-off where it's warranted. For the handful of high-risk agents, assessments are marked official, so the gate judges the one a security engineer promoted rather than the most recent run. The rest of the fleet stays fully automated on whatever the pipeline just produced.

Build It

1. Decide Where Assessments Come From

   The gate can only be as fresh as the assessments feeding it, so red teaming has to run on its own — not when someone remembers. There are two ways to produce the assessments, and fleets usually run both:

   In CI with DeepTeamOn a schedule in the platform

   Run code-driven red teaming against the release candidate as a pipeline step, using DeepTeam. You write the script once and every agent repository runs the same one.

   Best for agents that only exist inside your pipeline, or when you want custom vulnerabilities and attacks. The next step writes this script.

2. Write the Red Teaming Script

   This is the script every agent's pipeline runs. Install DeepTeam and point CONFIDENT_API_KEY at the agent's project, so the resulting risk assessment lands in the project the gate will assess:

   pip install -U deepteam
   export CONFIDENT_API_KEY="confident_us_proj_..."

   Now the part that standardizes the fleet: pull the framework from Confident AI instead of hardcoding one in each repository. Configure the security framework once in the platform, then have every agent's script pull that same framework by id. DeepTeam brings down every risk category with its configured vulnerability types and attack methods, so all 400 agents are attacked by the identical test suite — and when security adds a vulnerability to the framework, the whole fleet picks it up on the next run without anyone touching a pipeline.

   from deepteam import red_team
   from deepteam.frameworks import RedTeamingFramework
   from deepteam.test_case import RTTurn, ToolCall
   
   from my_app import my_agent
   
   framework = RedTeamingFramework()
   framework.pull("your-framework-id")
   
   async def model_callback(input: str) -> RTTurn:
       # Point this at the agent build you're about to ship
       response = await my_agent(input)
       return RTTurn(
           role="assistant",
           content=response.output,
           retrieval_context=response.retrieved_docs,
           tools_called=[ToolCall(name=t) for t in response.tools_used],
       )
   
   red_team(
       model_callback=model_callback,
       framework=framework,
       identifier="release-candidate",
       run_all_attacks=True,
   )

   Three things to get right here, all covered in Red Team Using DeepTeam:

   • The framework id is the last segment of the URL on the framework's configuration page in the platform.
   • The callback takes the adversarial input as a single string and returns an RTTurn with role="assistant". Pass retrieval_context and tools_called when the agent is a RAG or agentic system, so attacks are judged against what the agent actually retrieved and called rather than its final text alone.
   • The identifier names the assessment in the risk profile. Use a stable one per pipeline — "release-candidate" here — so it's obvious at a glance which assessments came from the gated pipeline and which were somebody experimenting.

3. Decide Which Assessment Gates the Release

   A project accumulates assessments — scratch runs, one-off experiments, scheduled sweeps. The control assesses the project's latest completed risk assessment, so by default the last thing anyone ran is the evidence a release is gated on.

   If that's too loose, mark assessments as official: the control then assesses the latest official assessment instead, and a scratch run can't quietly become the basis for a deploy decision. Marking is done from the risk profile page.

4. Define the Requirement Once

   This is the step that makes the gate standardized: the security bar is written once, in one place, by the people who own it — not copied into hundreds of pipelines where each copy drifts.

   Create the policy that every agent must clear, and add the control to it:

   1. Navigate to your organization's Governance page and open (or create) the policy.
   2. Add a pre-deployment red teaming control.
   3. Point it at the assessments you settled on in the previous step — the latest, or the latest official.
   4. Add filters so the assessment must clear your pass-rate bar and match the application, model, and attack configuration you actually approved.
   5. Set Importance to Critical or High, then save.

   Filters are what stop a technically-passing gate from being meaningless. Without them, an assessment that hit a stubbed endpoint with a single weak attack satisfies the control just as well as a full OWASP sweep against production.

   Add a pre-deployment eval control to the same policy so one gate covers both quality and security, and runtime controls to catch regressions after the deploy.

5. Enroll Every Agent's Project

   A policy has no effect on a project until the project is assigned to it, and the gate errors out when a project belongs to no policy. At fleet scale, assigning by hand is exactly the manual step you're trying to delete.

   If you already provision a project per agent — see Provision Projects for Agents on the Fly — enroll each one into the policy in the same provisioning code with Assign Projects to Governance Policies on the Fly. Assignment is safe to re-run on every pipeline execution, so an agent is governed from its first deploy and nobody has to remember to add it.

   Each project belongs to at most one policy, so the policy you assign must represent the complete set of requirements that agent has to satisfy — which is why the shared bar belongs on a base policy the team policies extend.

6. Run the Same Gate in Every Pipeline

   Every agent's pipeline runs the identical two lines after red teaming, using that project's Project API Key. There is nothing agent-specific to configure — the requirements come from the policy, so the pipeline snippet is copy-paste across all of them:

   PythonTypeScript

   export CONFIDENT_API_KEY="confident_us_proj_..."
   deepeval gate

   deepeval gate assesses every control in the project's policy — including controls inherited from a base policy — and exits 0 only when the whole policy passes. Any other exit code stops the deployment.

   Put together, this is the workflow you standardize on — the one every agent repository gets. It red teams the release candidate, then lets the governance gate make the deployment decision:

   name: Red team and gate
   
   on:
     pull_request:
     push:
       branches:
         - main
   
   jobs:
     red-team-and-gate:
       runs-on: ubuntu-latest
       env:
         CONFIDENT_API_KEY: ${{ secrets.CONFIDENT_API_KEY }}
         OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
   
       steps:
         - name: Check out repository
           uses: actions/checkout@v4
   
         - name: Set up Python
           uses: actions/setup-python@v5
           with:
             python-version: "3.11"
   
         - name: Install DeepTeam and DeepEval
           run: pip install -U deepteam deepeval
   
         - name: Red team the release candidate
           continue-on-error: true
           run: python tests/red_team.py
   
         - name: Run governance deployment gate
           run: deepeval gate
   
         - name: Deploy
           run: ./scripts/deploy.sh

   Two details carry the whole design:

   • The red teaming step uses continue-on-error. A failing assessment shouldn't abort the job before the gate runs — you want the gate to make the call, not a raw exit code, because the gate is the thing that knows your organization's thresholds and importance levels.
   • Steps run sequentially, which is what makes the control's "latest assessment" the right one. GitHub Actions finishes the red teaming step before starting the gate, so by the time deepeval gate runs, the newest risk assessment in the project is the one this job just uploaded.

   Done! No agent in your fleet can ship without current, correctly configured red teaming evidence, and a passing gate means the same thing for every one of them.

Gate on the Policy, Not the Red Team Exit Code

It's tempting to block deploys directly on the red teaming script's result. Failing the build on that alone gives you a much weaker gate:

• A crashed or skipped assessment looks like a pass. The control resolves to NO_DATA and fails; a script that never uploaded results exits however it likes.
• Coasting on old evidence is visible. Every gate run records which assessment it judged, so an agent passing on an assessment nobody re-ran shows up in governance history. A pipeline that skipped red teaming this time simply says nothing.
• A weakened configuration looks like a real test. Filters require the approved application, model, and attack configuration. An exit code can't tell a full OWASP sweep from one toothless attack.
• The requirement lives in one place. Security owns the policy in Confident AI, and every governed project inherits the same bar. Tightening the standard for hundreds of agents is one edit on a base policy instead of hundreds of pull requests against pipelines you don't own.
• The bar can't drift per team. When each pipeline encodes its own thresholds, "the gate passed" means something slightly different in every repository — which is precisely what breaks down at a hundred agents.

Next Steps

Last updated on August 13, 2026