Launch Week 02 wrapped — explore all five launches

Gate Deployments in CI/CD

Block deployments that don't meet an assigned governance policy.

Included on the Enterprise plan. Book a demo, opens in a new tab. Not included on the Team plan. Not included on the Starter plan. Not included on the Free plan.

Use a governance policy as a deployment gate to prevent a project from shipping when it doesn't meet your organization's requirements.

The gate assesses every control in the project's assigned policy, including controls inherited from its base policy. It passes when every control above Low importance passes.

Before you begin

Assign the project to a governance policy before running the gate. The assess endpoint returns an error when the project doesn't belong to a policy.

Run the deployment gate

Run the gate from your CI/CD pipeline using the deepeval CLI, available in both Python and TypeScript, or call the public API directly.

deepeval gate

The CLI exits with code 0 when the policy passes and a non-zero code when it fails, allowing a failing policy to stop the pipeline.

All three methods call the POST /v1/governance/assess endpoint using the project's API key. The response indicates whether the policy passed, identifies the evaluated policy, and reports the status of every control:

{
  "success": true,
  "data": {
    "passed": false,
    "governancePolicy": {
      "id": "GOVERNANCE-POLICY-ID",
      "name": "EU AI Act"
    },
    "governanceControls": [
      {
        "id": "GOVERNANCE-CONTROL-ID",
        "name": "No user data vulnerabilities",
        "type": "PRE_DEPLOYMENT_RED_TEAMING",
        "severity": "HIGH",
        "status": "FAIL"
      },
      {
        "id": "GOVERNANCE-CONTROL-ID",
        "name": "Nightly evals pass rate",
        "type": "PRE_DEPLOYMENT_EVALS",
        "severity": "LOW",
        "status": "NO_DATA"
      }
    ]
  },
  "deprecated": false
}

A control sets passed to false when its status is anything other than PASS and its severity is anything other than LOW. In the example above, the red teaming control fails the gate while the Low-importance control is reported and ignored.

Example: evaluate, red team, and gate a project

The following GitHub Actions workflow installs DeepEval and DeepTeam, runs code-based evaluations, runs code-based red teaming, and then evaluates the project's deployment gate:

governance-gate.yml
name: Governance deployment gate

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  governance:
    runs-on: ubuntu-latest
    env:
      CONFIDENT_API_KEY: ${{ secrets.CONFIDENT_API_KEY }}
      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.11"

      - name: Install DeepEval and DeepTeam
        run: pip install -U deepeval deepteam

      - name: Run code-based evaluations
        continue-on-error: true
        # test_llm_app.py uses DeepEval to define and run evaluation tests.
        run: deepeval test run tests/test_llm_app.py

      - name: Run code-based red teaming
        continue-on-error: true
        # red_team.py uses DeepTeam to define and run a risk assessment.
        run: python tests/red_team.py

      - name: Run governance deployment gate
        run: deepeval gate

Replace tests/test_llm_app.py with your DeepEval test file and tests/red_team.py with the Python file that calls DeepTeam's red_team() function.

GitHub Actions runs steps within a job sequentially, so the deployment gate runs after the evaluation and red teaming results have been uploaded. The validation steps use continue-on-error to guarantee that the final gate still runs when either assessment reports a failure; the gate then makes the final deployment decision from the policy's controls.

Next steps

Scaling beyond prototype?For teams evaluating Confident AI in productionTalk to us

Last updated on

Built byConfident AI