Red Team Using DeepTeam

Overview

Confident AI's red teaming capabilities offer a variety of features to test AI safety and security in development for a pre-deployment workflow, offering a wide range of features for:

• Vulnerability assessment: Systematically identify weaknesses like bias, toxicity, PII leakage, and prompt injection vulnerabilities.
• Adversarial testing: Simulate real-world attacks using jailbreaking, prompt injection, and other sophisticated attack methods.
• Risk profiling: Comprehensive evaluation across 40+ vulnerability types with detailed risk assessments and remediation guidance.

All vulnerabilities and attacks on DeepTeam are also available on Confident AI.

Create a Risk Assessment

This examples goes through a comprehensive safety assessment using adversarial attacks to identify vulnerabilities in your AI system.

Running red teaming locally executes attacks on your machine and uploads results to Confident AI. This gives full control over custom vulnerabilities and attack methods.

1. Install DeepTeam

   Install DeepTeam, Confident AI's open-source red teaming framework:

   pip install -U deepteam

2. Set Your API Key

   Set your Confident AI API key so results are uploaded to the platform:

   deepteam login

   Or set it as an environment variable:

   export CONFIDENT_API_KEY=YOUR-API-KEY

3. Set Up Your Target Model

   Define your AI system as a model callback function. This is the AI application you want to red team:

   from deepteam.test_case import RTTurn, ToolCall
   
   async def model_callback(input: str) -> str:
       # Replace this with your actual LLM application
       # This could be a RAG pipeline, chatbot, agent, etc.
       return RTTurn(
           role="assistant",
           content="Your agent's response here...",
           retrieval_context=["Your retieval context here"],
           tools_called=[
               ToolCall(name="SearchDatabase")
           ]
       )

4. Pull Your Security Framework

   Pull any security framework you've configured on Confident AI:

   from deepteam.frameworks import RedTeamingFramework
   
   framework = RedTeamingFramework()
   framework.pull("your-framework-id")

   Your framework's id is the last segment of the URL on its configuration page. Pulling a framework brings down every risk category, along with the vulnerability types and attack methods configured for each one.

5. Run the red team assessment

   Run the assessment against the framework you pulled:

   from deepteam import red_team
   
   red_team(
       model_callback=model_callback,
       framework=framework,
       identifier="my-local-assessment",
       run_all_attacks=True
   )

   This runs your model_callback against every risk category, vulnerability type, and attack method in the framework. The risk assessment is printed to your console and also uploaded to Confident AI. You can now view these results in the Risk Profile section on the Confident AI platform.

Using a Pre-defined Framework

deepteam also ships with pre-defined frameworks like OWASP, NIST AI RMF, and MITRE ATLAS:

from deepteam.frameworks import OWASPTop10
from deepteam import red_team

# Run with framework
red_team(
    model_callback=model_callback,
    framework=OWASPTop10(),
)

Results will be posted to the Confident AI platform automatically.

Best Practices

1. Start with frameworks: Use OWASP Top 10 or NIST AI RMF for comprehensive coverage
2. Test early and often: Integrate red teaming into your development cycle
3. Focus on your use case: Customize vulnerabilities based on your application's risks
4. Monitor continuously: Set up ongoing safety assessments for production systems
5. Document and remediate: Keep detailed records of findings and remediation efforts

Next Steps

Last updated on August 12, 2026