Red Team Using DeepTeam

Overview

Confident AI’s red teaming capabilities offer a variety of features to test AI safety and security in development for a pre-deployment workflow, offering a wide range of features for:

• Vulnerability assessment: Systematically identify weaknesses like bias, toxicity, PII leakage, and prompt injection vulnerabilities.
• Adversarial testing: Simulate real-world attacks using jailbreaking, prompt injection, and other sophisticated attack methods.
• Risk profiling: Comprehensive evaluation across 40+ vulnerability types with detailed risk assessments and remediation guidance.

All vulnerabilities and attacks on DeepTeam are also available on Confident AI.

Create a Risk Assessment

This examples goes through a comprehensive safety assessment using adversarial attacks to identify vulnerabilities in your AI system.

Running red teaming locally executes attacks on your machine and uploads results to Confident AI. This gives full control over custom vulnerabilities and attack methods.

$
pip install -U deepteam

$
deepteam login

$
export CONFIDENT_API_KEY=YOUR-API-KEY

1
from deepteam.test_case import RTTurn, ToolCall
2
3
async def model_callback(input: str) -> str:
4
    # Replace this with your actual LLM application
5
    # This could be a RAG pipeline, chatbot, agent, etc.
6
    return RTTurn(
7
        role="assistant",
8
        content="Your agent's response here...",
9
        retrieval_context=["Your retieval context here"],
10
        tools_called=[
11
            ToolCall(name="SearchDatabase")
12
        ]
13
    )

1
from deepteam import red_team
2
from deepteam.vulnerabilities import Bias, Toxicity, PIILeakage
3
from deepteam.attacks.single_turn import PromptInjection
4
from deepteam.attacks.multi_turn import LinearJailbreaking
5
6
# Define vulnerabilities to test
7
vulnerabilities = [
8
    Bias(types=["race", "gender", "political"]),
9
    Toxicity(types=["profanity", "insults", "threats"]),
10
    PIILeakage(types=["direct_disclosure", "api_and_database_access"])
11
]
12
13
# Define attack methods
14
attacks = [
15
    PromptInjection(weight=2),  # Higher weight = more likely to be selected
16
    LinearJailbreaking(weight=1)
17
]

1
# Run comprehensive red teaming
2
risk_assessment = red_team(
3
    model_callback=model_callback,
4
    vulnerabilities=vulnerabilities,
5
    attacks=attacks,
6
    attacks_per_vulnerability_type=3,
7
    max_concurrent=5
8
)

Using Security Frameworks

Instead of configuring vulnerabilities and attacks manually, you can use pre-defined security frameworks like OWASP, NIST AI RMF, and MITRE ATLAS via deepteam:

1
from deepteam.frameworks import OWASPTop10
2
from deepteam import red_team
3
4
# Run with framework
5
red_team(
6
    model_callback=model_callback,
7
    framework=OWASPTop10(),
8
)

Results will be posted to the Confident AI platform automatically.

Best Practices

1. Start with frameworks: Use OWASP Top 10 or NIST AI RMF for comprehensive coverage
2. Test early and often: Integrate red teaming into your development cycle
3. Focus on your use case: Customize vulnerabilities based on your application’s risks
4. Monitor continuously: Set up ongoing safety assessments for production systems
5. Document and remediate: Keep detailed records of findings and remediation efforts

Next Steps