Create Control

POSThttps://api.confident-ai.com/v2/organization/governance-controls

Creates a governance control and returns its id. Supplying the config that matches the control's type snapshots its first version in the same call; omitting it creates the control with no definition, which assesses as ERROR until you add a version. Operational controls are seeded by Confident AI and cannot be created here. A new control governs nothing until a governance policy holds it.

POST/v2/organization/governance-controls

curl -X POST "https://api.confident-ai.com/v2/organization/governance-controls" \
  -H "CONFIDENT_API_KEY: <ORGANIZATION-API-KEY>" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "Production error rate under 2%",
  "description": "Traces must error on fewer than 2% of production requests over the last day.",
  "type": "RUNTIME",
  "runtimeConfig": {
    "dataModel": "TRACE",
    "aggregation": "Avg cost",
    "thresholdSettings": {
      "value": 0.02,
      "direction": "above"
    },
    "extraQueryParams": {
      "category": "TRACE"
    },
    "filters": {
      "operator": "AND",
      "groups": [
        {
          "operator": "AND",
          "filters": [
            {
              "category": "User Id",
              "condition": "Is less than",
              "value": "string",
              "key": "string"
            }
          ]
        }
      ]
    },
    "severity": "CRITICAL"
  },
  "preDeploymentConfig": {
    "identifier": "pre-release",
    "window": {
      "days": 30
    },
    "officialOnly": false,
    "filters": {
      "operator": "AND",
      "groups": [
        {
          "operator": "AND",
          "filters": [
            {
              "category": "User Id",
              "condition": "Is less than",
              "value": "string",
              "key": "string"
            }
          ]
        }
      ]
    },
    "severity": "CRITICAL"
  },
  "governancePolicyId": "<GOVERNANCE-POLICY-ID>"
}'

200

{
  "success": true,
  "data": {
    "id": "<GOVERNANCE-CONTROL-ID>"
  },
  "link": "https://app.confident-ai.com/organization/<ORGANIZATION-ID>/governance/controls/<GOVERNANCE-CONTROL-ID>",
  "deprecated": false
}

Headers

CONFIDENT_API_KEYstringRequired
The organization API key for your Confident AI organization.

Request body

namestringRequired
The name of the control, unique within your organization.
descriptionstring | null
What the control checks and why. Send null to leave it unset.
typeenumRequired
The control types you can create. OPERATIONAL is absent because Confident AI seeds those controls from its own registry.
Show 3 enum valuesHide 3 enum values
- RUNTIME
- PRE_DEPLOYMENT_EVALS
- PRE_DEPLOYMENT_RED_TEAMING
runtimeConfigobject
The rule a RUNTIME control evaluates, read as one sentence: aggregate aggregation over dataModel for the trailing 24 hours, restricted to filters, and fail when the result sits on the direction side of the threshold. Aggregating Error rate over TRACE against a threshold of 0.02 above fails a project whose traces errored on more than 2% of requests in the last day. The window is fixed and is not part of the definition. The aggregation has to be one the data model supports.
Show 6 propertiesHide 6 properties
- dataModelenum | nullRequired
  The production data a runtime control measures: TRACE for whole requests, SPAN for individual steps, THREAD for conversations, METRIC_DATA for evaluation scores, and ANNOTATION for human ratings. It decides which aggregations are valid.
  Show 5 enum valuesHide 5 enum values
  - TRACE
  - SPAN
  - THREAD
  - METRIC_DATA
  - ANNOTATION
- aggregationenum | nullRequired
  How a runtime control reduces the data it measures to the single number it compares against its threshold. Each aggregation is only valid for some data models — Avg score and Pass rate need METRIC_DATA, Avg rating needs ANNOTATION, Input tokens needs SPAN — and a pairing the selected dataModel does not support is rejected.
  Show 22 enum valuesHide 22 enum values
  - Avg cost
  - Avg latency
  - Avg rating
  - Avg score
  - Count
  - Error count
  - Error rate
  - Failure rate
  - Input cost
  - Input tokens
  - Median score
  - Output cost
  - Output tokens
  - P50 latency
  - P90 latency
  - P99 latency
  - Pass rate
  - Total cost
  - Total tokens
  - Unique end users
  - Unique metadata values
  - Unique threads
- thresholdSettingsobject | nullRequired
  The comparison that turns a runtime control's measured value into a verdict.
  Show 2 propertiesHide 2 properties
  - valuenumberRequired
    The number the aggregated value is compared against, in the unit the aggregation produces — a rate is a fraction between 0 and 1, a latency is in milliseconds, a cost is in USD.
  - directionenumRequired
    Which side of the threshold fails: above fails once the measured value rises past value, below fails once it drops under it.
    Show 2 enum valuesHide 2 enum values
    above
    below
- extraQueryParamsobject | null
  Extra scoping for the data a runtime control measures, beyond its data model and filters.
  Show 1 propertyHide 1 property
  - categoryenum
    Which kind of item the evaluation scores were recorded on, for a control measuring METRIC_DATA.
    Show 3 enum valuesHide 3 enum values
    TRACE
    SPAN
    THREAD
- filtersobject | nullRequired
  A set of filter groups combined by a top-level operator. Each group combines its filter rows by its own operator, and each row matches one property, such as Name or User Id, against a value with a condition such as Is or Contains.
  Show 2 propertiesHide 2 properties
  - operatorenumRequired
    Show 2 enum valuesHide 2 enum values
    AND
    OR
  - groupslist of objectsRequired
    Show 2 propertiesHide 2 properties
    operatorenumRequired
    Show 2 enum valuesHide 2 enum values
    AND
    OR
    filterslist of objectsRequired
    Show 4 propertiesHide 4 properties
    categoryenumRequired
    Show 80 enum valuesHide 80 enum values
    User Id
    Thread Id
    Trace Uuid
    Trace Name
    Trace Version
    Trace Status
    Trace Tags
    Trace
    Span Uuid
    Name
    Span Name
    Span Type
    Span Status
    Metrics Status
    Error Status
    Name
    Model
    Provider
    Integration
    Embedder
    Chunk Size
    Top-K
    Hyperparameter
    Dataset
    Dataset Name
    Test Run ID
    Identifier
    Test File
    Status
    Official
    Evals Mode
    Tests Passed
    Tests Failed
    Pass Rate
    Fail Rate
    Star Rating
    Thumbs Rating
    Explanation
    Expected Output
    Expected Outcome
    Annotator
    End User
    Annotation Type
    Annotation Name
    Criteria
    Annotation Date
    Metric Score
    Metric Status
    Name
    Metadata
    Classifier
    Metric
    Metric Name
    Trace Count
    Test Case ID
    Requested review from
    Assigned to
    Tags
    Labels
    Tools Called
    Finalized
    Golden ID
    Ingestion Task
    Latency
    Environment
    Review flag
    Vulnerability
    Vulnerability Type
    Attack Method
    Risk Category
    Framework
    Assessment ID
    Prompt Alias
    Prompt Version
    Prompt Label
    Prompt Commit Hash
    Prompt
    Annotations
    Status Code
    Actor Type
    conditionenum | enum | enum | enum | enum | enum | enum | enum | enum | enumRequired
    Show 10 variantsHide 10 variants
    enum
    Show 6 enum valuesHide 6 enum values
    Is less than
    Is equal or less than
    Is greater than
    Is equal or greater than
    Is equal to
    Does not equal
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Has
    Has not
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Is
    Is not
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Is one of
    Is not one of
    OR
    enum
    Show 4 enum valuesHide 4 enum values
    Is
    Is not
    Is empty
    Is not empty
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Contains
    Does not contain
    OR
    enum
    Show 3 enum valuesHide 3 enum values
    Contains
    Contains only
    Does not contain
    OR
    enum
    Show 4 enum valuesHide 4 enum values
    Has decreased by more than
    Has decreased by less than
    Has increased by more than
    Has increased by less than
    OR
    enum
    Show 1 enum valueHide 1 enum value
    Has changed from
    OR
    enum
    Show 1 enum valueHide 1 enum value
    Is between
    valuestring | number | list of stringsRequired
    Show 3 variantsHide 3 variants
    string
    OR
    number
    OR
    list of strings
    keystring
- severityenum | nullRequired
  How much a failing control matters, set per version rather than per control. LOW never blocks a deployment gate; CRITICAL, HIGH and MEDIUM block, and so does leaving the severity unset.
  Show 4 enum valuesHide 4 enum values
  - CRITICAL
  - HIGH
  - MEDIUM
  - LOW
preDeploymentConfigobject
The rule a PRE_DEPLOYMENT_EVALS or PRE_DEPLOYMENT_RED_TEAMING control evaluates, read as one sentence: find the project's newest completed run whose identifier is identifier within the last window.days days, and pass when that run satisfies filters. An identifier of pre-release with a 30-day window and a filter of Pass rate >= 0.9 fails a project whose last pre-release run scored below 90%, and reports NO_DATA when it has not run one at all. PRE_DEPLOYMENT_EVALS looks at test runs and PRE_DEPLOYMENT_RED_TEAMING at red teaming runs; the endpoint picks which by the control's own type, so the same shape serves both. Send a non-empty identifier unless officialOnly is true.
Show 5 propertiesHide 5 properties
- identifierstringRequired
  The identifier of the run to gate on, as sent when the test run or red teaming run was created. Send an empty string when officialOnly is true.
- windowobjectRequired
  The rolling lookback a pre-deployment control searches for the run it gates on.
  Show 1 propertyHide 1 property
  - daysintegerRequired
    How many days back the control looks for a run. It is stored as a day count rather than as dates, so the gate does not go stale as it is re-assessed.
- officialOnlyboolean
  Gate on the project's most recent official run instead of on a run matching identifier, which ignores identifier and the window. Defaults to false.
- filtersobject | nullRequired
  A set of filter groups combined by a top-level operator. Each group combines its filter rows by its own operator, and each row matches one property, such as Name or User Id, against a value with a condition such as Is or Contains.
  Show 2 propertiesHide 2 properties
  - operatorenumRequired
    Show 2 enum valuesHide 2 enum values
    AND
    OR
  - groupslist of objectsRequired
    Show 2 propertiesHide 2 properties
    operatorenumRequired
    Show 2 enum valuesHide 2 enum values
    AND
    OR
    filterslist of objectsRequired
    Show 4 propertiesHide 4 properties
    categoryenumRequired
    Show 80 enum valuesHide 80 enum values
    User Id
    Thread Id
    Trace Uuid
    Trace Name
    Trace Version
    Trace Status
    Trace Tags
    Trace
    Span Uuid
    Name
    Span Name
    Span Type
    Span Status
    Metrics Status
    Error Status
    Name
    Model
    Provider
    Integration
    Embedder
    Chunk Size
    Top-K
    Hyperparameter
    Dataset
    Dataset Name
    Test Run ID
    Identifier
    Test File
    Status
    Official
    Evals Mode
    Tests Passed
    Tests Failed
    Pass Rate
    Fail Rate
    Star Rating
    Thumbs Rating
    Explanation
    Expected Output
    Expected Outcome
    Annotator
    End User
    Annotation Type
    Annotation Name
    Criteria
    Annotation Date
    Metric Score
    Metric Status
    Name
    Metadata
    Classifier
    Metric
    Metric Name
    Trace Count
    Test Case ID
    Requested review from
    Assigned to
    Tags
    Labels
    Tools Called
    Finalized
    Golden ID
    Ingestion Task
    Latency
    Environment
    Review flag
    Vulnerability
    Vulnerability Type
    Attack Method
    Risk Category
    Framework
    Assessment ID
    Prompt Alias
    Prompt Version
    Prompt Label
    Prompt Commit Hash
    Prompt
    Annotations
    Status Code
    Actor Type
    conditionenum | enum | enum | enum | enum | enum | enum | enum | enum | enumRequired
    Show 10 variantsHide 10 variants
    enum
    Show 6 enum valuesHide 6 enum values
    Is less than
    Is equal or less than
    Is greater than
    Is equal or greater than
    Is equal to
    Does not equal
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Has
    Has not
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Is
    Is not
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Is one of
    Is not one of
    OR
    enum
    Show 4 enum valuesHide 4 enum values
    Is
    Is not
    Is empty
    Is not empty
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Contains
    Does not contain
    OR
    enum
    Show 3 enum valuesHide 3 enum values
    Contains
    Contains only
    Does not contain
    OR
    enum
    Show 4 enum valuesHide 4 enum values
    Has decreased by more than
    Has decreased by less than
    Has increased by more than
    Has increased by less than
    OR
    enum
    Show 1 enum valueHide 1 enum value
    Has changed from
    OR
    enum
    Show 1 enum valueHide 1 enum value
    Is between
    valuestring | number | list of stringsRequired
    Show 3 variantsHide 3 variants
    string
    OR
    number
    OR
    list of strings
    keystring
- severityenum | nullRequired
  How much a failing control matters, set per version rather than per control. LOW never blocks a deployment gate; CRITICAL, HIGH and MEDIUM block, and so does leaving the severity unset.
  Show 4 enum valuesHide 4 enum values
  - CRITICAL
  - HIGH
  - MEDIUM
  - LOW
governancePolicyIdstring
Accepted but not acted on: the control is created unattached whether or not you send it. Attach it through the governance policy's own controls endpoint.

Response

Create Control succeeded.

successboolean
Indicates if the request was successful.
dataobject
A reference to a governance control by its id.
Show 1 propertyHide 1 property
- idstring
  The id of the governance control.
linkstring
This is the URL of the resource on the Confident AI platform.
deprecatedboolean
Indicates if this endpoint is deprecated.

Built byConfident AI