Create Scheduled Alert

(v2)

POST

Creates an alert that re-runs an aggregate query on a schedule and notifies when the result crosses the threshold, and returns its id. Notifications are delivered through the project's integrations that have alerting enabled for the alert's severity, so an alert in a project with no such integration still evaluates but reaches nobody.

POST/v2/scheduled-alerts

curl -X POST "https://api.confident-ai.com/v2/scheduled-alerts" \
  -H "CONFIDENT_API_KEY: <PROJECT-API-KEY>" \
  -H "Content-Type: application/json" \
  -d '{
  "name": "Trace error rate spike",
  "description": "Errors above 5% over the last hour.",
  "type": "THRESHOLD",
  "dataModel": "TRACE",
  "aggregation": "ERROR_RATE",
  "thresholdSettings": {
    "value": 0.05,
    "direction": "above"
  },
  "recurrence": "INTERVAL",
  "repeatEvery": 1,
  "repeatUnit": "HOUR",
  "severity": "ERROR",
  "enabled": true
}'

200

{
  "success": true,
  "data": {
    "id": "<SCHEDULED-ALERT-ID>"
  },
  "link": "https://app.confident-ai.com/project/<PROJECT-ID>/alerts",
  "deprecated": false
}

Headers

CONFIDENT_API_KEYstringRequired
The API key of your Confident AI project.

Request body

recurrenceenum
How often a schedule fires: ONCE runs a single time at startAt, INTERVAL repeats every repeatEvery repeatUnits.
Show 2 enum valuesHide 2 enum values
- ONCE
- INTERVAL
repeatEveryinteger | null
How many repeatUnits apart the runs are, for an INTERVAL schedule. Send null to clear it.
repeatUnitenum | null
The unit repeatEvery counts for an INTERVAL schedule.
Show 5 enum valuesHide 5 enum values
- MINUTE
- HOUR
- DAY
- WEEK
- MONTH
startAtstring | null
When the schedule first runs, as an ISO 8601 datetime. Send null to start it immediately.
maxRunsinteger | null
How many times the schedule runs before it stops. Send null to let it run indefinitely.
endAtstring | null
When the schedule stops running, as an ISO 8601 datetime. Send null to leave it open-ended.
descriptionstring | null
What the alert means and what to do about it, included in the notification. Send null to clear it.
filtersobject | null
A set of filter groups combined by a top-level operator. Each group combines its filter rows by its own operator, and each row matches one property, such as Name or User Id, against a value with a condition such as Is or Contains.
Show 2 propertiesHide 2 properties
- operatorenumRequired
  How the groups combine: AND matches a record only when every group matches it, and OR matches it when any one group does.
  Show 2 enum valuesHide 2 enum values
  - AND
  - OR
- groupslist of objectsRequired
  The groups of filters to apply, each combining its own filters with its own operator.
  Show 2 propertiesHide 2 properties
  - operatorenumRequired
    Show 2 enum valuesHide 2 enum values
    AND
    OR
  - filterslist of objectsRequired
    Show 4 propertiesHide 4 properties
    categoryenumRequired
    Show 100 enum valuesHide 100 enum values
    User Id
    Customer Id
    Customer Name
    Thread Id
    Trace UUID
    Trace Name
    Trace Version
    Trace Status
    Trace Tags
    Trace
    Span UUID
    Name
    Span Name
    Span Type
    Span Status
    Metrics Status
    Error Status
    Called tool
    Handed off to
    Handed off by
    Nearest parent of type
    Anywhere under
    Top level
    Name
    Model
    Provider
    Integration
    Input argument
    Embedder
    Chunk Size
    Top-K
    Hyperparameter
    Dataset
    Dataset Name
    Test Run ID
    Identifier
    Test File
    Status
    Official
    Evals Mode
    Tests Passed
    Tests Failed
    Pass Rate
    Fail Rate
    Star Rating
    Thumbs Rating
    Explanation
    Expected Output
    Expected Outcome
    Annotator
    End User
    Annotation Type
    Annotation Name
    Annotation Date
    Annotation Rating
    Annotation Thumbs
    Annotation Choice
    Annotation Yes / No
    Annotation Text
    Metric Score
    Metric Status
    Metric Version
    Name
    Metadata
    Classifier
    Classifier Version
    Metric
    Metric Name
    Trace Count
    Test Case ID
    Requested review from
    Assigned to
    Tags
    Labels
    Tools Called
    Finalized
    Golden ID
    Ingestion Task
    Latency
    Environment
    Review flag
    Vulnerability
    Vulnerability Type
    Attack Method
    Risk Category
    Framework
    Assessment ID
    Prompt Alias
    Prompt Version
    Prompt Label
    Prompt Commit Hash
    Prompt
    Annotations
    Failure Mode
    Recommended
    Sub-mode Name
    Sub-mode Recommended
    Sub-mode Certainty
    Status Code
    Actor Type
    conditionenum | enum | enum | enum | enum | enum | enum | enum | enum | enumRequired
    Show 10 variantsHide 10 variants
    enum
    Show 6 enum valuesHide 6 enum values
    Is less than
    Is equal or less than
    Is greater than
    Is equal or greater than
    Is equal to
    Does not equal
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Has
    Has not
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Is
    Is not
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Is one of
    Is not one of
    OR
    enum
    Show 4 enum valuesHide 4 enum values
    Is
    Is not
    Is empty
    Is not empty
    OR
    enum
    Show 2 enum valuesHide 2 enum values
    Contains
    Does not contain
    OR
    enum
    Show 3 enum valuesHide 3 enum values
    Contains
    Contains only
    Does not contain
    OR
    enum
    Show 4 enum valuesHide 4 enum values
    Has decreased by more than
    Has decreased by less than
    Has increased by more than
    Has increased by less than
    OR
    enum
    Show 1 enum valueHide 1 enum value
    Has changed from
    OR
    enum
    Show 1 enum valueHide 1 enum value
    Is between
    valuestring | number | list of stringsRequired
    Show 3 variantsHide 3 variants
    string
    OR
    number
    OR
    list of strings
    keystring
severityenum
How urgent the alert is. It also decides who hears about it: an integration receives an alert only when it subscribes to that severity.
Show 4 enum valuesHide 4 enum values
- CRITICAL
- ERROR
- WARNING
- INFO
namestringRequired
A name for the alert, shown in the notification.
typeenum
What the alert watches. THRESHOLD alerts compare an aggregation against thresholdSettings. REGRESSION, ANOMALY and SIGNAL_SPIKE alerts notify on every run while Confident AI detects one inside the alert's filters, as configured by detectionSettings. Defaults to THRESHOLD.
Show 4 enum valuesHide 4 enum values
- THRESHOLD
- REGRESSION
- ANOMALY
- SIGNAL_SPIKE
dataModelenumRequired
What kind of production item an alert measures over. TRACE and SPAN alerts aggregate single requests; THREAD alerts aggregate conversations.
Show 8 enum valuesHide 8 enum values
- TRACE
- SPAN
- LLM_SPAN
- AGENT_SPAN
- RETRIEVER_SPAN
- TOOL_SPAN
- CUSTOM_SPAN
- THREAD
aggregationstring
What to measure, as an aggregation token. Which tokens are valid depends on dataModel: TRACE accepts COUNT, ERROR_RATE, PASS_RATE, UNIQUE_END_USERS, UNIQUE_THREADS, AVG_LATENCY, P50_LATENCY, P90_LATENCY, P99_LATENCY, TOTAL_COST, AVG_COST, UNIQUE_METADATA_VALUES; SPAN accepts COUNT, AVG_LATENCY, P50_LATENCY, P90_LATENCY, P99_LATENCY, ERROR_RATE, ERROR_COUNT, INPUT_COST, OUTPUT_COST, TOTAL_COST, AVG_COST, INPUT_TOKENS, OUTPUT_TOKENS, TOTAL_TOKENS, UNIQUE_METADATA_VALUES; LLM_SPAN accepts COUNT, AVG_LATENCY, P50_LATENCY, P90_LATENCY, P99_LATENCY, ERROR_RATE, ERROR_COUNT, INPUT_COST, OUTPUT_COST, TOTAL_COST, AVG_COST, INPUT_TOKENS, OUTPUT_TOKENS, TOTAL_TOKENS, UNIQUE_METADATA_VALUES; AGENT_SPAN accepts COUNT, AVG_LATENCY, P50_LATENCY, P90_LATENCY, P99_LATENCY, ERROR_RATE, ERROR_COUNT, INPUT_COST, OUTPUT_COST, TOTAL_COST, AVG_COST, UNIQUE_METADATA_VALUES; RETRIEVER_SPAN accepts COUNT, AVG_LATENCY, P50_LATENCY, P90_LATENCY, P99_LATENCY, ERROR_RATE, ERROR_COUNT, INPUT_COST, OUTPUT_COST, TOTAL_COST, AVG_COST, UNIQUE_METADATA_VALUES; TOOL_SPAN accepts COUNT, AVG_LATENCY, P50_LATENCY, P90_LATENCY, P99_LATENCY, ERROR_RATE, ERROR_COUNT, INPUT_COST, OUTPUT_COST, TOTAL_COST, AVG_COST, UNIQUE_METADATA_VALUES; CUSTOM_SPAN accepts COUNT, AVG_LATENCY, P50_LATENCY, P90_LATENCY, P99_LATENCY, ERROR_RATE, ERROR_COUNT, INPUT_COST, OUTPUT_COST, TOTAL_COST, AVG_COST, UNIQUE_METADATA_VALUES; THREAD accepts COUNT, UNIQUE_USERS, UNIQUE_METADATA_VALUES. Required for THRESHOLD alerts only.
thresholdSettingsobject
When the alert fires. Latency is compared in seconds, cost in USD, and rates such as ERROR_RATE as fractions between 0 and 1.
Show 2 propertiesHide 2 properties
- valuenumberRequired
  The number the measured value is compared against.
- directionenumRequired
  Whether the alert fires when the measured value rises above the threshold or falls below it.
  Show 2 enum valuesHide 2 enum values
  - above
  - below
detectionSettingsobject | object | object
Which detections notify, for REGRESSION, ANOMALY and SIGNAL_SPIKE alerts. Regressions and anomalies take an optional traceName (omit for every trace), optional metrics (omit for all of avg_score, error_rate, avg_latency, avg_cost, negative_label_rate), direction (WORSENED or ANY) and minAffectedCount, the fewest affected traces a detection needs; regressions also take significantOnly. Signal spikes take classifierIds or classifierCollectionId, direction and minAffectedCount, counted in spans when dataModel is SPAN and in threads when dataModel is THREAD.
Show 3 variantsHide 3 variants
- Regression Detection Settingsobject
  Which regressions notify. traceName narrows to one trace, metrics to one set of measures, direction decides whether only worsened versions count, minAffectedCount is the fewest affected traces a detection needs, and significantOnly keeps statistically significant regressions only.
  Show 5 propertiesHide 5 properties
  - traceNamestring
    The name of the one trace this alert watches. Omit traceName to watch every trace in the project.
  - metricslist of enums
    The measures a detection is reported for. Omit metrics to report on all of avg_score, error_rate, avg_latency, avg_cost and negative_label_rate.
    Show 5 enum valuesHide 5 enum values
    avg_score
    error_rate
    avg_latency
    avg_cost
    negative_label_rate
  - directionenumRequired
    Which detections notify: WORSENED reports only the detections that moved in the worse direction, and ANY reports improvements as well.
    Show 2 enum valuesHide 2 enum values
    WORSENED
    ANY
  - minAffectedCountintegerRequired
    The fewest affected traces a detection needs before the alert notifies. A detection below this count is ignored.
  - significantOnlybooleanRequired
    Whether only statistically significant regressions notify. When significantOnly is false, every regression that meets minAffectedCount notifies.
- OR
- Anomaly Detection Settingsobject
  Which anomalies notify. traceName narrows to one trace, metrics to one set of measures, direction decides whether only worsened buckets count, and minAffectedCount is the fewest affected traces a detection needs.
  Show 4 propertiesHide 4 properties
  - traceNamestring
    The name of the one trace this alert watches. Omit traceName to watch every trace in the project.
  - metricslist of enums
    The measures a detection is reported for. Omit metrics to report on all of avg_score, error_rate, avg_latency, avg_cost and negative_label_rate.
    Show 5 enum valuesHide 5 enum values
    avg_score
    error_rate
    avg_latency
    avg_cost
    negative_label_rate
  - directionenumRequired
    Which detections notify: WORSENED reports only the detections that moved in the worse direction, and ANY reports improvements as well.
    Show 2 enum valuesHide 2 enum values
    WORSENED
    ANY
  - minAffectedCountintegerRequired
    The fewest affected traces a detection needs before the alert notifies. A detection below this count is ignored.
- OR
- Signal Spike Detection Settingsobject
  Which classifier label spikes notify. classifierIds or classifierCollectionId pick the classifiers to watch, direction decides whether only worsened counts count, and minAffectedCount is the fewest affected traces a detection needs, counted in spans when dataModel is SPAN and in threads when dataModel is THREAD.
  Show 4 propertiesHide 4 properties
  - classifierIdslist of stringsRequired
    The ids of the classifiers whose label counts this alert watches for a spike. Leave empty when classifierCollectionId is set.
  - classifierCollectionIdstring
    A classifier collection to watch instead of classifierIds. The alert follows the collection's current classifiers, so editing the collection changes what it watches.
  - directionenumRequired
    Which spikes notify: WORSENED reports only spikes in labels where a rise is bad, and ANY reports spikes in labels where a rise is good as well.
    Show 2 enum valuesHide 2 enum values
    WORSENED
    ANY
  - minAffectedCountintegerRequired
    The fewest affected traces a spike needs before the alert notifies, counted in spans when dataModel is SPAN and in threads when dataModel is THREAD.
enabledboolean
Whether the schedule runs. Defaults to true.

Response

Create Scheduled Alert succeeded.

successboolean
Indicates if the request was successful.
dataobject
A reference to a scheduled alert by its id.
Show 1 propertyHide 1 property
- idstring
  The id of the scheduled alert, generated by Confident AI.
linkstring
This is the URL of the resource on the Confident AI platform.
deprecatedboolean
Indicates if this endpoint is deprecated.

Built byConfident AI