Introducing Confident Monitor

Export Traces

Export trace data as CSV on demand, or on a recurring schedule to an S3 bucket or Snowflake.

Included on the Enterprise plan. Book a demo, opens in a new tab. Included on the Team plan. Included on the Starter plan. Not included on the Free plan.

Traces can be exported as CSV in two ways: a one-off export triggered directly from the Observatory, or a recurring scheduled export that uploads to your preferred data store on a set frequency.

Scheduled exports in Project Settings

One-Off Export

Export traces on demand from the traces list in the Observatory. You can export based on a time range and filters, or export a specific selection of traces you've checked in the table.

To export traces:

  1. Navigate to Observatory → Traces
  2. Apply any filters or select specific rows you want to export
  3. Click Export in the toolbar
  4. If exporting by time range: set the Time range and optionally refine with filters
  5. Click Start export

For exports with 100 or fewer traces, the CSV downloads immediately to your browser. For larger exports, the file is generated in the background and you'll receive an email with a download link when it's ready.

Scheduled Exports

Schedule recurring exports that automatically upload a CSV to a data store at a chosen frequency. This is configured under Project Settings → Exports: the Export Destinations tab, and a schedules tab for each kind of data (Trace Schedules, Thread Schedules, Annotation Schedules, and Test Run Schedules).

Export Destinations

An export destination defines the data store where scheduled export files are uploaded.

To add an export destination:

  1. Navigate to Project Settings → Exports → Export Destinations
  2. Click Add destination, configure it, and save it
Export destinations

Snowflake

Confident AI signs in to Snowflake with key-pair authentication, so you create a service user for it once and give it a key. Only Traces and Annotations schedules can use a Snowflake destination.

  1. Generate a key pair

    Generate an unencrypted private key and its public key:

    openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt
    openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub

    To use an encrypted key instead, drop -nocrypt and enter the passphrase in the destination's optional settings.

  2. Create a role, warehouse and service user

    Run the following as ACCOUNTADMIN, replacing the names as you like. The role needs USAGE on the warehouse, database and schema, and CREATE TABLE on the schema.

    USE ROLE ACCOUNTADMIN;
    
    CREATE ROLE IF NOT EXISTS CONFIDENT_EXPORT_ROLE;
    CREATE WAREHOUSE IF NOT EXISTS CONFIDENT_WH
      WAREHOUSE_SIZE = XSMALL AUTO_SUSPEND = 60 AUTO_RESUME = TRUE INITIALLY_SUSPENDED = TRUE;
    CREATE DATABASE IF NOT EXISTS ANALYTICS;
    CREATE SCHEMA IF NOT EXISTS ANALYTICS.CONFIDENT;
    
    GRANT USAGE ON WAREHOUSE CONFIDENT_WH TO ROLE CONFIDENT_EXPORT_ROLE;
    GRANT USAGE ON DATABASE ANALYTICS TO ROLE CONFIDENT_EXPORT_ROLE;
    GRANT USAGE, CREATE TABLE ON SCHEMA ANALYTICS.CONFIDENT TO ROLE CONFIDENT_EXPORT_ROLE;
    
    CREATE USER IF NOT EXISTS CONFIDENT_EXPORTER
      TYPE = SERVICE
      DEFAULT_ROLE = CONFIDENT_EXPORT_ROLE
      RSA_PUBLIC_KEY = '<contents of rsa_key.pub without the BEGIN/END lines>';
    GRANT ROLE CONFIDENT_EXPORT_ROLE TO USER CONFIDENT_EXPORTER;

    Find your account identifier with:

    SELECT CURRENT_ORGANIZATION_NAME() || '-' || CURRENT_ACCOUNT_NAME();
  3. Add the destination

    Navigate to Project Settings → Exports → Export Destinations, click Add destination and choose Snowflake. Enter the account identifier, username, role, warehouse, database and schema, and paste the contents of rsa_key.p8 as the private key. The key is masked after saving.

  4. Test and save

    Click Test connection. Confident AI signs in, uses the role, warehouse, database and schema, and loads a row into a temporary table, so a passing test means scheduled loads have the access they need. Then click Save destination.

Each run loads into one of two tables in the destination's schema, which Confident AI creates on the first run:

TableExport typeOne row perKey
CONFIDENT_TRACESTracestraceUUID
CONFIDENT_ANNOTATIONSAnnotationsannotationID
  • The columns are the CSV export's columns in UPPER_SNAKE_CASE, for example START_TIME and METRICS_DATA. Timestamps are TIMESTAMP_TZ in UTC, and nested fields such as METADATA, TAGS and METRICS_DATA are VARIANT, so you can query them directly (METRICS_DATA[0]:score).
  • Loads are upserts on the key: a trace or annotation that appears in more than one run is updated in place, never duplicated.
  • CONFIDENT_EXPORTED_AT records when each row was last loaded.
  • When the export gains new columns, they are added to the table automatically. Existing columns are never changed.
  • Snowflake limits a row to 16 MB. A trace larger than that has its longest text fields shortened so the rest of the row still loads.

Schedules

A schedule defines what to export, how often, and to which destination. Each run exports a CSV to your data store.

To create a schedule:

  1. Navigate to Project Settings → Exports and open the tab for what you're exporting, e.g. Trace Schedules

  2. Click Add schedule

  3. Configure the schedule:

    • Destination — select a configured export destination
    • Name — a concise identifier for this schedule
    • What to export — the kind of data the tab covers, e.g. traces as CSV or traces with their spans as JSONL on Trace Schedules
    • Frequency — how often the export runs; each run covers data from the previous interval
    FrequencyInterval covered
    Every 30 minutesPrevious 30 minutes
    HourlyPrevious hour
    DailyPrevious day
    WeeklyPrevious week
    MonthlyPrevious month
    • Filters — optionally limit the export to traces matching specific conditions; leave empty to export all traces in each window
    • Description (optional) — internal context about the schedule's purpose or owner
  4. Click Create schedule

Ready to monitor AI in production?Connect traces, alerts, dashboards, and evals in one production workflowBook a demo

Last updated on

Built byConfident AI