Launch Week 3: Five days of launches

Reports

Overview

The Confident AI SDK exposes every Report method on the platform. This page documents how to call these methods in all supported languages. See the introduction to install the SDK and set your API key.

Methods

List Reports

Lists the reports in your Confident AI project one page at a time, newest first, without their sections. Narrow the page with reportTemplateId, status, or a startDate/endDate window; retrieve a report by id to read its sections.

from confident_ai import ConfidentAI
from confident_ai.reports import ReportStatus

client = ConfidentAI()

result = client.reports.list(
    report_template_id="<REPORT-TEMPLATE-ID>",
    status=ReportStatus.IN_PROGRESS,
    start_date="2025-01-01T00:00:00+00:00",
    end_date="2025-01-31T23:59:59+00:00",
    page=1,
    page_size=25,
)

For async mode, call a_list and await it as shown below:

result = await client.reports.a_list(...)

Parameters

ParameterTypeDescription
report_template_idOptional[str]Only return reports written under this report template.
statusOptional[ReportStatus]See ReportStatus.
start_dateOptional[str]Only return reports created at or after this ISO 8601 datetime.
end_dateOptional[str]Only return reports created at or before this ISO 8601 datetime.
pageOptional[int]The page to return. Defaults to 1.
page_sizeOptional[int]The number of results per page, at most 100. Defaults to 25.

Returns

This method returns an object of type ReportList.

Create Report

Writes a report into your Confident AI project and returns its id. You supply the finished section content and it is stored and rendered exactly as given, under the report template you name.

from confident_ai import ConfidentAI
from confident_ai.reports import ReportContentSectionInput
from confident_ai.reports import ReportDateRange
from confident_ai.reports import ReportMetadataInput
from confident_ai.reports import ReportNarrativeContent
from confident_ai.reports import ReportStatus

client = ConfidentAI()

result = client.reports.create(
    report_template_id="<REPORT-TEMPLATE-ID>",
    sections=[
        ReportContentSectionInput(
            type="CONTENT",
            heading="Overview",
            start_on_new_page=False,
            content=ReportNarrativeContent(
                kind="narrative",
                narrative="Traffic held steady while the error rate fell by a third, and spend stayed inside budget.",
                sources=["Traces, 1-8 Jan 2025"]
            )
        )
    ],
    status=ReportStatus.IN_PROGRESS,
    error="<ERROR>",
    metadata=ReportMetadataInput(
        report_title="Weekly Health Check",
        description="Production health for the last week.",
        date_range=ReportDateRange(
            start_date="2025-01-01T00:00:00+00:00",
            end_date="2025-01-08T00:00:00+00:00"
        )
    ),
)

For async mode, call a_create and await it as shown below:

result = await client.reports.a_create(...)

Parameters

ParameterTypeDescription
report_template_idstrRequired. The report template to write this report under. Required, since a report is read under its template.
sectionsList[ReportSectionInput]Required. The report's sections, in render order. At least one is required. See ReportSectionInput.
statusOptional[ReportStatus]See ReportStatus.
errorOptional[str]Why the report failed, when creating it as ERRORED.
metadataOptional[ReportMetadataInput]See ReportMetadataInput.

Returns

This method returns an object of type ReportRef.

Get Report

Retrieves a report by id, with every section it renders in order. A section Confident AI is still writing comes back with null content.

from confident_ai import ConfidentAI

client = ConfidentAI()

result = client.reports.get(report_id="<REPORT-ID>")

For async mode, call a_get and await it as shown below:

result = await client.reports.a_get(...)

Parameters

ParameterTypeDescription
report_idstrRequired. The id of the report.

Returns

This method returns an object of type Report.

Update Report

Updates a report and returns it. Only the fields you send are changed: metadata is merged onto the report's stored header, while sections replaces its section list wholesale.

from confident_ai import ConfidentAI
from confident_ai.reports import ReportContentSectionInput
from confident_ai.reports import ReportDateRange
from confident_ai.reports import ReportMetadataInput
from confident_ai.reports import ReportNarrativeContent
from confident_ai.reports import ReportStatus

client = ConfidentAI()

result = client.reports.update(
    report_id="<REPORT-ID>",
    status=ReportStatus.IN_PROGRESS,
    error="<ERROR>",
    metadata=ReportMetadataInput(
        report_title="Weekly Health Check",
        description="Production health for the last week.",
        date_range=ReportDateRange(
            start_date="2025-01-01T00:00:00+00:00",
            end_date="2025-01-08T00:00:00+00:00"
        )
    ),
    sections=[
        ReportContentSectionInput(
            type="CONTENT",
            heading="Overview",
            start_on_new_page=False,
            content=ReportNarrativeContent(
                kind="narrative",
                narrative="Traffic held steady while the error rate fell by a third, and spend stayed inside budget.",
                sources=["Traces, 1-8 Jan 2025"]
            )
        )
    ],
)

For async mode, call a_update and await it as shown below:

result = await client.reports.a_update(...)

Parameters

ParameterTypeDescription
report_idstrRequired. The id of the report.
statusOptional[ReportStatus]See ReportStatus.
errorOptional[str]Why the report failed. Pair it with a status of ERRORED, or send null to clear it.
metadataOptional[ReportMetadataInput]See ReportMetadataInput.
sectionsOptional[List[ReportSectionInput]]The report's sections, in render order. The list replaces the report's current sections rather than adding to them. See ReportSectionInput.

Returns

This method returns an object of type Report.

Delete Report

Permanently deletes a report and all of its sections. The report template it was written under is kept.

from confident_ai import ConfidentAI

client = ConfidentAI()

result = client.reports.delete(report_id="<REPORT-ID>")

For async mode, call a_delete and await it as shown below:

result = await client.reports.a_delete(...)

Parameters

ParameterTypeDescription
report_idstrRequired. The id of the report.

Returns

This method returns an object of type ReportRef.

Types

Report

A report written under a report template, with every section it renders.

class Report:
    id: str
    report_template_id: Optional[str] = Field(alias="reportTemplateId")
    status: ReportStatus
    error: Optional[str]
    metadata: Optional[ReportMetadata]
    created_at: str = Field(alias="createdAt")
    updated_at: str = Field(alias="updatedAt")
    sections: List[ReportSection]

idstrRequired

The id of the report, generated by Confident AI.

Example: "<REPORT-ID>"

report_template_idOptional[str]Required

The id of the report template this report is written under, or null once that template has been deleted, which leaves the report unreachable on the platform.

Example: "<REPORT-TEMPLATE-ID>"

statusReportStatusRequired

errorOptional[str]Required

Why the report failed, when it did.

metadataOptional[ReportMetadata]Required

created_atstrRequired

When the report was created.

Example: "2025-01-08T00:00:00+00:00"

updated_atstrRequired

When the report was last updated.

Example: "2025-01-08T00:00:00+00:00"

sectionsList[ReportSection]Required

The report's sections, ordered as they render.

See ReportSection.

ReportAdmonitionContent

The content of an ADMONITION section — a callout carrying a severity.

class ReportAdmonitionContent:
    severity: ReportAdmonitionSeverity
    text: str
    sources: Optional[List[str]] = None

severityReportAdmonitionSeverityRequired

textstrRequired

One to three sentences.

Example: "The error rate is falling but still sits above the 1% target for a second week."

sourcesOptional[List[str]]

Reader-facing labels for the data this section draws on, rendered beneath it. Omit them unless you want the section to cite where its numbers came from.

Example: ["Traces, 1-8 Jan 2025"]

ReportAdmonitionSeverity

How an ADMONITION section's callout is styled.

class ReportAdmonitionSeverity(Enum):
    INFO = "INFO"
    SUCCESS = "SUCCESS"
    WARNING = "WARNING"
    DANGER = "DANGER"

INFO · SUCCESS · WARNING · DANGER

ReportDateRange

The window a report describes, shown in its header.

class ReportDateRange:
    start_date: str = Field(alias="startDate")
    end_date: str = Field(alias="endDate")

start_datestrRequired

The start of the window, as an ISO 8601 datetime.

Example: "2025-01-01T00:00:00+00:00"

end_datestrRequired

The end of the window, as an ISO 8601 datetime.

Example: "2025-01-08T00:00:00+00:00"

ReportGraphConfigContent

The content of a GRAPH section that Confident AI generated as a live query rather than a snapshot. It carries no data of its own: the query is run against your project when the report is rendered. Read-only — a chart you write yourself is always a snapshot.

class ReportGraphConfigContent:
    type: Literal["config"]
    source: Optional[str] = None
    title: str
    data_model: str = Field(alias="dataModel")
    metric: str
    dimension: Optional[str] = None
    granularity: Optional[str] = None
    span_type: Optional[str] = Field(default=None, alias="spanType")
    start_date: Optional[str] = Field(default=None, alias="startDate")
    end_date: Optional[str] = Field(default=None, alias="endDate")
    sources: Optional[List[str]] = None

typeLiteral["config"]Required

Always config.

Example: "config"

sourceOptional[str]

Which data source resolves the query.

Example: "aggregate"

titlestrRequired

The chart title.

Example: "Daily trace error rate"

data_modelstrRequired

The data model queried, such as TRACE, SPAN or METRIC_DATA.

Example: "TRACE"

metricstrRequired

The aggregate plotted, such as error_rate or total_cost.

Example: "error_rate"

dimensionOptional[str]

The property the metric is split by, giving one line per value. Null for a plain trend over time.

granularityOptional[str]

The time bucket, such as hour, day, week or month.

Example: "day"

span_typeOptional[str]

The span type queried, for SPAN charts only.

start_dateOptional[str]

The start of the queried window as an ISO 8601 datetime, pinning the chart to a point in time.

Example: "2025-01-01T00:00:00+00:00"

end_dateOptional[str]

The end of the queried window as an ISO 8601 datetime.

Example: "2025-01-08T00:00:00+00:00"

sourcesOptional[List[str]]

Reader-facing labels for the data this section draws on, rendered beneath it. Omit them unless you want the section to cite where its numbers came from.

Example: ["Traces, 1-8 Jan 2025"]

ReportGraphContent

The content of a GRAPH section — a chart with its data baked in. This is the only chart form you can write over the API, so the chart always renders exactly the numbers you supply.

class ReportGraphContent:
    type: Literal["snapshot"]
    graph_type: ReportGraphType = Field(alias="graphType")
    categories: List[str]
    series: List[ReportGraphSeries]
    x_axis_label: Optional[str] = Field(default=None, alias="xAxisLabel")
    y_axis_label: Optional[str] = Field(default=None, alias="yAxisLabel")
    sources: Optional[List[str]] = None

typeLiteral["snapshot"]Required

Always snapshot.

Example: "snapshot"

graph_typeReportGraphTypeRequired

categoriesList[str]Required

The x-axis labels. At least one is required.

Example: ["2025-01-06","2025-01-07","2025-01-08"]

seriesList[ReportGraphSeries]Required

One entry per plotted line. Every series' values must be the same length as categories.

See ReportGraphSeries.

x_axis_labelOptional[str]

A label for the x-axis.

Example: "Day"

y_axis_labelOptional[str]

A label for the y-axis.

Example: "Errored traces / total traces"

sourcesOptional[List[str]]

Reader-facing labels for the data this section draws on, rendered beneath it. Omit them unless you want the section to cite where its numbers came from.

Example: ["Traces, 1-8 Jan 2025"]

ReportGraphSeries

One plotted line on a GRAPH section.

class ReportGraphSeries:
    name: str
    values: List[float]
    color: Optional[str] = None

namestrRequired

The series label, which doubles as its legend entry.

Example: "Error Rate"

valuesList[float]Required

One number per category, aligned positionally with them.

Example: [0.031,0.025,0.021]

colorOptional[str]

A colour for the series. Confident AI picks one when omitted.

Example: "#F97316"

ReportGraphType

The chart style a GRAPH section renders as.

class ReportGraphType(Enum):
    LINE = "LINE"
    AREA = "AREA"
    BAR = "BAR"
    STACKED_BAR = "STACKED_BAR"

LINE · AREA · BAR · STACKED_BAR

ReportList

One page of reports, with the total across all pages.

class ReportList:
    reports: List[ReportSummary]
    total_reports: int = Field(alias="totalReports")
    page: int
    page_size: int = Field(alias="pageSize")

reportsList[ReportSummary]Required

The reports for the current page, newest first.

See ReportSummary.

total_reportsintRequired

The total number of reports matching the filters.

Example: 12

pageintRequired

The page this response covers.

Example: 1

page_sizeintRequired

The number of reports per page.

Example: 25

ReportMetadata

The report's stored header information. A field the report was written without comes back as null.

class ReportMetadata:
    report_title: Optional[str] = Field(alias="reportTitle")
    description: Optional[str]
    date_range: Optional[ReportDateRange] = Field(alias="dateRange")
    generated_at: Optional[str] = Field(alias="generatedAt")

report_titleOptional[str]Required

The report's title, as rendered in its header.

Example: "Weekly Health Check"

descriptionOptional[str]Required

One line on what the report covers.

Example: "Production health for the last week."

date_rangeOptional[ReportDateRange]Required

generated_atOptional[str]Required

When the report was written. Always stamped by Confident AI.

Example: "2025-01-08T00:00:00+00:00"

ReportMetadataInput

The report's header information. Confident AI stamps generatedAt itself.

class ReportMetadataInput:
    report_title: Optional[str] = Field(default=None, alias="reportTitle")
    description: Optional[str] = None
    date_range: Optional[ReportDateRange] = Field(default=None, alias="dateRange")

report_titleOptional[str]

The report's title. Defaults to the name of the report template it is written under.

Example: "Weekly Health Check"

descriptionOptional[str]

One line on what the report covers.

Example: "Production health for the last week."

date_rangeOptional[ReportDateRange]

ReportNarrativeContent

The content of a CONTENT section — a block of prose.

class ReportNarrativeContent:
    kind: Literal["narrative"]
    narrative: str
    sources: Optional[List[str]] = None

kindLiteral["narrative"]Required

Always narrative.

Example: "narrative"

narrativestrRequired

Plain text only — no markdown headings, bold, or code fences. Do not repeat the section's heading, which renders above this text. Express a list as one item per line, each starting with "- ".

Example: "Traffic held steady while the error rate fell by a third, and spend stayed inside budget."

sourcesOptional[List[str]]

Reader-facing labels for the data this section draws on, rendered beneath it. Omit them unless you want the section to cite where its numbers came from.

Example: ["Traces, 1-8 Jan 2025"]

ReportRef

A reference to a report by its id.

class ReportRef:
    id: str

idstrRequired

The id of the report, generated by Confident AI.

Example: "<REPORT-ID>"

ReportSection

One section of a report. Its content is null while Confident AI is still writing it.

class ReportSection:
    id: str
    type: ReportSectionType
    heading: Optional[str]
    order: int
    content: Optional[Union[ReportNarrativeContent, ReportAdmonitionContent, ReportStatCardsContent, ReportTableContent, ReportGraphContent, ReportGraphConfigContent]]
    error: Optional[str]
    start_on_new_page: Optional[bool] = Field(alias="startOnNewPage")

idstrRequired

The id of the section, generated by Confident AI.

Example: "<REPORT-SECTION-ID>"

typeReportSectionTypeRequired

headingOptional[str]Required

The heading rendered above the section, or null when it has none.

Example: "Overview"

orderintRequired

The section's position in the report, starting at 0.

Example: 0

contentOptional[Union[ReportNarrativeContent, ReportAdmonitionContent, ReportStatCardsContent, ReportTableContent, ReportGraphContent, ReportGraphConfigContent]]Required

errorOptional[str]Required

Why this section failed to generate, when it did.

start_on_new_pageOptional[bool]Required

Whether the section starts on a new page in the exported report.

Example: false

ReportSectionInput

One section to write into a report. Its content must match its type, and sections render in the order you send them.

ReportSectionInput = Union[
    ReportContentSectionInput,
    ReportAdmonitionSectionInput,
    ReportStatCardsSectionInput,
    ReportTableSectionInput,
    ReportGraphSectionInput,
]

A ReportSectionInput is one of the shapes below. Send the fields of one of them, never a mix of both.

A section of prose.

class ReportContentSectionInput:
    type: Literal["CONTENT"]
    heading: Optional[str] = None
    start_on_new_page: Optional[bool] = Field(default=None, alias="startOnNewPage")
    content: ReportNarrativeContent

typeLiteral["CONTENT"]Required

Always CONTENT.

Example: "CONTENT"

headingOptional[str]

The heading rendered above the section. Omit it for an unheaded section.

Example: "Overview"

start_on_new_pageOptional[bool]

Whether the section starts on a new page in the exported report. Defaults to false.

Example: false

contentReportNarrativeContentRequired

ReportSectionType

What a section renders as, which decides the shape of its content: CONTENT is prose, ADMONITION a callout, STAT_CARDS a row of headline numbers, TABLE a grid, and GRAPH a chart.

class ReportSectionType(Enum):
    CONTENT = "CONTENT"
    STAT_CARDS = "STAT_CARDS"
    TABLE = "TABLE"
    GRAPH = "GRAPH"
    ADMONITION = "ADMONITION"

CONTENT · STAT_CARDS · TABLE · GRAPH · ADMONITION

ReportStatCard

One headline number on a STAT_CARDS section.

class ReportStatCard:
    label: str
    value: str
    caption: Optional[str] = None

labelstrRequired

A short Title Case phrase of 2-4 words — never a sentence or a raw column name.

Example: "Error Rate"

valuestrRequired

A number, percentage, or short phrase, with numbers rounded to 2 decimal places.

Example: "2.10%"

captionOptional[str]

One short supporting line of 10 words or fewer.

Example: "412 of 19,600 requests"

ReportStatCardsContent

The content of a STAT_CARDS section — a row of headline numbers.

class ReportStatCardsContent:
    cards: List[ReportStatCard]
    highlights: Optional[List[ReportStatHighlight]] = None
    sources: Optional[List[str]] = None

cardsList[ReportStatCard]Required

Three to five cards read best. At least one is required.

See ReportStatCard.

highlightsOptional[List[ReportStatHighlight]]

At most three standout findings. Omit rather than padding.

See ReportStatHighlight.

sourcesOptional[List[str]]

Reader-facing labels for the data this section draws on, rendered beneath it. Omit them unless you want the section to cite where its numbers came from.

Example: ["Traces, 1-8 Jan 2025"]

ReportStatHighlight

A standout finding shown alongside the stat cards.

class ReportStatHighlight:
    label: str
    value: str

labelstrRequired

A short Title Case phrase.

Example: "Costliest Model"

valuestrRequired

The highlighted value.

Example: "gpt-4o, $96.20"

ReportStatus

Where a report is in its life: IN_PROGRESS while its sections are still being written, COMPLETED once it is readable, ERRORED when writing it failed.

class ReportStatus(Enum):
    IN_PROGRESS = "IN_PROGRESS"
    COMPLETED = "COMPLETED"
    ERRORED = "ERRORED"

IN_PROGRESS · COMPLETED · ERRORED

ReportSummary

A report as it appears in a list: its status and header information, without its sections.

class ReportSummary:
    id: str
    report_template_id: Optional[str] = Field(alias="reportTemplateId")
    status: ReportStatus
    error: Optional[str]
    metadata: Optional[ReportMetadata]
    created_at: str = Field(alias="createdAt")
    updated_at: str = Field(alias="updatedAt")

idstrRequired

The id of the report, generated by Confident AI.

Example: "<REPORT-ID>"

report_template_idOptional[str]Required

The id of the report template this report is written under, or null once that template has been deleted, which leaves the report unreachable on the platform.

Example: "<REPORT-TEMPLATE-ID>"

statusReportStatusRequired

errorOptional[str]Required

Why the report failed, when it did.

metadataOptional[ReportMetadata]Required

created_atstrRequired

When the report was created.

Example: "2025-01-08T00:00:00+00:00"

updated_atstrRequired

When the report was last updated.

Example: "2025-01-08T00:00:00+00:00"

ReportTableContent

The content of a TABLE section — headers and the rows beneath them.

class ReportTableContent:
    headers: List[str]
    rows: List[List[str]]
    sources: Optional[List[str]] = None

headersList[str]Required

The column headers. At least one is required.

Example: ["Model","Total Cost"]

rowsList[List[str]]Required

The rows. Every row must hold exactly as many cells as there are headers, in the same order.

Example: [["gpt-4o","$96.20"],["gpt-4o-mini","$32.20"]]

sourcesOptional[List[str]]

Reader-facing labels for the data this section draws on, rendered beneath it. Omit them unless you want the section to cite where its numbers came from.

Example: ["Traces, 1-8 Jan 2025"]

Building a production pipeline?Design a scalable API workflow for evals, datasets, traces, and promptsTalk to an engineer

Last updated on

Built byConfident AI