Query Databricks served endpoints

One row per group of Databricks model serving endpoints, carrying spend (split into request and idle spend), request counts, error rate and token throughput for each endpoint with usage over the requested period, or the list of endpoints and their configuration when only configuration fields are requested.

Returns one row per combination of the requested dimensions, carrying the requested measures. Each row is keyed by dimension and measure name. row_count is the number of rows in items: results are not paginated, so an aggregation with more rows than limit is cut short at limit rather than continued under a page_token. Send X-Team-Id to evaluate the query as a team you belong to; rows are then limited to what that team’s roles reach.

Use this route for the spend, traffic and errors of Databricks Model Serving endpoints. Usage measures return only endpoints with requests or billed usage in the period. sum_spend is dollars and splits into request_spend, billed to requests, and idle_spend, billed while the endpoint served no requests. idle_spend is the billed DBUs left after the requests that were logged, so an endpoint with billing but no request logs, such as a pay-per-token endpoint that appears only in billing, reads as entirely idle. idle_spend means unused capacity only for endpoints billed for uptime, such as provisioned endpoints. Pay-per-token endpoints are billed per token, so for them it is billed usage that no logged request accounts for, which can be large even when the endpoint logs requests. Do not read it as unused capacity for them. error_rate_percent is 0 to 100, and the sum_*_tokens measures are token counts. created_by is the creator’s display name, or the raw ID where no name is known (created_by_is_unresolved flags that case). Group by usage_group to categorize endpoints into your usage groups.

The fields in the request pick the source. endpoint_count counts the list of endpoints. It reads that list, which includes endpoints with no usage and deleted ones, and takes config and identity dimensions (entity_types, tasks, served_entity_names, entity_names, scale_to_zero_enabled, endpoint_delete_time) and usage_group. It takes no date filter, and a request that pairs it with a usage field such as day is rejected. active_endpoint_count counts endpoints with requests or billed usage in the period, and reads usage. Usage groups apply to both, but the list carries no billing fields, so a rule that uses databricks_sku_name or databricks_billing_origin_product matches no endpoint on the list while it still groups the same endpoint’s spend. Counts from the list and spend by usage group can differ for such rules. The creator comes from the endpoint’s config, so an endpoint that appears only in billing has no creator, and a rule on databricks_created_by does not match it on any source of this route. Endpoints that appear only in billing are in the list too, with their name from billing and null config dimensions. Because they have no config, endpoint_delete_time is always null for them, so they are never marked deleted, even if they last billed long ago. Naming hour or hour_of_day reads hourly data, and any other request with a usage field reads daily data. For total Model Serving spend, use spend with databricks_billing_origin_product = "Model Serving". For individual requests, use databricks-model-serving.