Skip to main content

Two flavors, one lifecycle

Both collections expose the same five operations. Everything below is written against /v1/messages/batches; substitute the path and item shape for the OpenAI flavor.

Create a batch

Send an inline requests array. Each item carries a custom_id (yours, for matching results) and a params object — the same body you would send to the live endpoint.
Create
Rules the create enforces:
  • custom_id is 1–64 characters of A–Z, a–z, 0–9, _, -, and unique within the batch.
  • Item params take the same shape as the live endpoint, but must not set stream.
  • An optional processing_mode (or lane) field picks the execution lane: standard (default), flex, native-preferred or native-required. The last requires a provider-native batch transport; without one the create returns 422 native_batch_unavailable.
  • Batches bill from the same monthly included-usage pool as live requests, at the same meter — so PAYG-mode batches are refused up front with 400 payg_batch_not_supported; send synchronous requests on PAYG instead.
Create accepts an Idempotency-Key header: a retry with the same key and body replays the original response instead of submitting twice.

Lifecycle

Processing is asynchronous. processing_status flows in_progress → ended (via canceling if you cancel). Poll the status endpoint; fetch /results only once the batch has ended — before that it returns 409. The batch view carries request_counts — processing, succeeded, errored, canceled, expired — so a poll shows progress, not just state. A cancel keeps the results of items that already ran; the un-started tail comes back canceled. Batches and their results are kept for 24 hours from creation; after that, results return 410 and the storage is reclaimed.

Results

Results are JSONL — one line per request, keyed by your custom_id, with a discriminated result.type of succeeded | errored | canceled | expired. Order is not guaranteed to match submission order; always match on custom_id.
Results (JSONL, one line per request)
Batch items draw from the same monthly included-usage pool as live requests, at the same meter — see Plans & limits.