Two flavors, one lifecycle
Both collections expose the same five operations. Everything below is written against
/v1/messages/batches; substitute the path and item shape for the OpenAI flavor.
Create a batch
requests array. Each item carries a custom_id (yours, for matching
results) and a params object — the same body you would send to the live endpoint.
Create
Rules the create enforces:
custom_idis1–64characters ofA–Z,a–z,0–9,_,-, and unique within the batch.- Item
paramstake the same shape as the live endpoint, but must not setstream. - An optional
processing_mode(orlane) field picks the execution lane:standard(default),flex,native-preferredornative-required. The last requires a provider-native batch transport; without one the create returns422 native_batch_unavailable. - Batches bill from the same monthly included-usage pool as live requests, at the same
meter — so PAYG-mode batches are refused up front with
400 payg_batch_not_supported; send synchronous requests on PAYG instead.
Idempotency-Key header: a retry with the same key
and body replays the original response instead of submitting twice.
Lifecycle
processing_status flows in_progress → ended (via
canceling if you cancel). Poll the status endpoint; fetch /results only once the batch
has ended — before that it returns 409.
The batch view carries request_counts — processing, succeeded, errored, canceled,
expired — so a poll shows progress, not just state. A cancel keeps the results of items
that already ran; the un-started tail comes back canceled.
Batches and their results are kept for 24 hours from creation; after that, results
return 410 and the storage is reclaimed.
Results
Results are JSONL — one line per request, keyed by yourcustom_id, with a discriminated
result.type of succeeded | errored | canceled | expired. Order is not guaranteed to
match submission order; always match on custom_id.
Results (JSONL, one line per request)
Batch items draw from the same monthly included-usage pool as live requests, at the same
meter — see Plans & limits.