> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sideren.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Prompt caching.

> Agent sessions resend the same context every turn — the system prompt, the attached documents, the conversation so far. On models that cache input, that repeated context is recognized and served from cache automatically, and it uses fewer credits than fresh input.

## Zero configuration

There is nothing to turn on and nothing to mark up. Send requests the way you already do —
no special headers. When a request repeats context the API has seen before — a stable
system prompt, documents pasted into the conversation, a growing message history — the
repeated part is recognized and served from cache.

There are also no cache lifetimes to manage and nothing to invalidate. If the repeated
context is available in cache you get the discount; if it isn't, the request simply counts
as normal input. Correctness never depends on it.

If you already send Anthropic-style `cache_control` markers, they are honored and passed
through untouched — and when you place your own markers, the gateway uses your breakpoints
instead of its own. On the OpenAI shape, a stable `prompt_cache_key` (or `user`) value
keeps a conversation's turns on the same cache.

## How it bills

Every request takes your [credits](/plans) for the model that served it. Each kind of token
counts at that model's own rate:

| Tokens | Counts as |
| - | - |
| Fresh input | Input |
| Cache reads | Cached input, which uses fewer credits than fresh input. Ordinary input where the model doesn't cache. |
| Cache writes | Ordinary input: any provider write premium is absorbed, never passed through |
| Output | Output |

(PAYG mode is the one exception: there, cache writes count at the model's cache-write rate.)

Where a model doesn't cache input, repeated context is still recognized and still returns
faster, but it counts as ordinary input: we don't discount what the model doesn't. Usage
reporting is protocol-standard on both dialects either way. On the OpenAI shape, `prompt_tokens` includes the cached subset, broken out
under `prompt_tokens_details.cached_tokens`:

```json title="OpenAI dialect — usage on a cached turn" theme={null}
{
  "usage": {
    "prompt_tokens": 24211,
    "completion_tokens": 411,
    "total_tokens": 24622,
    "prompt_tokens_details": { "cached_tokens": 23040 }
  }
}
```

On the Anthropic shape, `input_tokens` counts fresh input only and cached input is reported
as `cache_read_input_tokens`:

```json title="Anthropic dialect — usage on a cached turn" theme={null}
{
  "usage": {
    "input_tokens": 1171,
    "output_tokens": 411,
    "cache_read_input_tokens": 23040,
    "cache_creation_input_tokens": 512
  }
}
```

## Getting the most from it

Caching keys off repetition, so the discount follows how you shape your prompts:

* **Stable content first.** Put the system prompt, documents, and examples at the top; the
  part that changes each turn — the user's latest message — last.
* **Keep the prefix byte-stable.** A timestamp, a random id, or a reordered field near the
  top of the prompt makes every turn look new. Append to the conversation; don't rewrite
  what came before.
* **Long sessions benefit most.** Agent sessions and document Q\&A resend a large stable
  prefix on every turn — exactly the shape the cache rewards.
* **Keep a conversation together.** A cache is warm only where the previous turn ran. Send
  a stable `prompt_cache_key` (OpenAI shape) or place `cache_control` markers on the same
  blocks each turn (Anthropic shape) so your turns land together.
* **Churn benefits least.** Tool-call-heavy loops whose context changes substantially
  between turns repeat less, so less of their input is served from cache.

The workloads that gain the most are text-heavy: long-document analysis,
retrieval-augmented chat, and coding sessions over a large stable context.

## Where it shows up

Two places. First, the `usage` block of every response, in the dialect-standard fields
above — your existing token accounting keeps working unchanged. Second, your
[credits](/plans): on a model that caches input, a request whose input is mostly cached
uses far fewer credits than its raw token count suggests. The dashboard's
[usage page](https://www.sideren.io/dashboard/usage) shows what each request used.

<Note>
  Two things cache less than you might expect. Heavy tool-loop turns with fast-changing
  context have little that repeats — the discount follows repetition and can't manufacture
  it. And on a model that doesn't cache input there is no discount to apply. Budget every
  workload as if nothing were cached and treat cache savings as upside.
</Note>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.