LLM API cost per request: usage.cost, logs and totals

Read usage.cost from a reply and log it

A completed request answers 200 with one JSON object, and its usage block has four fields: prompt_tokens, completion_tokens, total_tokens and cost. The first three are counts. cost is the US dollars charged for this request — the amount taken from the balance. It is always present, on text and image models alike, and it is the same figure the request log in your account shows under the reply’s id, the one that starts with gen-.

That makes the reply itself the best place to record what a call cost. Write one line per request — the id, the model, the counters and the amount — and every later question about spend becomes a query over your own file:

import json
import time

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gtokens.io/api/v1",
    api_key="sk-sl-your-key",
)

completion = client.chat.completions.create(
    model="anthropic/claude-sonnet-5",
    messages=[{"role": "user", "content": "Summarise this ticket in two lines."}],
)

usage = completion.usage
record = {
    "ts": int(time.time()),
    "id": completion.id,                  # gen-…, the id in your request log
    "model": completion.model,
    "prompt_tokens": usage.prompt_tokens,
    "completion_tokens": usage.completion_tokens,
    "finish_reason": completion.choices[0].finish_reason,
    "cost": usage.cost,                   # USD taken from the balance
}
with open("costs.jsonl", "a") as f:
    f.write(json.dumps(record) + "\n")

import { appendFile } from 'node:fs/promises'
import OpenAI from 'openai'

const client = new OpenAI({
  baseURL: 'https://api.gtokens.io/api/v1',
  apiKey: 'sk-sl-your-key',
})

const completion = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-5',
  messages: [{ role: 'user', content: 'Summarise this ticket in two lines.' }],
})

const usage = completion.usage!
const record = {
  ts: Math.floor(Date.now() / 1000),
  id: completion.id, // gen-…, the id in your request log
  model: completion.model,
  prompt_tokens: usage.prompt_tokens,
  completion_tokens: usage.completion_tokens,
  finish_reason: completion.choices[0].finish_reason,
  // cost is a gateway field, so the library's types do not know it
  cost: (usage as any).cost as number,
}
await appendFile('costs.jsonl', JSON.stringify(record) + '\n')

Two details in the counters matter for the money. On a text model prompt_tokens includes what was written to and read from the prompt cache; the read part is repeated in prompt_tokens_details.cached_tokens, present only when a cache was involved, and cache reads and writes are billed at rates of their own. So the counters alone do not give you the amount — cost does. And a reply cut off by length is billed for what it produced, so a retry with a higher ceiling costs the truncated attempt as well; that is why finish_reason is in the record.

In a stream: the last chunk before [DONE]

With stream: true the same figure arrives once, at the end. Text comes in delta, finish_reason is null until the last chunk, and that last chunk carries the usage of the whole request, cost included. It is sent whether or not stream_options asked for it, so there is nothing to switch on:

import openai

record = {"ts": int(time.time()), "id": None, "model": "anthropic/claude-haiku-4-5", "cost": None}
try:
    stream = client.chat.completions.create(
        model=record["model"],
        messages=[{"role": "user", "content": "Write a haiku about invoices."}],
        stream=True,
    )
    for chunk in stream:
        record["id"] = chunk.id
        if chunk.choices:
            print(chunk.choices[0].delta.content or "", end="", flush=True)
        if chunk.usage:                   # the last chunk before [DONE]
            record["cost"] = chunk.usage.cost
except openai.APIStatusError:
    record["cost"] = 0.0                  # refused before the stream: nothing charged
    raise
finally:
    # a stream that broke mid-answer leaves cost as None: billed, amount unknown
    with open("costs.jsonl", "a") as f:
        f.write(json.dumps(record) + "\n")

const record: { ts: number; id?: string; model: string; cost: number | null } = {
  ts: Math.floor(Date.now() / 1000),
  model: 'anthropic/claude-haiku-4-5',
  cost: null,
}
try {
  const stream = await client.chat.completions.create({
    model: record.model,
    messages: [{ role: 'user', content: 'Write a haiku about invoices.' }],
    stream: true,
  })
  for await (const chunk of stream) {
    record.id = chunk.id
    process.stdout.write(chunk.choices[0]?.delta.content ?? '')
    if (chunk.usage) record.cost = (chunk.usage as any).cost // the last chunk before [DONE]
  }
} catch (e) {
  // an HTTP status means refused before the stream: nothing charged
  if (e instanceof OpenAI.APIError && e.status !== undefined) record.cost = 0
  throw e
} finally {
  // a stream that broke mid-answer leaves cost null: billed, amount unknown
  await appendFile('costs.jsonl', JSON.stringify(record) + '\n')
}

The two failure paths are not the same case. A failure that can be judged before the model starts writing arrives as a real HTTP status, and nothing is charged for it. A failure after the first byte arrives inside the stream, as a chunk with finish_reason: "error" and an error body, followed by [DONE]; the OpenAI library turns that chunk into an exception. What the model produced before it is billed, but no usage chunk reaches you — so the record keeps the amount empty rather than zero, and the totals further down include it.

An image model does not stream progress: its stream is one chunk with every frame in delta.images, and being the last chunk, it is the one that carries usage.

The Messages route and the image route

The amount is reported in a different place on each route, and on one of them not at all:

Scroll the table sideways for where to look.

The Messages route answers in the shape its client libraries expect, and that shape has no field for money. usage carries input_tokens, output_tokens, cache_creation_input_tokens and cache_read_input_tokens, each billed at its own rate for that model. In a stream the input count arrives in message_start and the final output count in the last message_delta, and billing follows those numbers. If a stream ends before that last message_delta — you disconnected halfway, say — its output is billed by what had been delivered up to that point. Every model that answers on this route also answers on chat/completions with the same billing; if you want the amount inside the reply, that is the route that carries it.

On the image route usage.cost covers the whole request. Fewer frames can come back than you asked for, and you are billed for the ones that arrive. In a stream there is one image_generation.completed event per frame and usage rides on the last one only: the earlier events carry none, and adding up what you see would count the same money twice. That reply has no id — it is created, data and usage — so give each image call a reference of your own in the log.

What to keep in your own log

A cost log earns its keep when each line can be matched to something on the gateway’s side. These fields do that:

• id — the gen-… identifier the request log in your account uses, so a line in your file and a line there describe the same call, with the same amount.
• model — as returned: the catalogue identifier you asked for. There is no substitution, so it is also the model that answered.
• cost — from usage.cost. Zero for a refused request, empty when the amount never arrived because a stream broke.
• The counters and finish_reason — they explain an amount: a long prompt, a cache hit, a reply cut off by length.
• Which key sent it. GET /api/v1/key reports spend per key, so a key per service or per environment turns that route into a per-service total.

Check the totals against the key and credits routes

Two read-only routes report what has been charged. Neither needs a balance or costs anything, so both are safe to call on a schedule.

• GET /api/v1/key describes the key in the header: usage over its whole life, and usage_daily, usage_weekly and usage_monthly for the current calendar day, week and month.
• GET /api/v1/credits covers the whole account, every key on it: total_credits and total_usage, both lifetime. total_usage is summed from the request log — the same amounts as usage.cost — and total_credits - total_usage is the balance.

import json
from datetime import datetime
from zoneinfo import ZoneInfo

import requests

BASE = "https://api.gtokens.io/api/v1"
HEADERS = {"Authorization": "Bearer sk-sl-your-key"}
# The time zone your browser reports when you open the dashboard; until then, UTC.
ACCOUNT_ZONE = ZoneInfo("UTC")


def read(path):
    r = requests.get(BASE + path, headers=HEADERS, timeout=30)
    r.raise_for_status()
    return r.json()["data"]


# Your own log for this key, from midnight in that zone
since = datetime.now(ACCOUNT_ZONE).replace(hour=0, minute=0, second=0, microsecond=0).timestamp()
with open("costs.jsonl") as f:
    lines = [x for x in map(json.loads, f) if x["ts"] >= since]
logged = sum(x["cost"] or 0 for x in lines)
unknown = sum(1 for x in lines if x["cost"] is None)

key = read("/key")           # this key: calendar day, week, month and lifetime
credits = read("/credits")   # the whole account, every key: lifetime totals

print(f"day: logged {logged:.6f}, charged {key['usage_daily']:.6f}, "
      f"{unknown} lines without an amount")
print(f"balance: {credits['total_credits'] - credits['total_usage']:.2f} USD")

The windows are calendar windows in the account’s own time zone: the day starts at midnight, the week on Monday, the month on the first. The zone is the time zone your browser reports when you open the dashboard; until then, UTC. Use that zone in the script, or your day and the key’s day will cover different hours.

A gap between the two numbers should have a reason. The counters are built from finished requests only, so a call still in flight is not in them yet. Lines without an amount — streams that broke after the first byte — are charged but not summed by your script. And another process using the same key adds to the key’s figure without writing to your file. Amounts arrive as plain JSON numbers, so compare against a tolerance rather than testing for equality.

Estimate the cost before you send

GET /api/v1/models returns the whole catalogue, needs no key, and gives every model a pricing object. Each value there is a decimal string: the price in US dollars of one token, not of a million. Multiply by the counters and you have the amount, with no conversion in between. These are the rates your account is billed at; the models page shows the same rates per million tokens, so a difference of six zeros between the two is the unit, not a disagreement.

For a text request the arithmetic is prompt_tokens × pricing.prompt + completion_tokens × pricing.completion, with cached input moved to its own rate. Each counter has its key:

Scroll the table sideways for the rate key.

For a rough upper bound, take the prompt at the input rate and max_tokens at the output rate. That same ceiling decides how much of the balance is held while the request runs — settled against the tokens actually used — so an inflated max_tokens can turn a request the balance would have covered into a 402. On chat/completions a body without it gets 4096. A model takes max_tokens as a length limit only when it lists the field in supported_parameters in the catalogue; for one that does not, plan with what similar calls cost in your log.

Two more limits on the estimate. On an image model the output count is set by the shape of the frame rather than its area, so the figure to reconcile against is usage.cost; the image guide lists measured counts for common frames. And an entry with status coming_soon carries zeros in pricing: that means not priced yet, not free. The current rates of every model are on the models page.