Messages API with a custom base URL in Python and TypeScript

Create the client

Install the vendor’s client library for the Messages API — anthropic on PyPI, @anthropic-ai/sdk on npm — or keep the one you already have:

pip install anthropic

npm install @anthropic-ai/sdk

Two arguments change: the base URL and the credential. The base URL is the root of the gateway; the library adds the version segment and the route path itself, so a base URL that already ends in /v1 produces /v1/v1/messages and a 404.

from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.gtokens.io",       # the root: the library appends /v1/messages
    auth_token="sk-sl-your-key",   # auth_token, not api_key
)

message = client.messages.create(
    model="anthropic/claude-sonnet-5-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Explain HTTP 402 in one sentence."}],
)
print(message.content[0].text)
print(message.usage)

import Anthropic from '@anthropic-ai/sdk'

const client = new Anthropic({
  baseURL: 'https://api.gtokens.io', // the root: the library appends /v1/messages
  authToken: 'sk-sl-your-key', // authToken, not apiKey
})

const message = await client.messages.create({
  model: 'anthropic/claude-sonnet-5-5',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Explain HTTP 402 in one sentence.' }],
})
console.log(message.content)
console.log(message.usage)

The model is named by its catalogue identifier in the form vendor/model. There are no aliases: a name the catalogue does not carry comes back 404, and the model you name is the model that answers.

Or leave the code alone and use the environment

Both libraries read the base URL and the auth token from the environment when the constructor is called without them. Code that does Anthropic() with no arguments moves to the gateway with three lines and no edit:

export ANTHROPIC_BASE_URL=https://api.gtokens.io
export ANTHROPIC_AUTH_TOKEN=sk-sl-your-key
export ANTHROPIC_API_KEY=

Keep ANTHROPIC_API_KEY empty so the auth token is the only credential in play: the gateway reads Authorization: Bearer and ignores x-api-key.

Stream the reply

Streaming uses the library’s own helpers. The gateway answers with text/event-stream in the shape the helpers parse — message_start, content deltas, message_delta with the final counters and message_stop — so text arrives while the model is still writing:

with client.messages.stream(
    model="anthropic/claude-sonnet-5-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Write a haiku about latency."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
    final = stream.get_final_message()

print()
print(final.usage.input_tokens, final.usage.output_tokens)

const stream = client.messages.stream({
  model: 'anthropic/claude-sonnet-5-5',
  max_tokens: 1024,
  messages: [{ role: 'user', content: 'Write a haiku about latency.' }],
})

stream.on('text', (text) => process.stdout.write(text))

const final = await stream.finalMessage()
console.log('\n', final.usage.input_tokens, final.usage.output_tokens)

The counters arrive in two places: input in message_start, final output in the last message_delta. The helpers assemble them into the final message for you, and billing follows the same numbers. If the stream ends before that last message_delta — you disconnected halfway, say — the output is billed by what had been delivered up to that point.

Read usage and work out the cost

Every reply carries four counters, and each is billed at its own rate for that model:

Scroll the table sideways for the description.

The cost of a call is each counter times its rate per token, summed. A few models step their rate up past a prompt size; the step is named on the model page. Rates are published per model on the models page and in GET /api/v1/models.

Cache the part that repeats

Prompt caching works as the library documents it: mark the long, stable part of the prompt with cache_control, and later calls that start with the same blocks read it from the cache. The first call writes the cache, the next ones read it, and both show up in their own counters:

message = client.messages.create(
    model="anthropic/claude-sonnet-5-5",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": LONG_INSTRUCTIONS,              # the part that repeats
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[{"role": "user", "content": question}],
)

u = message.usage
print(u.cache_creation_input_tokens, u.cache_read_input_tokens)

A cache read is the cheapest of the four rates, so long system prompts, tool definitions and documents you ask about repeatedly are worth caching.

Handle errors by status

Failures come back in the library’s native error envelope, so its exception classes work unchanged. Next to error.type the body carries error.error_type — the same vocabulary every other route uses — and request_id is present but may be null, so read it without requiring a value:

import anthropic

try:
    message = client.messages.create(...)
except anthropic.APIStatusError as e:
    print(e.status_code)                        # 401, 402, 404, 503, ...
    error = (e.body or {}).get("error", {})
    print(error.get("error_type"))              # same vocabulary on every route
    print(e.response.headers.get("retry-after"))  # set on 429 and 503

Scroll the table sideways for what to do.

Both libraries also retry some statuses on their own; the max_retries option (maxRetries in TypeScript) decides how many times. Set it to 0 if you would rather honour Retry-After yourself.