OpenAI SDK with a custom base URL: text, streaming and images

Create the client

The request and the reply keep the shape of the OpenAI chat API, so code already written against it works after two arguments change. The model is named by its catalogue identifier in the form vendor/model; there are no aliases and no substitution.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.gtokens.io/api/v1",
    api_key="sk-sl-your-key",
)

completion = client.chat.completions.create(
    model="anthropic/claude-sonnet-5-5",
    messages=[{"role": "user", "content": "Explain HTTP 402 in one sentence."}],
)
print(completion.choices[0].message.content)
print(completion.usage.cost, "USD charged")

import OpenAI from 'openai'

const client = new OpenAI({
  baseURL: 'https://api.gtokens.io/api/v1',
  apiKey: 'sk-sl-your-key',
})

const completion = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-5-5',
  messages: [{ role: 'user', content: 'Explain HTTP 402 in one sentence.' }],
})
console.log(completion.choices[0].message.content)
// cost is a gateway field, so the library's types do not know it
console.log((completion.usage as any).cost, 'USD charged')

The answer is in choices[0].message.content. Next to the token counters, usage carries cost in USD: the amount actually taken from the balance for this request, not an estimate. When you do not send max_tokens, the request gets 4096.

Stream the reply

With stream: true the reply arrives as server-sent chunks. The library stops at the closing [DONE] by itself, and the last chunk before it carries usage with the same cost — no extra option is needed to get it:

stream = client.chat.completions.create(
    model="anthropic/claude-sonnet-5-5",
    messages=[{"role": "user", "content": "Write a haiku about latency."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices:
        print(chunk.choices[0].delta.content or "", end="", flush=True)
    if chunk.usage:                      # the last chunk carries it
        print("\ncharged", chunk.usage.cost)

const stream = await client.chat.completions.create({
  model: 'anthropic/claude-sonnet-5-5',
  messages: [{ role: 'user', content: 'Write a haiku about latency.' }],
  stream: true,
})
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta.content ?? '')
  if (chunk.usage) console.log('\ncharged', (chunk.usage as any).cost)
}

The last chunk carries an empty delta and the finish reason next to usage. If the stream fails after the first byte, the failure arrives as a final error event rather than an HTTP status; the streaming reference shows both cases.

Declare tools

Tool definitions use the OpenAI shape. A text model that decides to call one answers with tool_calls — an id, the function name and its arguments as a JSON string — and your code runs the function and sends the result back as a tool message, as with any OpenAI-compatible endpoint:

tools = [{
    "type": "function",
    "function": {
        "name": "get_balance",
        "description": "Return the prepaid balance of the current account in USD.",
        "parameters": {"type": "object", "properties": {}},
    },
}]

completion = client.chat.completions.create(
    model="anthropic/claude-sonnet-5-5",
    messages=[{"role": "user", "content": "How much credit do I have left?"}],
    tools=tools,
)
call = completion.choices[0].message.tool_calls[0]
print(call.function.name, call.function.arguments)   # arguments: a JSON string

Some models accept text only; what each model accepts is listed on its page in the catalogue.

Generate an image at an exact size

An image model is called on the same route with the same body. What changes is the model and the reply: the frame comes back as a data URL in choices[0].message.images[0].image_url.url. Geometry is a gateway field, so in Python it travels in extra_body:

import base64

completion = client.chat.completions.create(
    model="openai/gpt-5.4-image-2",
    messages=[{"role": "user", "content": "A lighthouse at dawn, wide shot"}],
    # gateway fields travel in extra_body; the library passes them through
    extra_body={"image_config": {"size": "1080x1920"}},
)

url = completion.choices[0].message.images[0]["image_url"]["url"]
with open("frame.png", "wb") as f:
    f.write(base64.b64decode(url.split(",", 1)[1]))   # strip the data: prefix
print(completion.usage.cost, "USD charged")

import { writeFile } from 'node:fs/promises'

const completion = await client.chat.completions.create({
  model: 'openai/gpt-5.4-image-2',
  messages: [{ role: 'user', content: 'A lighthouse at dawn, wide shot' }],
  // @ts-expect-error image_config is a gateway field
  image_config: { size: '1080x1920' },
})

const [image] = (completion.choices[0].message as any).images
await writeFile('frame.png', Buffer.from(image.image_url.url.split(',')[1], 'base64'))
console.log((completion.usage as any).cost, 'USD charged')

Scroll the table sideways for what you get.

Both fields are read at the top level of the body and inside image_config, so it does not matter which of the two your client fills in. Up to ten frames per request with n.

The image route: plain HTTP instead of images.generate

The library’s images.generate and images.edit helpers call /images/generations and /images/edits; the gateway does not serve those paths, so they come back 404. Its image route is POST /api/v1/images, and it covers both jobs: the prompt at the top level, reference frames in input_references when you want an edit, several variants with n, and the reply in data[].b64_json:

import base64
import requests

r = requests.post(
    "https://api.gtokens.io/api/v1/images",
    headers={"Authorization": "Bearer sk-sl-your-key"},
    json={
        "model": "openai/gpt-5.4-image-2",
        "prompt": "Replace the sky with a clear night",
        "size": "1080x1920",
        "n": 3,
        # leave input_references out and the frames are drawn from the prompt alone
        "input_references": [
            {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
        ],
    },
    timeout=600,
)
body = r.json()
for i, frame in enumerate(body["data"]):
    open(f"frame-{i}.png", "wb").write(base64.b64decode(frame["b64_json"]))
print(body["usage"]["cost"], "USD charged")

Handle errors by error_type

Failures come back as {"error": {"code", "message", "metadata": {"error_type"}}}. Branch on metadata.error_type rather than on the HTTP status: it is a closed vocabulary — authentication, payment_required, invalid_request, not_found, rate_limit_exceeded, provider_overloaded and a few more — and the specific reason sits next to it in metadata.reason.

import openai

try:
    completion = client.chat.completions.create(...)
except openai.APIStatusError as e:
    error = e.response.json().get("error", {})
    print(e.status_code, error.get("message"))
    print(error.get("metadata", {}).get("error_type"))   # branch on this
    print(e.response.headers.get("retry-after"))          # set on 429 and 503

Retry-After is always set on 429 and 503. A 402 means the balance or a spending ceiling cannot cover the request, and nothing is charged; a 400 with reason model_not_available means the model is listed but not served yet.