Models

OpenAI SDK

The official openai packages talk to any endpoint that implements the OpenAI HTTP API. Change the base URL and the key, and everything else — retries, streaming, pagination, typed errors — works as documented.

Install and configure

A Grafilab API key. In the console open API (https://app.grafilab.ai/api) and click Generate New Key — see API Keys.

Keep the key out of your source. The examples below read it from the environment.

Python
TypeScript

pip install openai

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["GRAFILAB_API_KEY"],
    base_url="https://llm.grafilab.ai/v1",
)

npm install openai

import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.GRAFILAB_API_KEY,
  baseURL: "https://llm.grafilab.ai/v1",
});

If you would rather not touch the constructor — handy when you are pointing an existing script at Grafilab — set OPENAI_BASE_URL to https://llm.grafilab.ai/v1 and OPENAI_API_KEY to your Grafilab key, and construct the client with no arguments. The base URL must end in /v1; the SDK appends /chat/completions and the rest.

Pick a model id

The model catalog is live, so list it rather than copying ids from a page. Every entry’s id is the exact string to use in any tool or SDK.

curl https://llm.grafilab.ai/v1/models \
  -H "Authorization: Bearer $GRAFILAB_API_KEY" | jq -r '.data[].id'

from openai import OpenAI

client = OpenAI(api_key="sk-grafilab-...", base_url="https://llm.grafilab.ai/v1")
for model in client.models.list():
    print(model.id)

import OpenAI from "openai";

const client = new OpenAI({ apiKey: "sk-grafilab-...", baseURL: "https://llm.grafilab.ai/v1" });
for await (const model of client.models.list()) {
  console.log(model.id);
}

The same ids appear as cards in the Playground, with context size and prices.

response = client.chat.completions.create(
    model="grafilab/qwen3.6-flash",
    messages=[
        {"role": "system", "content": "You are terse."},
        {"role": "user", "content": "Why is the sky blue?"},
    ],
)

print(response.choices[0].message.content)
print(response.usage)

const response = await client.chat.completions.create({
  model: "grafilab/qwen3.6-flash",
  messages: [
    { role: "system", content: "You are terse." },
    { role: "user", content: "Why is the sky blue?" },
  ],
});

console.log(response.choices[0].message.content);
console.log(response.usage);

Fields other than model, messages, stream, and stream_options are passed straight through to the model as parameters. That means temperature, top_p, max_tokens, tools, and response_format work wherever the underlying model supports them — and an unknown parameter is ignored or rejected by the model rather than by Grafilab. Text is billed after delivery, in one transaction per request.

Streaming

Ask for usage explicitly. Without stream_options, a streamed response carries no token counts at all.

Python
TypeScript

stream = client.chat.completions.create(
    model="grafilab/qwen3.6-flash",
    messages=[{"role": "user", "content": "Count to five."}],
    stream=True,
    stream_options={"include_usage": True},
)

for chunk in stream:
    if chunk.usage:
        print("\nusage:", chunk.usage)
    elif chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")

const stream = await client.chat.completions.create({
  model: "grafilab/qwen3.6-flash",
  messages: [{ role: "user", content: "Count to five." }],
  stream: true,
  stream_options: { include_usage: true },
});

for await (const chunk of stream) {
  if (chunk.usage) {
    console.log("\nusage:", chunk.usage);
  } else if (chunk.choices[0]?.delta?.content) {
    process.stdout.write(chunk.choices[0].delta.content);
  }
}

The usage-carrying chunk arrives last, immediately before the data: [DONE] terminator, and its choices array is empty.

Reasoning models

Reasoning traces come back on the message under both reasoning and reasoning_content, with identical content. Models that emit thinking as inline tags are normalised into the same two fields, so you never have to parse them out of the visible text. Neither field is part of the OpenAI type definitions — read it off the raw object.

List models

Python
TypeScript

for model in client.models.list():
    print(model.id, getattr(model, "region", None), getattr(model, "tags", None))

for await (const model of client.models.list()) {
  console.log(model.id, (model as any).region, (model as any).tags);
}

Every id is a Grafilab alias and the exact string to send as model. Aliases match case-insensitively and may contain /, and the response echoes the canonical spelling, which can differ from what you sent. Alongside the standard OpenAI fields, each entry carries two Grafilab extensions: region and tags.

Handle errors

Both SDKs raise a typed exception per status code. Grafilab uses two of them in ways that will surprise you if you assume OpenAI semantics.

Exception	Status	What it means on Grafilab
`AuthenticationError`	`401`	The key is unknown, revoked, or malformed. Also raised when a per-key IP allowlist excludes the caller
`PermissionDeniedError`	`403`	`billing_currency_unset` — your account has no billing currency. Set your country on your profile
`NotFoundError`	`404`	`model_not_found`. The alias is wrong, or the model exists but is not priced in your currency
`RateLimitError`	`429`	Insufficient balance, not throughput. Top up rather than backing off
`APIStatusError`	`500`	An infrastructure failure. Gateway errors are returned as `500` and never `502` or `504`

A 500 is safe to retry unless the response carries x-should-retry: false, which marks work that was already done and billed. Every response carries an X-Request-Id header (req_ followed by 24 hex characters) — quote it in any support request.

Python
TypeScript

import openai

try:
    raw = client.chat.completions.with_raw_response.create(
        model="grafilab/qwen3.6-flash",
        messages=[{"role": "user", "content": "Hello"}],
    )
    print(raw.headers.get("x-request-id"))
    completion = raw.parse()
except openai.RateLimitError:
    print("Top up your Grafilab credit balance.")
except openai.APIStatusError as err:
    print(err.status_code, err.response.headers.get("x-should-retry"), err.response.text)

try {
  const { data, response } = await client.chat.completions
    .create({
      model: "grafilab/qwen3.6-flash",
      messages: [{ role: "user", content: "Hello" }],
    })
    .withResponse();
  console.log(response.headers.get("x-request-id"), data.choices[0].message.content);
} catch (err) {
  if (err instanceof OpenAI.RateLimitError) {
    console.log("Top up your Grafilab credit balance.");
  } else if (err instanceof OpenAI.APIError) {
    console.log(err.status, err.headers?.["x-should-retry"], err.message);
  }
}

Embeddings and images

Embeddings

encoding_format accepts float (a JSON array) or base64 (little-endian float32, roughly a third the bytes on the wire). Like text, embeddings are billed after delivery.

Python
TypeScript

result = client.embeddings.create(
    model="<embedding-model-id>",
    input=["the first document", "the second document"],
    encoding_format="float",
)
print(len(result.data[0].embedding))

const result = await client.embeddings.create({
  model: "<embedding-model-id>",
  input: ["the first document", "the second document"],
  encoding_format: "float",
});
console.log(result.data[0].embedding.length);

Images

images.generate is synchronous: it generates first and charges in the same transaction, so a failure costs nothing. response_format is url or b64_json, and n accepts 1 to 10.

Python
TypeScript

image = client.images.generate(
    model="<image-model-id>",
    prompt="a lighthouse in fog, long exposure",
    n=1,
    response_format="url",
)
print(image.data[0].url)

const image = await client.images.generate({
  model: "<image-model-id>",
  prompt: "a lighthouse in fog, long exposure",
  n: 1,
  response_format: "url",
});
console.log(image.data[0].url);

images.edit uploads multipart form data and needs an edit-capable model — anything else returns 400 with code edit_unsupported. Each file is capped at 20 MB and a request at 10 files.

Python
TypeScript

edited = client.images.edit(
    model="<edit-capable-model-id>",
    image=open("source.png", "rb"),
    prompt="replace the sky with a clear night sky",
)

import fs from "node:fs";
import { toFile } from "openai";

const edited = await client.images.edit({
  model: "<edit-capable-model-id>",
  image: await toFile(fs.createReadStream("source.png")),
  prompt: "replace the sky with a clear night sky",
});

Video

Video is asynchronous: create a job, poll it, then download the result. A job moves through queued → in_progress → completed or failed. Polling is what advances the state, so poll until you reach a terminal status.

Video is charged at creation, inside the submission transaction, and refunded automatically if the job later fails. Completed results expire seven days after completion, so download what you want to keep.

Python
TypeScript

import time

video = client.videos.create(model="<video-model-id>", prompt="a paper boat crossing a puddle")

while video.status in ("queued", "in_progress"):
    time.sleep(5)
    video = client.videos.retrieve(video.id)

if video.status == "failed":
    raise RuntimeError(video.error)

content = client.videos.download_content(video.id)
content.write_to_file("out.mp4")

let video = await client.videos.create({
  model: "<video-model-id>",
  prompt: "a paper boat crossing a puddle",
});

while (video.status === "queued" || video.status === "in_progress") {
  await new Promise((r) => setTimeout(r, 5000));
  video = await client.videos.retrieve(video.id);
}

if (video.status === "failed") throw new Error(String(video.error));

const content = await client.videos.downloadContent(video.id);
// content is a Response — stream or buffer its body to a file.

GET /v1/videos/{video_id}/content answers with a 302 to a publicly readable MP4 URL that needs no key of its own — convenient for handing a link to a browser, and a reason not to log it. Grafilab implements creation, listing, retrieval, and content download; the SDK’s remix, edit, extend, and delete methods have no counterpart here, and remixed_from_video_id is always null.

Next steps

Quickstart: your first API callThe shortest path from a key to a first response.

API referenceThe full OpenAPI spec for every endpoint on this page.

ClineThe same endpoint, driven by a coding agent.