Streaming

Send "stream": true and the reply arrives as server-sent events (SSE) while the model writes it. Every free model streams on all three endpoints.

Turn streaming on for long replies and for models that think first. Chat apps such as JanitorAI give up on a reply that sends nothing for a while ("No response from bot"), and very long replies without streaming can time out.

Chat Completions

Each event is a data: line with a chat.completion.chunk. The text is in choices[0].delta.content, and the stream ends with data: [DONE].

curl

curl -N https://api.freetheai.org/v1/chat/completions \
  -H "Authorization: Bearer $FTAI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fta/zai/glm-5.3",
    "stream": true,
    "messages": [{"role": "user", "content": "Count from 1 to 5."}]
  }'

Python

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.freetheai.org/v1", api_key=os.environ["FTAI_KEY"])

stream = client.chat.completions.create(
    model="fta/zai/glm-5.3",
    messages=[{"role": "user", "content": "Count from 1 to 5."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)

TypeScript

import OpenAI from 'openai'

const client = new OpenAI({ baseURL: 'https://api.freetheai.org/v1', apiKey: process.env.FTAI_KEY })

const stream = await client.chat.completions.create({
  model: 'fta/zai/glm-5.3',
  messages: [{ role: 'user', content: 'Count from 1 to 5.' }],
  stream: true,
})
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '')
  if (chunk.usage) console.log('\n', chunk.usage)
}

The raw stream looks like this:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","model":"fta/zai/glm-5.3","choices":[{"index":0,"delta":{"role":"assistant","content":"1"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","model":"fta/zai/glm-5.3","choices":[{"index":0,"delta":{"content":", 2, 3, 4, 5"},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","model":"fta/zai/glm-5.3","choices":[],"usage":{"prompt_tokens":15,"completion_tokens":13,"total_tokens":28}}

data: [DONE]

Token usage

FreeTheAI includes token usage in the final chunk of every stream, as an ordinary chunk with a usage field and an empty choices list. You can also send "stream_options": {"include_usage": true} as in the OpenAI API; the result is the same.

Reasoning in the stream

Models that think before answering stream their reasoning in delta.reasoning_content, separate from delta.content. Show it or skip it; it is not part of the answer. See Thinking and reasoning.

Responses

Responses streams typed events. The text arrives in response.output_text.delta events, and response.completed carries the whole response with its usage.

stream = client.responses.create(
    model="fta/zai/glm-5.3",
    input="Count from 1 to 5.",
    stream=True,
)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
    elif event.type == "response.completed":
        print("\n", event.response.usage)

Anthropic Messages

Messages streams Anthropic's events (message_start, content_block_delta, message_delta, message_stop). The Anthropic SDK's helpers work as usual:

import os
import anthropic

client = anthropic.Anthropic(base_url="https://api.freetheai.org", api_key=os.environ["FTAI_KEY"])

with client.messages.stream(
    model="fta/zai/glm-5.3",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Count from 1 to 5."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

When a stream breaks

If the model fails after the stream started, the stream ends with an error event instead of [DONE]:

event: error
data: {"error":{"message":"stream interrupted. Error id: freetheai-1a2b3c4d5e6f","type":"gateway_stream_error","error_id":"freetheai-1a2b3c4d5e6f"}}

Regenerate the reply. A cut-off that delivered almost nothing does not count against your limits. If it keeps happening, open a ticket with the error ID. Errors before the stream starts (a missing check-in, a busy model) come back as a normal JSON error with an HTTP status, see Errors.