---
url: https://docs.freetheai.org/concepts/streaming.md
description: >-
  Stream FreeTheAI replies as server-sent events on Chat Completions, Responses,
  and Anthropic Messages, read token usage from the final chunk, and handle a
  broken stream.
---

# Streaming

Send `"stream": true` and the reply arrives as server-sent events (SSE) while the model writes it. Every free model streams on all three endpoints.

Turn streaming on for long replies and for models that think first. Chat apps such as JanitorAI give up on a reply that sends nothing for a while ("No response from bot"), and very long replies without streaming can time out.

## Chat Completions

Each event is a `data:` line with a `chat.completion.chunk`. The text is in `choices[0].delta.content`, and the stream ends with `data: [DONE]`.

::: code-group

```bash [curl]
curl -N https://api.freetheai.org/v1/chat/completions \
  -H "Authorization: Bearer $FTAI_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "fta/zai/glm-5.3",
    "stream": true,
    "messages": [{"role": "user", "content": "Count from 1 to 5."}]
  }'
```

```python [Python]
import os
from openai import OpenAI

client = OpenAI(base_url="https://api.freetheai.org/v1", api_key=os.environ["FTAI_KEY"])

stream = client.chat.completions.create(
    model="fta/zai/glm-5.3",
    messages=[{"role": "user", "content": "Count from 1 to 5."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print("\n", chunk.usage)
```

```ts [TypeScript]
import OpenAI from 'openai'

const client = new OpenAI({ baseURL: 'https://api.freetheai.org/v1', apiKey: process.env.FTAI_KEY })

const stream = await client.chat.completions.create({
  model: 'fta/zai/glm-5.3',
  messages: [{ role: 'user', content: 'Count from 1 to 5.' }],
  stream: true,
})
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? '')
  if (chunk.usage) console.log('\n', chunk.usage)
}
```

:::

The raw stream looks like this:

```text
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","model":"fta/zai/glm-5.3","choices":[{"index":0,"delta":{"role":"assistant","content":"1"}}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","model":"fta/zai/glm-5.3","choices":[{"index":0,"delta":{"content":", 2, 3, 4, 5"},"finish_reason":"stop"}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","model":"fta/zai/glm-5.3","choices":[],"usage":{"prompt_tokens":15,"completion_tokens":13,"total_tokens":28}}

data: [DONE]
```

### Token usage

FreeTheAI includes token usage in the final chunk of every stream, as an ordinary chunk with a `usage` field and an empty `choices` list. You can also send `"stream_options": {"include_usage": true}` as in the OpenAI API; the result is the same.

### Reasoning in the stream

Models that think before answering stream their reasoning in `delta.reasoning_content`, separate from `delta.content`. Show it or skip it; it is not part of the answer. See [Thinking and reasoning](/concepts/reasoning).

## Responses

Responses streams typed events. The text arrives in `response.output_text.delta` events, and `response.completed` carries the whole response with its usage.

```python
stream = client.responses.create(
    model="fta/zai/glm-5.3",
    input="Count from 1 to 5.",
    stream=True,
)
for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
    elif event.type == "response.completed":
        print("\n", event.response.usage)
```

## Anthropic Messages

Messages streams Anthropic's events (`message_start`, `content_block_delta`, `message_delta`, `message_stop`). The Anthropic SDK's helpers work as usual:

```python
import os
import anthropic

client = anthropic.Anthropic(base_url="https://api.freetheai.org", api_key=os.environ["FTAI_KEY"])

with client.messages.stream(
    model="fta/zai/glm-5.3",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Count from 1 to 5."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
```

## When a stream breaks

If the model fails after the stream started, the stream ends with an `error` event instead of `[DONE]`:

```text
event: error
data: {"error":{"message":"stream interrupted. Error id: freetheai-1a2b3c4d5e6f","type":"gateway_stream_error","error_id":"freetheai-1a2b3c4d5e6f"}}
```

Regenerate the reply. A cut-off that delivered almost nothing does not count against your limits. If it keeps happening, open a ticket with the error ID. Errors before the stream starts (a missing check-in, a busy model) come back as a normal JSON error with an HTTP status, see [Errors](/errors).