Streaming

Set "stream": true on either streaming-capable endpoint and read text/event-stream. The two endpoints do not share a frame format, read the right section below for whichever one you’re calling.

/v1/chat/completions

This is a byte-honest relay of whatever the upstream model server (Ollama or OpenRouter) sends: each data: ... line is forwarded as-is, including the terminal sentinel.

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","model":"ministral-3:3b","choices":[{"delta":{"role":"assistant"},"finish_reason":null}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","model":"ministral-3:3b","choices":[{"delta":{"content":"Nairobi"},"finish_reason":null}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","model":"ministral-3:3b","choices":[{"delta":{"content":" traffic"},"finish_reason":null}]}

data: {"id":"chatcmpl-1","object":"chat.completion.chunk","model":"ministral-3:3b","choices":[{"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":18,"completion_tokens":24,"total_tokens":42}}

data: [DONE]

A final usage-bearing chunk is requested on every streaming call (stream_options.include_usage), so the frame just before [DONE] usually carries token counts. If the upstream fails mid-stream, one synthetic error frame replaces the rest of the stream and there is no [DONE] after it:

data: {"error":{"message":"upstream request timed out","type":"server_error","code":"upstream_error"}}

/v1/threads/{id}/continue

This endpoint keeps conversation state on the gateway, and its stream is a small custom protocol rather than an OpenAI relay, there is no [DONE] sentinel; the stream simply ends after a done or error frame.

data: {"type":"delta","text":"The"}

data: {"type":"delta","text":" Act"}

data: {"type":"delta","text":" requires..."}

data: {"type":"done","message_id":"c4b1...-uuid","finish_reason":"stop","appended":"The Act requires..."}

Calling this endpoint with "stream": false does not switch to a plain JSON response shape, it drains the same frame generator internally and returns the last frame’s body directly, so a non-streaming continuation response is the "type": "done" (or "type": "error") object above, not a chat.completion object.

Next: rate limits and token budgets that apply to both endpoints.