You can get a model's answer in one shot, or you can get it in pieces as it is generated. The second option is streaming over server-sent events. Here is when it pays off and when it just adds code for no real benefit.
What actually happens
On a plain request the server holds the connection open until the whole answer is ready, then sends it in one block. With streaming the same server sends events as tokens become available: the client opens an HTTP connection with Accept: text/event-stream and reads a stream of data: {...} lines separated by blank lines.
Anthropic's format differs from OpenAI's. Anthropic sends named events: message_start, content_block_delta, message_stop, and so on, each carrying its own type. OpenAI's stream is simpler: the same data: lines carrying a text delta, ending with data: [DONE]. Both formats work through Claudexia, and streaming is toggled with the same stream: true flag you would use against the original SDKs.
Where streaming genuinely helps
In a chat interface the gap between "a spinner for five seconds" and "text starts appearing after three hundred milliseconds" is felt strongly, even when the total time is the same. People start reading sooner and read the system as responsive.
The same applies to long answers. If the model writes a couple thousand tokens, the reader gets through the beginning while the end is still being generated. A static screen the whole time feels slow even when it technically is not.
Streaming also matters if you want to let a user cancel mid-generation. They see the answer going the wrong way, hit stop, the connection closes, and you are not billed for tokens that were never sent.
Where you can skip it
If the answer is short: a classification, a single extracted field, a plain yes or no, the reader will not get through it in pieces anyway, and the extra code buys nothing.
Server-side processing with no interface, where the result goes straight into a database or another system, does not benefit either. That path needs a complete, validated answer, and a partial text would just get buffered whole before use, which defeats the point of streaming in the first place.
Batch processing dozens or hundreds of requests almost always stays simpler without streaming: less state to track, simpler retries, simpler token accounting.
Handling partial chunks
The trap: a network chunk is not the same thing as a meaningful unit of text. A single chunk can cut a word or a JSON object in half, especially when you are streaming tool use with partial function arguments at the same time.
What works: accumulate text deltas into a buffer and do not try to parse JSON until a block-completion event arrives. Python example with the Anthropic SDK:
import anthropic
client = anthropic.Anthropic(
api_key="your-claudexia-key",
base_url="https://api.claudexia.tech/v1",
)
with client.messages.stream(
model="claude-sonnet-4.6",
max_tokens=1024,
messages=[{"role": "user", "content": "Write a short poem about autumn"}],
) as stream:
for text in stream.text_stream:
print(text, end="", flush=True)
final = stream.get_final_message()
The SDK assembles the deltas into full text and hands you the complete message with exact token counts at the end. If you are working with raw SSE without an SDK, replicate the same logic by hand: hold lines in a buffer and only parse JSON at an event boundary.
Disconnects and reconnecting
A long stream can drop: the client goes to the background, a mobile connection blips, a proxy closes the connection on a timeout. LLM APIs have no built-in "resume from where it stopped" mechanism, unlike file uploads with resumable transfer.
The pattern that works: treat a disconnect as an ordinary network error, keep whatever text you already received, and decide separately what to do with it, show the partial answer marked as interrupted, or resend the request. For a resend, it is worth trimming max_tokens down to what you actually still need so a retry does not double your bill for nothing.
If disconnects cluster specifically on long generations, the cause is often not the client but an intermediate proxy or load balancer with a short connection timeout. Raise the client-side timeout and check that nothing between your app and Claudexia is cutting long-lived connections on its own.
In short
SSE pays off where a human is waiting and reading: chat, long answers, the ability to cancel mid-stream. In server-side processing with no interface it just adds complexity. Buffer the deltas, parse JSON only at event boundaries, and treat a dropped connection as a plain network error that preserves whatever text already arrived.