SSE vs WebSockets for LLM streaming: why one-way token streams want SSE

When you first stream model output to a browser, WebSockets feel like the "real-time" choice. For a chat feature they are usually the wrong one. Look at the actual traffic shape: the client sends one request, then the server sends a long stream of tokens back, then it is over. That is one-way after the request — exactly what Server-Sent Events were built for — and choosing a socket instead buys you infrastructure you then have to run.

The traffic shape decides it

A chat turn is request in, stream out. Nothing needs to flow from the browser to the server mid-generation except "stop," and stopping is just closing the connection. SSE is a plain HTTP response with a text/event-stream body, so a Next.js route handler returns it like any other Response. The kit's chat route does exactly that: the stream helper emits data: <json> frames, each terminated by a blank line, carrying typed events — text deltas, a final done with usage and cost, or an error.

What SSE gets you for free

• It is just HTTP. No upgrade handshake, no separate protocol. Auth cookies, CORS, logging, and rate limiting all work the way they already do for your API routes.
• It runs where your routes run. A standard App Router route handler returns a Response; it has no WebSocket upgrade path. A socket means a separate long-running server or a managed realtime service. SSE needs neither.
• Proxies cope — with the right headers. An SSE response passes through CDNs and reverse proxies as long as you tell them not to buffer it; that is its own set of gotchas, covered in why your SSE stream arrives as one blob.
• Stop means close. When the user leaves or hits stop, the connection closes and the stream's cancel hook aborts the upstream model call — no message protocol required. That is the money-saving half of streaming, in aborting on disconnect.

The catch: EventSource cannot POST

Here is the part most SSE tutorials skip. The browser's built-in EventSource only makes GET requests with no body — and a chat request carries the whole message history. So you do not use EventSource; you POST with fetch and read the response body as a stream yourself. The kit's chat client does it like this:

const res = await fetch("/api/chat", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ messages: history }),
});
const reader = res.body.getReader();
const decoder = new TextDecoder();
let buffer = "";

for (;;) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  const frames = buffer.split("\n\n"); // SSE frames end with a blank line
  buffer = frames.pop() ?? "";
  for (const frame of frames) {
    const data = frame.split("\n").filter((l) => l.startsWith("data: ")).map((l) => l.slice(6)).join("");
    if (data) handle(JSON.parse(data)); // text | done | error
  }
}

The trade-off is that you give up what EventSource does automatically: reconnection and resuming from Last-Event-ID. For a chat turn that is usually fine — a dropped stream is a failed turn you can retry — but it is your code's job now, not the browser's. Note the buffer: a network chunk can end mid-frame, so you keep the trailing partial and only parse complete frames.

When a WebSocket genuinely wins

Reach for a socket when traffic really is two-way and continuous: live voice, collaborative editing and presence, multiplayer state, or a server that needs to push to an idle client with no request outstanding. If the browser has to keep talking while the server is talking, SSE is the wrong shape. For "send a prompt, stream an answer," it is the right one.

Things that bite

• Keep-alive on long silences. A model that thinks for a while before its first token can leave the connection quiet long enough for an idle timeout to close it; send a comment frame periodically if your generations have long gaps.
• One stream per turn. Open a stream for the request, close it when done arrives. Long-lived multiplexed SSE connections are where you start rebuilding a socket badly.
• Handle the error event, not just exceptions. A failure mid-stream arrives as a frame on a successful HTTP response; if the client only checks the status code, it will miss it.

Shipwright — the Next.js and Claude starter kit this blog documents — streams over SSE end to end: the route helper, the typed event frames, and the fetch-and-read client above. Watch the frames arrive in the live demo.