Rendering a streaming LLM response in React: the state updates that do not lose text

The network part of streaming a model's answer is the easy part. The bugs users actually see are in React: the answer comes out with words missing, an earlier reply gets overwritten by the new one, or a failed request leaves an empty grey bubble sitting in the transcript forever. All three come from how the component updates state as tokens arrive. Here is how the kit's demo chat handles it, and the one place it is still inconsistent.

The reader loop — fetching with POST, splitting the body into server-sent-event frames — is covered in SSE vs WebSockets for LLM streaming. This post is about what happens to each frame once you have it.

Show the turn before the first token arrives

When the user sends a message, the component appends two entries at once: the user's turn, and an empty assistant message that will fill in as tokens stream. The request body gets the history without that placeholder:

const history: ChatMessage[] = [...messages, { role: "user", content }];
setMessages([...history, { role: "assistant", content: "" }]);
setInput("");
setBusy(true);
setError(null);

const res = await fetch("/api/chat", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ messages: history }),
});

The empty bubble renders as "…" while busy is true, so the user sees their message land and a response begin immediately — instead of a pause while the model thinks. Sending history rather than the state that includes the placeholder keeps an empty assistant turn out of the request.

Append each delta with a functional update

This is the bug that drops text. The messages variable inside the send handler is a snapshot from when the handler started. Build each update from it — setMessages([...messages, ...]) — and every token overwrites the last, because each one starts from the same stale snapshot. Use the functional form, which receives the current state, and replace the last message immutably:

if (parsed.type === "text") {
  setMessages((current) => {
    const next = [...current];
    const last = next[next.length - 1];
    next[next.length - 1] = { ...last, content: last.content + parsed.text };
    return next;
  });
}

Copying the array and the last message (rather than mutating last.content) matters too: React compares by reference, and a mutated object can skip the re-render or show stale text.

You do not need to throttle renders

It looks like this re-renders the whole transcript once per token. In practice it is once per network chunk: React 18 and later batch state updates made in the same synchronous run, and one reader.read() often delivers several frames that are all applied in the same loop iteration. For typical model output speeds that is plenty smooth. Reach for manual throttling only if you measure a problem — for example, very long transcripts that re-render expensively on every chunk.

One cleanup path for both kinds of failure

A streaming request can fail two ways. It can fail before streaming — a 402 when the quota is used up, a 429 from the rate limiter, a network error — which throws and lands in catch. Or it can fail during streaming, which arrives as an error frame inside a successful 200 response. The kit's catch cleans up properly, removing the placeholder if no text arrived:

} catch (err) {
  setError(err instanceof Error ? err.message : "Something went wrong.");
  setMessages((current) =>
    current[current.length - 1]?.content === "" ? current.slice(0, -1) : current,
  );
} finally {
  setBusy(false);
}

The error-frame path, though, only sets the error message. If a stream fails before its first token, the empty placeholder stays — and once busy goes false it renders as an empty grey bubble. The fix is to route both failures through the same cleanup: on an error frame, run the same "remove the last message if it is still empty" update. Keep any partial text that did arrive; a half answer with an error underneath is more useful than nothing.

Things that bite

• Disable sending while busy. A second submit mid-stream starts a second placeholder and interleaves two streams into one transcript. The kit disables the button while busy is true.
• Index keys are fine — here. The transcript only grows at the end or loses its last item, so array indexes stay stable as keys. If you add editing or deleting messages in the middle, switch to stable ids.
• Scroll with care. Calling scrollIntoView on every delta keeps the newest text visible, but it also yanks the page down if the user has scrolled up to reread. Scroll only when they are already near the bottom.
• Show limits as messages, not crashes. A 402 or 429 is an expected outcome; surface the server's message ("You've used all 25 requests on the Free plan this month.") rather than a generic failure, so the user knows what to do next.

Shipwright — the Next.js and Claude starter kit this blog documents — ships this chat client wired to the streaming route, including the per-request cost panel fed by the stream's final done frame. Try it in the live demo.