Playwright CPU throttling for React render performance: the harness behind our streaming numbers, and why milliseconds mislead
Measure a streaming React UI with Playwright and CDP. At 4x CPU throttling, script time rose a third while renders per answer fell from 76 to 23.
The numbers in react-markdown streaming performance came from a small harness: Playwright, Chrome's DevTools Protocol, and a fake chat backend. Its most useful output was not a millisecond figure. With Playwright CPU throttling at 4x, script time per answer rose by about a third over a conversation while the number of renders per answer fell from 76 to 23. The page was not getting faster. It was falling behind and rendering in bigger jumps, and a timing-only benchmark would have reported a mild slowdown. This post is the harness, with fresh runs from today at four throttle levels.
A mock stream in front of the real build
Measuring against the real model gives a different answer length, token rate and latency every run, and costs money each time. So the harness puts a 58-line Node proxy in front of the production build (next start). It forwards every request, except POST /api/chat, which it answers with the kit's own server-sent events at a fixed pace:
// trimmed from the proxy: `body` is the answer text, `usage` a fixed token count
if (req.method === "POST" && req.url === "/api/chat") {
res.writeHead(200, { "content-type": "text/event-stream", "cache-control": "no-cache" });
let i = 0;
const tick = () => {
if (i >= body.length) {
res.end(`data: ${JSON.stringify({ type: "done", stopReason: "end_turn", model: "claude-haiku-4-5", usage, costUsd: 0.0016 })}\n\n`);
return;
}
res.write(`data: ${JSON.stringify({ type: "text", text: body.slice(i, i + CHUNK) })}\n\n`);
i += CHUNK;
setTimeout(tick, GAP);
};
return tick();
}
Every answer is the same 1,017-character markdown reply, with a heading, a numbered list, a code block, bullets and a link, sent as 85 deltas of 12 characters, 20 ms apart. The page under test is the real chat page from the real build, and only the network behind it is fake.
Playwright CPU throttling and the four numbers per answer
The driver opens the page, turns on throttling through a CDP session, and then asks 11 questions in a row, recording four numbers for each answer:
const cdp = await page.context().newCDPSession(page);
await cdp.send("Performance.enable");
await page.goto("http://localhost:3312/chat", { waitUntil: "networkidle" });
await cdp.send("Emulation.setCPUThrottlingRate", { rate: 4 });
await page.evaluate(() => {
window.__commits = 0;
new MutationObserver(() => { window.__commits++; })
.observe(document.querySelector("main"), { subtree: true, childList: true, characterData: true });
window.__long = [];
new PerformanceObserver((list) => { for (const e of list.getEntries()) window.__long.push(e.duration); })
.observe({ type: "longtask" });
});
const metrics = async () =>
Object.fromEntries((await cdp.send("Performance.getMetrics")).metrics.map((m) => [m.name, m.value]));
- Script time: the difference in
ScriptDurationfromPerformance.getMetricsbefore and after each answer. - Main-thread time: the same for
TaskDuration, which also counts layout, style and paint. - Long tasks: entries over 50 ms from a
longtaskPerformanceObserver. - DOM updates: MutationObserver callbacks on
<main>, the closest thing to "renders the user saw" that the page can count.
The throttle is set after the page loads, so the numbers describe the conversation rather than the initial load. To pin time on one library, a second script wraps an answer in Profiler.start and Profiler.stop, adds up self time per sample, and groups it by the script URL each function came from. To find which build chunk holds the library, grep .next/static/chunks for a string only it contains: react-markdown's parser is the one chunk mentioning micromark.
What the throttle levels show
Today's runs, on the kit's shipped chat page:
- 1x, three runs: answer 2 got 88 to 89 DOM updates and 207 to 246 ms of script. Answer 11 got 78 to 82 updates and 803 to 840 ms. No long tasks.
- 2x: answer 2 got 87 updates and 549 ms. Answer 11 got 50 updates and 969 ms, with 2 long tasks (the longest 60 ms).
- 4x: answer 2 got 76 updates and 807 ms. Answer 11 got 23 updates and 1,094 ms, with 8 long tasks (the longest 161 ms).
- 6x: answer 2 got 35 updates and 909 ms. Answer 11 got 18 updates and 1,039 ms, with 15 long tasks.
Read by script time alone, 4x and 6x look about the same as each other, roughly a second per answer. Read by updates, the stream went from 85 deltas shown nearly one by one to 18 visible jumps. Once the main thread cannot keep up, network chunks queue, each reader.read() returns several frames, and React folds them into one render, the batching described in rendering a streaming LLM response in React. Script time stops growing because the page has stopped doing all the work. The user sees choppier text.
The three 1x runs agree within about 5% on answer 11, which is the spread to beat before calling a change an improvement.
Things that bite
- Answer 1 includes warm-up. The first answer runs code the JavaScript engine has not optimized yet. Compare from answer 2 onward.
- Throttling is relative to the host. 4x slows the machine you run on, not a phone. On November 10, on a different container, the same page and build gave 88 updates on answer 11 unthrottled. Today it gave 78 to 82. Compare variants in one session on one machine, and keep the absolute numbers out of claims.
- Measure the production build.
next devruns the development build of React, a different and slower program, so its numbers describe that instead. - DOM updates are not React commits. A MutationObserver callback fires once per batch of DOM changes, and a commit that changes nothing in the DOM does not show up. For a streaming transcript that is the right thing to count, since it is what reaches the screen.
Shipwright, the Next.js and Claude starter kit this blog documents, streams Claude's answers into a markdown chat UI with per-request cost metering. You can watch it stream in the live demo.