Living inside the Workers subrequest budget

The music page on this site is a set of live sections – now playing, recent tracks, three charts, a genre breakdown, stats, a listening heatmap, a rhythm clock – all fed by Last.fm. Its first version worked the way I would have written it anywhere: one server-side load awaited a Promise.all over every section’s endpoint, and each endpoint then fanned out into whatever Last.fm calls it needed. Top artists have no usable image in Last.fm’s API, so each artist row fetched the artist’s page as HTML, buffered it whole, and regexed out the og:image.

That version passed every check locally on Node and intermittently threw 500s in production. The stack trace pointed nowhere useful, because the cause was not in the code. It was the execution model. A Cloudflare Worker on the free plan gets 50 subrequests per invocation and 10 ms of CPU, and one page render was making on the order of 100 Last.fm calls – including a roughly 1 MB HTML fetch per artist – inside a single invocation.

A limit with a different shape

Every constrained environment I had worked in before constrained memory or time. Those are continuous budgets with continuous remedies: for memory, stop buffering and stream; for CPU, profile and trim the hot path. Both were live problems here – buffering those artist pages is what blew the 10 ms CPU budget first, fixed by reading a bounded 48 KB prefix up to </head> and cancelling the rest of the body.

A request count is not like that. A subrequest cannot be streamed and cannot be made cheaper; the only lever is to not make it. The fix is therefore not an optimisation of the same program but a different shape of program: cache across requests, defer work between requests, and get fan-out out of the invocation entirely. The limit also fails invisibly locally, because Node has no per-invocation request counter – nothing on a dev machine corresponds to hitting it.

Restructuring around a budget

The restructure came in three moves.

One endpoint per invocation. The page shell now server-renders with per-section skeletons and the browser does the fetching, so each section lands in its own Worker invocation with its own budget of 50. The load function hands SSR a promise that never settles – so the server pass renders every skeleton and ships immediately – and the browser pass resolves each section for real:

const section = <T>(u: string): Promise<T | null> =>
	browser
		? fetch(u, { cache: 'no-store' })
				.then((r) => r.json() as Promise<T>)
				.catch(() => null)
		: UNRESOLVED;

Per-item lookups memoised in KV. Last.fm only exposes artwork and artist tags one item at a time, so a top-18 artist chart used to cost roughly 18 to 36 outbound requests per render. Each kind of lookup now lives in one KV key holding a whole key-to-value map: a request pays one KV read, at most budget live lookups, and one KV write. The budget is an explicit part of the interface:

export type MemoOptions = {
	kv?: KVNamespace;
	/** Max live lookups per request. Keeps us well under the subrequest cap. */
	budget?: number;
	/** Max simultaneous lookups. Workers allow 6 connections awaiting headers
	 *  across the whole invocation, and several endpoints share it. */
	concurrency?: number;
	hitTtlMs?: number;
	missTtlMs?: number;
};

Whatever the budget did not cover comes back as pending, which shortens that response’s cache TTL, so a cold chart converges over the next few requests instead of trying to do everything at once.

Budgets tuned to converge. Once artwork moved to Deezer and a lookup cost one request, the budgets were raised – artist tags from 4 to 10 per request, artwork to 12 per chart – so a cold 24-row mosaic fills over about two views. The endpoint does the arithmetic where the budget is declared:

// Artists whose tags are fetched live per request; the rest come from the KV memo.
// One `artist.getTopTags` each, so 10 live lookups = 15 subrequests worst case,
// and a cold chart of 20 artists converges over two responses.
const TAG_BUDGET = 10;

The twist worth sitting on: with the platform’s cap comfortably handled, the binding limit became the provider’s own quota. Deezer allows 50 requests per 5 seconds per IP and Workers egress is shared, so a burst of concurrent section loads is what actually blanked rows – verified by wiping the memo and firing 18 concurrent chart requests, which logged quota rejections and left 109 of 216 rows artless. Request count stayed the whole ballgame; only the owner of the number changed.

The rules are written down in PLAN.md now. The next page on this site should start inside the budget rather than be pushed into it.