Randomify

A shuffle button for the entire recorded catalog: tempered hierarchical sampling over a MusicBrainz-derived corpus, linked across eight streaming services

Screenshot of randomify showing a song card with cover art, playback controls, and buttons linking to six streaming services
One spin on randomify.net. The card shows the drawn recording with its cover art, a playing 30-second preview, and buttons that open the track on each streaming service.

Randomify is a shuffle button pointed at everything ever recorded. Each press draws one song from a corpus built out of MusicBrainz, starts a 30-second preview, and shows a row of buttons that open the same track on Spotify, Apple Music, YouTube Music, Tidal, Deezer, Amazon Music, and, when an exact link exists, Pandora and Bandcamp. There are no recommendations, no play counts, and no tuning knobs. The interesting problems are all underneath: what "random" should mean over a catalog where popularity follows a long tail, how to draw a song in one database round trip, and how to turn a MusicBrainz recording into working links on eight services that mostly do not want to be linked to.

The system is three parts. A SvelteKit frontend deploys to Cloudflare Pages. The sampler is a Cloudflare Worker that reaches a Neon Postgres corpus through Hyperdrive. And the corpus itself is built offline by a pipeline that runs as launchd jobs on a Mac.

From dump to backlog

A weekly job downloads the latest MusicBrainz full export, a compressed dump that unpacks to about 20 GB of header-less TSV files, and extracts seventeen tables: recordings, artists, releases, mediums, tracks, ISRCs, URL relationships, and the genre and year tables from the derived dump. DuckDB loads them in memory and joins them into normalized recordings in one SQL pass: the earliest dated release per recording, the top three tag-voted genres per release group, artist country, release language, and any exact streaming URLs MusicBrainz already knows via its URL relationships, mapped from hostname to platform.

The candidate set is filtered to recordings that carry an ISRC, the recording industry's per-track identifier. That is the natural "probably streamable somewhere" set, and it is far smaller than the full catalog of roughly 30 million recordings: about 6 million candidates. They land in a recording_backlog table in Neon with a priority order computed to make partial progress useful: ISRC-bearing recordings first, then round-robin across artists so early resolution spreads over many catalogs instead of draining one, then round-robin across decades so the resolved set never collapses onto the newest releases. A resolved_at cursor makes the whole thing idempotent; a skipped or repeated run causes drift, never corruption.

Tempered sampling

A long tail distribution with a tall head and a long shallow tail
A long tail distribution. Uniform sampling over songs lives in the green head; randomify samples somewhere between the head and the tail. Wikimedia Commons

Uniform sampling over songs sounds fair but is not what anyone wants. A catalog artist with 2,000 registered recordings would appear a thousand times more often than someone with two singles, and crowded genres would drown out everything else. Uniform sampling over artists overcorrects the other way. Randomify samples down a hierarchy instead: facet value, then artist, then release group, then recording, where the facet is one of genre, decade, country, or language, itself picked at random per spin. At every level, a node with \(n\) streamable songs beneath it is chosen with probability

\[ P(i) = \frac{n_i^{\alpha}}{\sum_j n_j^{\alpha}}, \qquad \alpha = 0.4. \]

The exponent tempers the popularity distribution. At \(\alpha = 1\) the walk reduces to uniform sampling over songs; at \(\alpha = 0\) every branch is equally likely and obscure artists are heavily over-weighted. The shipped value of 0.4 leans toward discovery while keeping one-track artists from swamping the catalog. It is fixed by design; the app exposes no way to change it.

One spin, one query

Probability density function of a continuous uniform distribution
The probability density of a continuous uniform distribution. Each level of the walk consumes one uniform draw in [0, 1). Wikimedia Commons

The pipeline precomputes the tempered weights into prefix-sum tables: each facet value, each artist within a facet value, and each release group within an artist stores its cumulative weight, indexed on (partition, cum_weight). Selecting a node is then a point lookup: scale a uniform draw by the partition's total and take the first row whose cumulative weight crosses it. The Worker generates the draws, and a single SQL query of chained CTEs resolves the entire four-level walk, joins the display metadata, and aggregates the platform links as JSON. One round trip per spin, and every level stays an indexed lookup no matter how popular the facet.

Repeats are handled without server state. The client keeps a ring buffer of the last 25 artist IDs it has seen and sends them as an exclude parameter. When exclusions are present the Worker generates eight artist draws instead of one, and the SQL walks all of them, preferring the first that lands on an unseen artist. If every draw hits the exclude list, the first draw wins anyway, so a spin always resolves.

Filtered spins

Filters (genre, decade, country, language, and a multi-artist picker) break the prefix-sum walk, because removing rows corrupts the cumulative weights. Filtered spins instead draw from a denormalized sample_recording table, one row per recording with all its filterable attributes and a precomputed weight that mirrors the artist, release group, recording sub-walk: each artist's total weight stays exactly \(a^\alpha\) for an artist with \(a\) tracks, release groups are tempered within the artist, and recordings are uniform within a release group. The draw itself uses the Efraimidis-Spirakis key: order the matching rows by \(-\ln(U)/w\) with \(U\) uniform, and the first row is an exact weighted sample over any subset in one pass. Genre matching runs against a GIN array index, and the filter pickers drill down: each dimension's options are counted over the set matching all the other active filters, so combinations that match nothing are hard to construct.

Finding the song on eight services

The corpus is anchored on Deezer, the one platform with a live resolver implementation: the Deezer API supports exact lookup by ISRC without authentication. The resolver queries it at about 4.5 requests per second, staying under Deezer's rate limit, and a recording enters the serving corpus only if the lookup returns an exact match with a playable preview. The Deezer record contributes the cover art, a release-year fallback, and the preview track ID.

Matches are verified rather than trusted blindly. Candidate and query are normalized (diacritics stripped, featuring credits and parenthetical remaster tags removed) and scored with the Sorensen-Dice coefficient over token sets: title weighted 0.5, artist 0.3, duration agreement 0.2, with the blend renormalized when a duration is missing. A fuzzy search match must score at least 0.82 to be accepted as exact. Even an ISRC match must clear a sanity floor of 0.5, because a bad ISRC in MusicBrainz or on the platform would otherwise ship a confidently wrong link. Resolved links are cached permanently in Postgres by ISRC and platform, so re-runs only touch what is new.

For the other platforms, exact URLs come from MusicBrainz URL relationships when they exist; otherwise the link degrades to a pre-filled search on that platform. Search fallbacks are only shown for full-catalog services where a search reliably surfaces the track. Pandora's search returns stations and Bandcamp mostly lacks mainstream releases, so those two appear only with an exact link, which is why a typical card shows six buttons. The pipeline also runs a canary: a golden set of well-known ISRCs with known-correct URLs per platform, checked every run, with per-resolver health that auto-demotes a broken adapter to search-fallback mode instead of shipping bad data.

Previews that expire

Deezer preview URLs expire within hours, so they cannot be stored. The corpus stores a stable /preview/{id} path instead, and the Worker mints a fresh preview URL at play time by re-fetching the track from Deezer and answering with a 302, cached at the edge for 45 seconds so replays cost nothing. The frontend preflights that redirect with redirect: "manual" before ever committing a song to the deck: a live preview resolves to an opaque redirect and a dead one to a clean 404, so no failed request hits the console and no unplayable song is ever shown. A candidate that fails preflight is skipped and the next spin is already prefetched, so shuffling feels instant.

The deck

The frontend treats spins as a deck: swipe or arrow forward to discover, backward to return to songs already heard, capped at 60 entries. Each new song autoplays its preview with a short volume fade, which requires some ceremony on Safari: the first user gesture runs a muted play-then-pause on the audio element to bless later script-initiated playback. Filters live in the URL query string, so a filtered shuffle is shareable and bookmarkable, and facet codes render through Intl.DisplayNames so the corpus can store ISO country and language codes while the UI shows names.

A Mac and a Worker

Production serving is entirely on Cloudflare: Pages for the frontend, a Worker per environment at api.randomify.net, and Hyperdrive pooling connections to a Neon branch per environment. The Worker opens one Postgres connection per request and closes it after, because Workers forbid reusing an I/O object across requests; Hyperdrive makes that cheap. Every request emits a telemetry point to Workers Analytics Engine, and failure modes raise alerts through a coalescing Durable Object that pushes to ntfy without flooding.

Corpus construction runs on a Mac under launchd: the dump refresh weekly, a resolve batch hourly, and a weight rebuild daily. Each job takes a lock, pings healthchecks.io on start, success, and failure, and pushes to ntfy when it fails. The dead-man's-switch matters because a powered-off Mac is silent; the absence of a ping is itself the alert. When a rebuild lands, the serving tables are truncated and reloaded inside a single transaction, so under Postgres MVCC readers see the previous corpus until commit and never a half-built one. The backlog keeps its cursor through all of this, and the catalog grows by up to a thousand resolved candidates an hour, indefinitely, toward the several million the dump has to offer.