Designing a URL shortener: load estimation, code allocation and choosing a redirect Intermediate

The problem

A user pastes a long URL and gets https://sho.rt/ab3. Anyone who opens the short link is sent to the original URL. A Map-based demo does this in 20 lines. The hard parts are elsewhere:

• Can two instances hand out the same code?
• Once a link is revoked, do browsers and the CDN still hold the old target?
• When one viral link gets more traffic than all the others combined, how does the database cope?

The two flows have opposite requirements. Writes (creating a link): few, need validation and a unique code. Reads (opening a link): very many, need a fast lookup and a correct redirect.

Key decisions in one table

Question	Suggested default	Change it when
Code length	6-7 base62 characters	You need 100 billion links or more, or harder-to-guess codes
Code allocation	ID range allocation + base62 (+ a permutation if guessing must be prevented)	Codes must be unguessable: random 7+ characters + UNIQUE + retry
Mapping storage	PostgreSQL, code UNIQUE	Writes exceed what one primary can handle: shard by code
Redirect	302 or 307 + short Cache-Control	The link never changes and you need SEO: 301/308
Click counting	Async events to a queue	You need exact numbers for billing: define “click” first

Load estimation (stated assumptions, recomputable)

These are the article’s assumptions, not measurements from a real system. Change the assumptions and the results change.

Assumption	Value
New links per month	100 million
Read : write ratio	100 : 1
Peak-to-average factor	10x
Retention	5 years, no deletion
Size of one row (code, URL, metadata, index)	~500 bytes
Size of one redirect response	~300 bytes

One month = 30 × 86,400 = 2,592,000 seconds.

Quantity	Calculation	Result
Average writes	100,000,000 / 2,592,000	~39 req/s
Average reads	38.58 × 100	~3,900 req/s (3,858)
Peak reads	3,858 × 10	~38,600 req/s
Links after 5 years	100 million × 12 × 5	6 billion
Storage after 5 years	6 × 10^9 × 500 bytes	~3 TB
Peak bandwidth	38,580 × 300 bytes	~11.6 MB/s
Hot cache (assuming 10 million links)	10^7 × ~200 bytes	~2 GB

The useful conclusion: writes are only ~39 req/s, so a single PostgreSQL primary is more than enough. Peak reads of ~38,600 req/s are what needs a cache. 3 TB does not force sharding right away, but it needs a partitioning or archiving plan.

Base62 and code length

62 characters 0-9a-zA-Z, so k characters give 62^k values.

k	62^k	Enough for
5	916,132,832 (~0.9 billion)	~9 months at 100 million links/month
6	56,800,235,584 (~56.8 billion)	6 billion links (5 years) with ~9x headroom
7	3,521,614,606,208 (~3.5 trillion)	Ample for random codes, very low collision rate

With an incrementing counter, 5 years needs only 6 characters. With random codes, 6 characters is too tight: 6 billion links fill 6×10^9 / 5.68×10^10 = 10.6% of the space, so roughly 1 in 10 first attempts collides. Use 7 characters.

Four ways to allocate codes

Approach	Mechanism	Pros	Cons
Counter / sequence	base62(nextval)	Never collides, shortest codes	Guessable, reveals scale, single coordination point
ID range allocation	Each instance claims a range (for example 1,000 IDs) and counts in RAM	Keeps the counter’s advantages; the allocator is almost never called	A crash loses the unused part of the range (acceptable); codes are not monotonic across instances
Random	7 random characters (CSPRNG) + UNIQUE + retry	Hard to guess, no coordination	Needs retries; collision probability grows as the space fills
Hash truncation	Truncate hash(URL) to 7 characters	Same URL yields the same code (natural dedupe)	Low birthday bound, see below; still needs retry with a salt

Range allocation is what PostgreSQL does itself with CREATE SEQUENCE ... CACHE n: each session reserves n numbers, unused numbers are lost when the session ends, and sessions are only guaranteed distinct values, not ordered ones.

Sequences also never roll back, so gaps happen even with CACHE 1. For a shortener, gaps are fine.

Birthday bound: why hash truncation is dangerous

The space is N = 62^7 ≈ 3.52 × 10^12 (about 41.7 bits). With n links, the probability of at least one colliding pair ≈ 1 − e^(−n² / 2N).

n links	Probability of any colliding pair
1 million	~13%
~2.2 million	~50% (n ≈ 1.18 × √N)
10 million	~100%

So truncating a hash to 7 characters means collisions show up very early, after just a few million links.

Random codes ask a different question: the probability that one new insert hits an existing code is n / N, which at 6 billion links is 6×10^9 / 3.52×10^12 ≈ 0.17%, so retries are cheap. Both need a UNIQUE constraint as the arbiter.

To use a counter without revealing the order, map the ID through a bijective permutation on [0, 62^7) (for example a keyed Feistel network) before base62. Uniqueness is preserved and the codes look random.

Redirect: 301, 302, 307 or 308?

Per RFC 9110, section 15.4:

Status	Meaning	Default caching	Changing POST to GET	SEO / analytics
301	Permanent new target	Heuristically cacheable	Client may change it	Google treats it as a canonical-target signal; browsers/CDNs may keep it for a long time, later requests skip the origin, clicks are lost, the target is hard to change
302	Temporary target	Cached only with explicit cache headers	Client may change it	Google does not treat it as canonical, the short URL stays the indexed URL; every click reaches the origin unless cached
307	Temporary target	Same as 302	Method change forbidden	Same as 302
308	Permanent new target	Heuristically cacheable	Method change forbidden	Same as 301; a few old clients do not recognize it

Points to remember:

• “Heuristically cacheable” means a cache may infer its own freshness lifetime (RFC 9111). Do not rely on that behavior: always send an explicit Cache-Control.
• A shortener only serves GET/HEAD, so the difference between 301 and 308 (or 302 and 307) is mostly historical; choose the newer pair (307/308) if you want strict semantics.
• The default should be 302 or 307 with Cache-Control: private, max-age=60 (or public to let the CDN cache it): you can count clicks and still change or revoke within tens of seconds.
• Use 301/308 when the link never changes and you accept not counting most repeat clicks.

How it works

The write path allocates a unique code; the read path goes through the CDN, then the cache, then the store

Enlarge diagram

Link-creating client → Write APIPOST target

◀ Prev▶ PlayNext ▶↺ RestartStep 1/6

Write: the Write API validates the URL (scheme, length, blocklist), then needs a unique code.

What to notice in the diagram:

1. The allocator is rarely called: at ~39 writes/second with 1,000-ID ranges, the whole system requests a range about 0.04 times per second (39 / 1,000).
2. Reads have three layers that can stop them: CDN, Redis, and only then the store. Most of the 38,600 req/s peak never touches the database.
3. The redirect’s cache policy decides which layer can absorb traffic: a long max-age lets the CDN absorb more, but revocation is slower.

Details to handle in the implementation:

• A non-existent link returns 404; a removed or expired link can return 410 Gone. Negative caching should last only a few seconds so a freshly created code is not stuck on 404.
• Hot links expiring from the cache at the same time cause a stampede: see cache stampede.
• When revoking a malicious link, delete it from Redis and purge the CDN; redirects already cached in browsers only go away when max-age runs out.
• A server that fetches the target URL itself (for a preview) opens up SSRF: see SSRF.

Real-world example

SMS campaign links: 2 million messages in 10 minutes

A campaign sends 2 million SMS messages, each with its own short link. Assume 20% of recipients open the link in the first 10 minutes: 400,000 clicks / 600 seconds ≈ 670 req/s, far below the 38,600 peak above.

But if the same link is sent to everyone, 670 req/s land on one key. Redis can take it; the store should not be touched at all: cache-aside with a TTL and request coalescing is mandatory.

The team uses 302 with max-age=60. When they find the target is broken, they fix it and clear the cache; within 60 seconds every user sees the new target. Had they used 301, people who already opened the link could still hold the old target in their browser.

Demo: code allocation and validation

npx --prefix examples ts-node examples/src/url-shortener/shortener-demo.ts

Real output:

{ code: '1', target: 'https://example.org/articles/42' }
{ missing: '404' }

The demo only checks base62, validation (https: and no credentials) and an in-process lookup. Two separately running instances both start at nextId = 1, so they produce duplicate codes: that is why you need an allocator and UNIQUE in shared storage.

The 404 string is printed by the demo itself, not an HTTP response.

Minimal table in PostgreSQL:

CREATE TABLE link (
  id          bigint PRIMARY KEY,
  code        text   NOT NULL UNIQUE,
  target_url  text   NOT NULL,
  owner_id    bigint NOT NULL,
  created_at  timestamptz NOT NULL DEFAULT now(),
  expires_at  timestamptz,
  disabled_at timestamptz
);

The UNIQUE index on code serves the lookup (see B-tree index). Add (owner_id, created_at) only when a link management page needs it. Do not reuse expired codes: old links would point to a new target.

🎮 Try it

• Scale Lab: URL shortener brief: 4 rounds from MVP to a viral link and bot spam.
• Build a URL shortener: one drag-and-drop level within a budget.

Production checklist

1. Enforce unique codes with a UNIQUE constraint; catch duplicates and reissue (random) or claim a new range (allocator).
2. Derive code length from the expected number of links, with 10x headroom.
3. Choose the redirect status from the change/revoke requirements, and always set an explicit Cache-Control.
4. Cache-aside for lookups, a finite TTL, negative caching for only a few seconds, and a way to purge on revocation.
5. Rate limit per IP on non-existent codes (anti-enumeration) and cap code length.
6. Validate target URLs: allowed schemes, no credentials, a phishing blocklist; never fetch the URL yourself before blocking SSRF.
7. Analytics through an async queue; define “click” (request, bot-filtered, or session) before reporting.
8. A takedown procedure for malicious links, including cache and CDN.

When to use / not use this design

• Use it: read load is tens of times higher than writes, links rarely change, and eventual consistency of a few tens of seconds on revocation is acceptable.
• Not enough: you need instant global revocation (use Cache-Control: no-store and accept that every click hits the origin), or exact per-person click counts (needs dedicated tracking, not inferred from redirects).

Common mistakes

Common pitfall

Checking for duplicate codes only in the application. Two instances both see “not there yet” and both insert. UNIQUE or an allocator must be the arbiter.

Common pitfall

Choosing 301 and expecting a target change to take effect immediately. 301/308 may be cached for a long time, even if you set no max-age. If you need to fix or revoke quickly, use 302/307 with a short max-age.

Common pitfall

Truncating a hash to 7 characters without handling collisions. The birthday bound puts the probability of a colliding pair at ~13% already at 1 million links.

Common pitfall

Counting every redirect as a user. Bots, prefetching and caching make the request count differ from the number of real people.

Summary

Key takeaways

• Estimate first: under the stated assumptions, writes are ~39 req/s, peak reads ~38,600 req/s, 3 TB after 5 years; writes are easy, reads need a cache.
• 6 base62 characters (56.8 billion) are enough for a counter; random codes use 7 characters.
• 7-character hash truncation collides at ~2.2 million links (birthday bound); every allocation scheme needs UNIQUE.
• 302/307 + a short Cache-Control is the safe default; 301/308 only for immutable links.
• Defining analytics and the revocation procedure are design requirements, not extras.

Self-check:

1. Recompute peak read QPS if the read : write ratio is 500 : 1.
2. With 8 base62 characters, at roughly how many links does hash truncation reach a 50% collision probability (hint: 1.18 × √(62^8))?
3. A shortener only serves GET, so how do 301 and 302 differ (caching, analytics, SEO)?

References

• RFC 9110, section 15.4 — Redirection 3xx
• RFC 9111 — Heuristic freshness
• Redirects and Google Search
• CREATE SEQUENCE — PostgreSQL 17
• Unique Indexes — PostgreSQL 17
• URL API — Node.js