URL Shortener System Design in Python

The URL shortener is a system design interview cliché for a reason: it looks trivial (a dict from short code to long URL) and is actually a compact lesson in three separate hard problems — generating unique IDs without a central bottleneck, encoding them compactly, and serving a wildly read-heavy workload (real-world shorteners see read:write ratios upward of 100:1) with low, predictable redirect latency. This post builds the real version, not the whiteboard shortcut.

Requirements that actually shape the design

Before any code: a redirect (GET /{code}) needs to resolve in single-digit milliseconds, because it sits in the critical path of someone else’s page load or marketing email click-through. URL creation (POST /shorten) can tolerate meaningfully more latency — nobody notices if link creation takes 50ms instead of 5ms. That asymmetry is the whole design: optimize the read path aggressively, and don’t let write-path complexity leak into it.

ID generation: why auto-increment doesn’t scale, and Snowflake does

The naive approach — an auto-incrementing integer primary key, base62-encoded — works fine until you need more than one database write node. A single auto-increment counter is, by definition, a single point of write contention and a single point of failure. The moment you shard your database for write throughput, “the next available ID” stops being a question one node can answer alone.

Twitter’s Snowflake algorithm solves this by making ID generation itself distributed and lock-free: each ID encodes a timestamp, a worker/shard identifier, and a per-millisecond sequence number, so any number of ID-generating nodes can mint unique, roughly time-sortable IDs independently, with no coordination between them.

import time
from threading import Lock

class SnowflakeGenerator:
    EPOCH = 1_700_000_000_000  # custom epoch, ms since Unix epoch
    WORKER_ID_BITS = 10
    SEQUENCE_BITS = 12
    MAX_WORKER_ID = (1 << WORKER_ID_BITS) - 1
    MAX_SEQUENCE = (1 << SEQUENCE_BITS) - 1

    def __init__(self, worker_id: int):
        if not (0 <= worker_id <= self.MAX_WORKER_ID):
            raise ValueError(f"worker_id must be 0-{self.MAX_WORKER_ID}")
        self.worker_id = worker_id
        self.sequence = 0
        self.last_timestamp = -1
        self.lock = Lock()

    def _now_ms(self) -> int:
        return int(time.time() * 1000)

    def next_id(self) -> int:
        with self.lock:
            timestamp = self._now_ms()
            if timestamp == self.last_timestamp:
                self.sequence = (self.sequence + 1) & self.MAX_SEQUENCE
                if self.sequence == 0:
                    while timestamp <= self.last_timestamp:
                        timestamp = self._now_ms()
            else:
                self.sequence = 0

            self.last_timestamp = timestamp
            return (
                ((timestamp - self.EPOCH) << (self.WORKER_ID_BITS + self.SEQUENCE_BITS))
                | (self.worker_id << self.SEQUENCE_BITS)
                | self.sequence
            )

Each of, say, 1024 worker nodes (WORKER_ID_BITS = 10) can independently mint up to 4096 IDs per millisecond (SEQUENCE_BITS = 12) with a mathematical uniqueness guarantee — no database round-trip, no coordination service, no shared lock across nodes. worker_id is typically assigned at deploy time (a pod ordinal in Kubernetes, an entry in a config map), not generated at runtime.

Base62 encoding: compact, URL-safe, case-sensitive

A raw Snowflake ID is a large integer — you don’t want that in a URL. Base62 ([0-9a-zA-Z]) packs it into a short, URL-safe string using every alphanumeric character, which is what gets you 6-7 character codes instead of the 10+ digits a base10 representation would need:

BASE62_ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"

def encode_base62(num: int) -> str:
    if num == 0:
        return BASE62_ALPHABET[0]
    digits = []
    base = len(BASE62_ALPHABET)
    while num:
        num, rem = divmod(num, base)
        digits.append(BASE62_ALPHABET[rem])
    return "".join(reversed(digits))

def decode_base62(code: str) -> int:
    base = len(BASE62_ALPHABET)
    num = 0
    for char in code:
        num = num * base + BASE62_ALPHABET.index(char)
    return num

A 63-bit Snowflake ID encodes to about 11 base62 characters worst case — noticeably longer than the 6-7 character codes you see from bit.ly. If short codes matter more to you than global coordination-free uniqueness, the trade-off is a smaller ID space per shard (fewer sequence/worker bits) or a random-code-plus-collision-check approach instead of Snowflake — there’s a real design decision here, not a free lunch.

Read path: cache-aside, and why write-through is the wrong default

The redirect endpoint is the one that has to be fast, so it’s the one that gets the cache. Cache-aside (check cache, fall through to DB on miss, populate cache on the way back) is the right default here over write-through, because writes (URL creation) are comparatively rare and low-latency-tolerant — paying a cache-population cost on every write to optimize a path that isn’t latency-sensitive is optimizing the wrong side of the 100:1 ratio.

import redis

r = redis.Redis()
CACHE_TTL_SECONDS = 3600

def resolve_short_url(code: str, db_lookup) -> str | None:
    cache_key = f"url:{code}"
    cached = r.get(cache_key)
    if cached is not None:
        return cached.decode()

    long_url = db_lookup(code)  # falls through to the database
    if long_url is None:
        return None

    r.setex(cache_key, CACHE_TTL_SECONDS, long_url)
    return long_url

For a real deployment, front this with an in-process LRU cache (functools.lru_cache won’t work directly since it doesn’t expire, but cachetools.TTLCache will) ahead of the Redis call — the most-clicked links (a viral tweet, a marketing campaign) benefit disproportionately from avoiding even a network hop to Redis for the hottest keys.

Sharding the database by code, not by creation time

Once URL storage outgrows one database node, shard by a hash of the short code (or the Snowflake ID’s low bits) rather than by creation time or worker ID. Sharding by worker ID means every redirect request for URLs created by worker 3 hits the same shard — a hot-shard problem the moment one worker’s traffic skews. Hashing the code spreads redirect load evenly regardless of which node originally created the URL:

NUM_SHARDS = 16

def shard_for_code(code: str) -> int:
    return hash(code) % NUM_SHARDS  # use a stable hash (e.g. hashlib) in production, not Python's salted hash()

Use hashlib.md5(code.encode()).hexdigest() truncated and converted to an int for the actual hash function — Python’s built-in hash() is salted per-process for security reasons (PYTHONHASHSEED), which means the same code can shard differently across restarts if you’re not careful to pin the seed or use a stable hash explicitly.

Custom aliases and collision handling

Users requesting a custom alias (short.ly/my-campaign) bypass the ID generator entirely — this path needs an explicit uniqueness check the Snowflake path doesn’t, since two users could request the same alias:

def create_custom_alias(alias: str, long_url: str, db) -> bool:
    if not alias.isalnum() or len(alias) > 32:
        raise ValueError("Invalid alias format")
    try:
        db.insert_url(code=alias, long_url=long_url)  # unique constraint on `code` column
        return True
    except UniqueConstraintViolation:
        return False  # caller surfaces "alias taken" to the user

Rely on the database’s unique constraint as the actual source of truth here rather than a check-then-insert in application code — a check-then-insert has a race window between two concurrent requests for the same alias that a database constraint closes for free.

Click analytics without slowing down the redirect

Recording click metadata (referrer, timestamp, geo) is valuable but must never block the 302 response — that’s the one thing users actually notice. Fire it into a queue and return immediately:

from fastapi import FastAPI
from fastapi.responses import RedirectResponse
import asyncio

app = FastAPI()

@app.get("/{code}")
async def redirect(code: str, request: Request):
    long_url = resolve_short_url(code, db_lookup=fetch_from_db)
    if long_url is None:
        raise HTTPException(status_code=404)

    asyncio.create_task(enqueue_click_event(code, request))  # fire-and-forget
    return RedirectResponse(long_url, status_code=302)

For production durability over asyncio.create_task‘s “best effort, lost on crash” semantics, publish to Kafka or SQS instead — the point stands either way: analytics writes are decoupled from the response path entirely, not awaited inline.

Related Reading

The takeaway

A URL shortener’s difficulty isn’t the redirect logic — it’s that “read-heavy” and “needs unique IDs at scale” pull the design in different directions if you don’t separate them explicitly. Snowflake IDs solve write-path coordination without a bottleneck; cache-aside plus code-based sharding solves read-path latency without over-optimizing the write path that doesn’t need it. Get that separation right and the rest — base62 encoding, alias handling, analytics — is comparatively mechanical.