0.5ms atomic trades with zero distributed locks: building an AMM inside Redis Lua

This post is part 1 of a 4-part series on engineering FTL, a live football trading platform where players trade virtual shares of World Cup 2026 footballers with prices driven by real match data. Published with the client’s permission. The production codebase is proprietary. A public skeleton of this component (redismm) is in preparation.

Picture a packed stand and fifty thousand phones. A goal goes in, everyone taps Buy on the same footballer, and the price has to move for all of them without two people spending the same coins. That is FTL: a virtual match-day market. In-game currency, footballer shares, a live crowd hitting one instrument.

This post is how we made that trade atomic in half a millisecond. At the end I wrote two short “what I’d do next time” notes: one for this game, and one if the same fill were real money.

Every trade on a market has to do five things at once: check the buyer’s balance, debit it, update their position, move the price, and update the leaderboard. If any two of those can interleave from two concurrent requests, you can double-spend. If the process dies between them, you can debit a wallet and never fill the order.

The textbook answers are a Postgres transaction with SELECT ... FOR UPDATE, or a distributed lock. Both work. Both make your trade path pay the latency of a network round trip to a disk-backed system under contention. On a live-match day, every position on the pitch trades against the same instrument, so contention is the shape of the workload.

We took a different route: the entire trade (validation, pricing, debit, position, leaderboard, even the price broadcast) is one Redis Lua script. Redis executes Lua scripts single-threaded and atomically. No lock exists in our trade path, because the execution model is the lock.

The market maker: a linear bonding curve

FTL isn’t an order book. There’s no counterparty waiting on the other side. It’s an automated market maker (AMM) with a linear bonding curve:

price = basePrice + k × netSharesSold

Buy and the price slides up the curve; sell and it slides back down. k controls liquidity: high k means prices move sharply on small trades. The nice property of a linear curve is that fills have a closed form. Buying q shares when net shares are outstanding costs an arithmetic series:

// internal/amm/amm.go
func ComputeTotalCost(basePrice, k float64, netSharesSold, quantity int) float64 {
    n := float64(quantity)
    net := float64(netSharesSold)
    total := n*basePrice + k*(n*net+n*(n+1)/2)
    return math.Round(total*100) / 100
}

No loop over shares, no partial-fill bookkeeping. One multiplication, one division, done. That matters when this runs inside a script whose entire execution budget is sub-millisecond.

One script, thirteen keys

Here’s the actual shape of trade_execute.lua (trimmed to the parts that matter):

-- KEYS: instrument hash, wallet hash, position hash,
--       2 leaderboard zsets, 3 dirty sets for the flusher,
--       per-user PnL hash, idempotency key, trade-stats hash

-- 1. Idempotency claim, before anything else
local claimed = redis.call('SET', idem_key, '1', 'NX', 'EX', 86400)
if not claimed then
    -- duplicate request: replay the cached response
    local cached = redis.call('GET', 'idem_result:' .. crid)
    if cached then
        local decoded = cjson.decode(cached)
        decoded.replayed = true
        return cjson.encode(decoded)
    end
    return cjson.encode({status = "duplicate", replayed = true})
end

-- 2. Validate: quantity, limit price, slippage tolerance, balance
-- 3. Compute fill via the arithmetic series
-- 4. HSET wallet balance, position shares/avg_price, instrument net_shares
-- 5. ZINCRBY leaderboard by the PnL delta
-- 6. PUBLISH the new price to the pub/sub channel
-- 7. SADD the touched entities into dirty sets for async persistence
-- 8. Cache the full JSON response and return it

Because Redis runs scripts atomically, there is no window between “check balance” and “debit balance” where another request can squeeze in. The worst-case interleaving is impossible by construction. Our execution budget for the full script is 0.1–0.5ms; end-to-end trade latency including HTTP is sub-5ms.

Idempotency: the part everyone gets subtly wrong

You’re on 4G in a stadium. You tap Buy. The spinner hangs. You tap again. Now the server has two identical trade requests. The first one filled. The second one must not charge you twice, and it also shouldn’t return an error, because from your phone the trade already worked.

The standard answer is “check if we’ve seen this request ID before.” That answer has a race: two concurrent duplicates can both pass the check before either records itself. The fix everyone reaches for is SET NX, but SET NX alone answers the wrong question. It tells you “you’re the duplicate,” not “here’s what the original returned.”

So we pair the claim with a response cache:

1. SET idem:{user}:{clientRequestId} 1 NX EX 86400. First request wins the claim.
2. Every successful execution caches its full JSON response at idem_result:{clientRequestId} (24h TTL).
3. A duplicate gets the cached response with replayed: true injected, the exact same payload the original caller saw.

That last detail matters more than it looks. The Go handler checks replayed and skips the non-idempotent side effects (XP awards, notifications, activity feed), so a retry can’t farm engagement rewards by hammering the retry button. And because the idem key embeds the userId, a stolen or guessed clientRequestId can’t retrieve another user’s cached result.

We still keep a Postgres composite index on (client_request_id, created_at) as a last-line defense for the rare case where Redis is down or the 24h TTL has elapsed. Defense in depth, but the Lua script is the authoritative idempotency boundary.

The leaderboard trap

This one took us a bug report to figure out properly. A user’s leaderboard score is their total unrealized PnL across all positions. When they trade instrument A, the naive implementation does:

ZINCRBY leaderboard <new pnl of A> <user>

Wrong. That replaces A’s contribution but the score is a sum, so you’ve just double-counted every other position they hold. The correct unit of update is the delta:

local old_pnl = redis.call('HGET', user_pnl_key, instrument_id) or 0
local delta = new_position_pnl - old_pnl
redis.call('HSET', user_pnl_key, instrument_id, new_position_pnl)
if delta ~= 0 then
    redis.call('ZINCRBY', lb_overall, delta, user_id)
end

A per-user hash tracks each instrument’s PnL contribution; every trade applies the diff to the zsets. And this lives inside the same atomic script as the trade, so a fill and its leaderboard movement can never diverge. No “trade succeeded but leaderboard lost the update” class of bug.

There’s an operational contract hidden here: the hash must be hydrated before a user’s first trade. If it isn’t, the delta computes as new_pnl - 0, which silently inflates the score. We enforce that in the Go service (EnsureLeaderboardState). When your correctness depends on a precondition, write the enforcer next to the code that needs it.

Things we got wrong (so you don’t have to)

Rounding drift. We round money to 2 decimal places with math.floor(x*100+0.5)/100, and it has to be the same function everywhere. Our first avg_price update multiplied quantity × fill_price, but fill_price is a rounded value, so quantity × fill_price ≠ total_cost by a paisa now and then. The VWAP numerator drifted from the actual debit. The fix was to compute averages from total_cost directly and treat the rounded fill price as display-only. In money code, there is exactly one source of truth per quantity, and it’s not the formatted one.

Lua floats. Lua 5.1 numbers are doubles. Fine for our price magnitudes, but we never compare computed floats for equality, only within epsilon, and we round at every boundary. If we ever needed integer paisa math, we’d do it in the response layer, not by hoping a double behaves.

Script size. Lua scripts are cached by hash; a 375-line script with an argument-layout contract of 13 positional KEYS is a real API. We froze the key positions (retired slots stay as empty placeholders) because changing them is a deploy-ordering nightmare across replicas. If we did it again, we’d consider RedisACL + smaller scripts, and atomicity requirements would have won anyway.

The freeze flag. During full-time liquidation we set HSET instrument:<id> frozen 1 so no trades land while the Postgres-side closeout is enumerating positions. Our first version didn’t have this, and a trade could slip in after the snapshot was taken. The freeze is cleared when the post-match handler finishes, and if it crashes, operators clear it manually. Boring beats clever.

Load-testing it: the bots are the market

An AMM with no users is a price that never moves. To test throughput and market behavior, we run an in-process bot engine (internal/bot/): goroutines with trading personas and per-strategy evaluators, safety-limited, hydrating their state through a sync.Map. The same bots serve two purposes: they’re the liquidity that makes charts look alive in a quiet market, and they’re the load generator we point at the trade path. Kubernetes-style chaos testing, except the chaos is a few hundred bot traders all reacting to a recorded goal at the same moment.

What this buys us at 50K users

• Zero locks in the hot path. Contention doesn’t exist at the data-structure level; Redis serializes scripts for you.
• Idempotency with replay, at the boundary where retries actually arrive.
• Atomic leaderboard + broadcast: the price update your phone receives was published by the same script that debited the wallet. Chart and wallet cannot disagree for a beat.
• Postgres stays out of the hot path entirely. It’s the ledger, filled in asynchronously by a write-behind flusher, which is exactly what part 3 of this series is about.

What I would do differently for this use case

Same game. Same virtual coins. Same Redis Lua. Re-reading the script for a public skeleton (redismm) found three bugs that only show up when a stranger reads the code, or when 100K people tap Buy at once.

The ticket window counted shares one by one. Redis is one clerk. While the Lua script runs, nobody else gets served. We used to loop for i = 1, quantity, so a 40-share buy held the whole market 40 times longer than a 1-share buy. Go already had the closed-form math, sitting in amm.go for the chart. We put that math in the script. One multiply, one divide, same answer. A property test still checks the closed form against the old loop.

A failed buy hogged the “already seen this” sticker. You tap Buy, you’re over the slippage limit, nothing moves. We still held the SET NX claim for 24 hours and never cached the error. Retry, and the server said {status: "duplicate"}. The real reason was gone. Claims should protect effects. A rejection now DELs the claim, so a retry gets a real answer (or fills once you fix your balance).

A typo minted money. Missing wallet? The script handed out a default balance. One mistyped user id, free coins, inside the hot path. Wallet creation is now an explicit, audited SeedWallet call. The trade path returns an error.

The pattern: invisible at 100 users, page one of the postmortem at 100K.

I’d also pin the idempotency keys. Put idem:* on a Redis DB (or instance) with maxmemory-policy noeviction, away from cache keys the flusher can squeeze, and WAIT one replica before returning 200. Still half a millisecond. A slightly smaller “Redis blinked” window. Still a game.

What I would do differently for a fintech

Same engine, except the wallet is rupees someone actually deposited. I’d keep the shape. I’d change five things.

1. Write it down before you say yes

In FTL we tell the phone “filled” the moment Lua returns. Postgres catches up in the background. That’s why a trade feels instant. If Redis dies in that gap, the user saw a fill the ledger never got. For in-game coins, that’s a support ticket. Part 3 is how we made the gap small.

If this were real money, I’d wait until the ledger has the row (or a local disk log has fsynced) and then return 200. Redis stays the fast matcher and the read cache. The phone waits on a local fsync, which is a different wait from a contended Postgres round trip.

2. Count coins as integers

Lua 5.1 numbers are doubles. We round to two decimals at every boundary and never ask two computed floats if they are “equal.” Fine for virtual shares. Over millions of real fills, the residue adds up. 0.1 + 0.2 really is 0.30000000000000004.

I’d move the arithmetic into Go on int64 paisa (or a decimal library). Redis holds state. The ledger column is numeric. Rounding for display happens once, at the edge.

3. A smaller “everything at once”

The script is 375 lines: wallet, position, two leaderboards, dirty sets, XP-adjacent stats, pub/sub. Redis is single-threaded. While it runs, the whole instance waits. In a game, putting the leaderboard in the same breath as the fill is a feature. Your rank and your wallet cannot drift. It is also a blast radius. One slow leaderboard line stalls the market.

I’d keep match + wallet + position as the atomic unit. Leaderboard, XP, feed, broadcast become listeners of the fill. The money path is ordinary Go you can step through in a debugger.

4. Plan for more than one clerk

Zero locks works because every trade queues on one Redis thread. At 0.5ms that’s a theoretical 2,000 trades a second on that core, less in real life. The script also touches the overall leaderboard and the user’s wallet, so you cannot split footballers across Redis nodes without a user buying A on node 1 and B on node 2 and overdrawing.

Plenty for a match-day game. For real money I’d shard by instrument, reserve funds on a wallet service before the match, or pin a user’s whole book to one shard. Spend the single-thread budget inside one footballer’s matching core, not across the whole market.

5. Remember the tap in the ledger

We claim with SET NX and cache the JSON for 24 hours. That’s the path retries actually hit. Redis under memory pressure can evict those keys. After eviction, a retry looks brand new. You get filled twice.

We already have a Postgres unique index on (client_request_id, created_at) for Redis-down and TTL-elapsed. For real money that row is the memory of the tap. Unique constraint, return the stored fill on conflict. Redis can still replay in microseconds. It is a cache.

The two designs, side by side

Decision	This game	If it were real money
Durability	Redis first, Postgres a beat later	Ledger commit before 200 OK
Money	Doubles, round at every boundary	int64 paisa / numeric
Atomic unit	375-line Lua: trade + leaderboard + pub/sub	Match + wallet + position
Scale	One Redis thread, ~2k TPS ceiling	Shard by instrument, reserved balances
Idempotency	SET NX + 24h cache; PG unique as backup	Ledger unique constraint is the record

A match-day spike is the product here. The same shape, with a ledger that says yes only after it has written, integer coins, a smaller atomic unit, a shard plan, and a durable “you already tapped Buy,” is what I’d build if the coins were real.

I’m open to backend/Go roles in fintech. If your team wrestles with atomic trading paths, I’d love to talk. LinkedIn

Resources

• Series: Part 2: WebSocket fan-out · Part 3: write-behind · Part 4: margin engine
• Studio: xpar.in
• Code: a public redismm repo is in preparation. The production FTL source is proprietary, so it is not linked here.

Next in the series: streaming those price updates to 50K WebSocket clients over a 95-byte wire budget, on a $200/month infra budget.