Skip to content
Amal.
← Back to writing

Your position gapped while you were offline: liquidation fairness via offline replay

A phone is off for eight minutes. A goal gaps the price. Their position should have died. At what price do you close it?

16 min read
A phone face-down beside a price tape with a sudden gap and an hourglass

Part 4 of a 4-part series on engineering FTL, a live football trading platform with leveraged CFD-style positions. Published with the client’s permission. The production codebase is proprietary. A public skeleton of this component (marginlite) is in preparation.

A user opens a leveraged position at 55. Then the phone goes in a pocket. WhatsApp, lock screen, whatever. Eight minutes later a goal is scored, the footballer gaps to 31, and their margin blows through the line. They never saw it happen.

That is FTL: a virtual match-day market with CFD-style positions. In-game coins, footballer shares, phones locking mid-game.

This post is how we liquidate that position fairly. At the end: what I’d change for this game, and what I’d change if a washout were real money.

Now they reopen the app. Here’s the question that decides whether the platform is fair or a casino:

At what price do you close them? The price now, or the price at the moment they actually went under?

Liquidating at the current price means the platform ate the gap on their behalf. If the price bounced back in those 8 minutes, they were liquidated on a phantom loss. Not liquidating at all means a position that should have died is still open, running on equity that doesn’t exist. Every leveraged product (CFDs, perpetuals, prediction markets) has this problem, and most systems quietly solve it wrong by doing whichever of the two wrong things is easier.

Our answer: rewind the tape they missed. When the user reconnects, we re-simulate every price tick they didn’t see, against their open positions, and liquidate at the breach-tick price. The price at the exact moment their margin level actually crossed the threshold.

The margin model, in one function

Margin level is the ratio that decides everything:

margin_level = equity / used_margin × 100
equity       = balance + Σ unrealized PnL across open positions

Cross below the margin-call threshold: warn. Cross below the washout threshold (50%): force-close positions until the level recovers. The decision core is a pure function. No I/O, no side effects:

func cascadeWashouts(userID string, balance float64, lives []live) (MarginState, bool, []WashoutStep) {
    for {
        totalUsed, totalPnL := 0.0, 0.0
        for _, l := range lives {
            totalUsed += l.margin
            totalPnL += l.pnl
        }
        equity := balance + totalPnL
        level := (equity / totalUsed) * 100

        // Stop when healthy or nothing left to close
        if totalUsed == 0 || level > WashoutThreshold || len(lives) == 0 {
            return state, emitCall, washouts
        }

        // Close the largest loser first
        sort.SliceStable(lives, func(i, j int) bool {
            if lives[i].pnl == lives[j].pnl {
                return lives[i].row.ID < lives[j].row.ID  // deterministic tiebreak
            }
            return lives[i].pnl < lives[j].pnl
        })
        washouts = append(washouts, WashoutStep{...worst...})

        // Simulate the close: realize the loss, drop the position, re-check
        balance += worst.pnl
        lives = lives[1:]
    }
}

Three design choices in there that each earn their place:

Cascade, not single-shot. One washout might not be enough. If the loss was catastrophic, closing the worst position may still leave the level below 50%. The loop closes the next-worst, recomputing equity each time, until the level recovers or nothing is left. Like paying off the most expensive bill first so you get back above water with the fewest sales.

Worst-first is a fairness policy, not an implementation detail. Closing the biggest loser recovers the most equity per close, so the cascade is shortest possible. The user loses the fewest positions that the math allows. An arbitrary close order would liquidate positions that didn’t need to die.

Deterministic tiebreaks. The stable sort falls back to position ID, so the same inputs always produce the same washout order. On the live path, in tests, and (this is the critical one) in the replay.

Pure core, thin shell

cascadeWashouts touches nothing. Redis and Postgres reads happen in a shell around it; the washout list it returns is applied by a separate executor that routes each close through the normal position-close path (so leaderboards, XP, and notifications behave identically to a manual close).

This separation is what makes the replay possible at all: the same decision function runs against live state or against reconstructed historical state, and produces identical decisions. When your risk engine is a pure function of (balance, positions, prices), “what would have happened” is just that function with old prices.

The replay: honoring the breach, at the breach price

Timeline: offline window, goal gap to 31, breach tick, reconnect replay liquidating at the breach price
The replay walks every missed tick. The liquidation fires at the tick where the margin actually crossed the threshold.

When a user’s last WebSocket connection drops, a hub edge hook (connection count transitioning 1 → 0) stamps last_seen:{userID} in Redis. When the first connection comes back (0 → 1), a symmetric hook fires ReplayLastSeen:

func (s *Service) ReplayLastSeen(ctx context.Context, userID string) ([]WashoutStep, error) {
    from := time.UnixMilli(redis.Get(ctx, "last_seen:"+userID))
    to   := time.Now()
    // 1. Pull positions that were open BEFORE the window started
    // 2. Pull price_ticks for those instruments over [from, to], chronologically
    // 3. Walk tick by tick; at every tick, run the cascade
    // 4. Washouts fire at their BREACH-TICK price, not today's price
    steps := replayLogic(balance, candidates, ticks, currentPrice, usedMargin)
    ...
}

The replayed washout steps carry a BreachTime, and the executor closes each position at the price on that historical tick. The user’s app opens, the replay completes within ~1 second, and the ledger says what would have happened if they’d been watching. No phantom losses from post-gap bounces, no zombie positions running on dead equity.

A bug from production worth admitting: the first version filtered replay candidates with OpenedAt < window_end, which wrongly admitted positions opened mid-window (during the user’s disconnection!). Replaying those against a balance that doesn’t reflect their opening produced garbage washouts. The correct filter is OpenedAt <= window_start. Small bug, big fairness consequence. Position-set selection deserves the same scrutiny as the math.

The same replay runs as ReplayUsersAtFT before full-time liquidation, so the end-of-match snapshot only sees positions that would genuinely have survived their offline windows.

The edges: throttles, backstops, and users who never come back

The live path evaluates margin on every price tick for users holding that instrument. A naive “evaluate on every tick” burns Redis with redundant work, and one washout cascade could re-trigger itself in a loop. Three guards:

  1. Per-user 10s throttle. A Redis SET NX key per user; evaluation skips if one ran <10s ago. A goal moving the price 30% doesn’t need 50 evaluations in the same second; one washout cascade handles it.
  2. 5-minute backstop sweeper. Catches users who never reconnect. If someone’s last evaluation is stale and they hold open positions, the sweeper replays their window server-side. Fairness can’t depend on the user’s app coming back online.
  3. Margin-call cooldown (30 min). One notification per cascade episode, not one per tick below the warning threshold.

Performance is a non-issue at honest scale: the replay is O(T × P²) worst case, but for a typical user (5 positions, ~1 tick/sec, 5-minute offline window ≈ 300 ticks) that’s ~7.5K operations. Microseconds of work for a fairness guarantee most platforms don’t offer.

Why this matters beyond football

Strip the sport out and this is any leveraged product where people put the phone down: the risk engine must be a pure, deterministic function of state, and “state” must be reconstructable for any past window. That requires:

  • Price history persisted per tick (our price_ticks table, which the write-behind flusher from part 3 fills).
  • Edge hooks on connection lifecycle (0→1 and 1→0 transitions, fired once per user, not per tab. Multi-device users taught us that the hard way).
  • Breach-price honesty in the washout executor.

The general pattern, offline-window replay with breach-price liquidation, is rare even in commercial trading platforms. A public marginlite skeleton (evaluator, cascade, tick-replay) is in preparation. The production FTL source stays with the client.

What I would do differently for this use case

Same game. Same virtual CFDs. Same “close them at the moment they went under.” Re-reading this for a public marginlite skeleton, I’d keep the pure core, the thin shell, and the OpenedAt confession. I’d change four things around the clocks.

One movie for people watching and people whose phone is off. Live holders run on every price tick, then a 10s SET NX skip. Offline users get tick-by-tick ReplayLastSeen at the breach price. Users who never return wait on the 5-minute sweeper. That is three clocks. The people watching the goal can get throttled “now”; the people who locked the phone get the first-breach tick. I would make server-side evaluation of all holders the primary path, independent of WebSocket presence, and record the breach (instrument, price, BreachTime, position set, equity) at decision time. Replay becomes catch-up and verification, not the only accurate movie. Keep 10s SET NX as a notification and re-entry throttle if Redis must be saved; do not let it skip the first tick that crossed 50%. A 30% goal spike still gets one cascade. I would fire that cascade on the goal, or on the first tick that crossed the line, so the price it sees is the breach-tick. Realizing PnL does not change equity (balance += worst.pnl and drop the live); the ratio recovers by freeing used margin, so I would close largest used-margin first, or keep worst-PnL as a product rule, not as “shortest”, and tiebreak on rounded PnL (the same two-decimal formula the series already uses), then row.ID.

Treat (last_seen, price_ticks, as-of book) as risk inputs. The window is Redis last_seen:{userID} millis versus time.Now(). Ticks are supposed to come from part 3’s flusher; the published contract there is coalesced latest-hash, not a tape. I would depend on the second-build tick tape from that postscript, a Redis list or stream drained into price_ticks, still write-behind, not on dirty-set snapshots. Pin last_seen against eviction (noeviction, same instinct as part 1’s idem:*). Bound the window to sweeper cadence, and refuse a walk from a stale or zero key. I would not treat a missing stamp as epoch. Share a per-position washout idempotency key so sweeper ∩ reconnect cannot double-apply through the normal close path. Advance last_seen / last_replayed_to after success. One clock: hub/server event time, not Redis millis versus process Now().

Shorten the zombie window; name house absorption. Five minutes of a World Cup match is a goal and a market that has not absorbed the unwind. Until reconnect or sweep, the position is still open; the unwind is late, and the rest of the market traded as if it were live. I would sweep on tick/goal cadence, seconds, or the same 10s already tolerated online, still server-side, still at the breach-tick. Honouring 31 when the live curve has moved is a virtual gap; I would book it to an explicit platform PnL bucket, not smuggle it through “manual close” plus XP and leaderboard. Keep ReplayUsersAtFT as a belt.

Finish reconstruction the way OpenedAt taught. The production bug was OpenedAt < window_end, which admitted mid-window opens; the fix is OpenedAt <= window_start. Small bug, big fairness consequence, and the same class of hole is still open. Reconstruct positions open at window_start and not closed before the tick under test; balance as-of window_start; used margin from that set. Drop currentPrice from the decision, or define it as the last tick on the tape. Reason codes (washout_replay / washout_live / washout_sweep); skip XP and “you closed” copy. Part 1 already refuses to farm XP on retries. Persist a decision record so reconnect, sweeper, and FT all no-op if that tick already closed. Property-test live versus replay on the same tape until the washout lists match. Say cross versus isolated out loud: one equity over all footballers can wash a different row.ID than the instrument that ticked; isolated matches the lede (“that goal”); if we keep cross, I would say so, and I would not mark other legs at currentPrice while replaying a partial tape.

The pattern across all four: the fairness slogan is only as good as the clock and the tape that feed the pure function. Pure cascadeWashouts, the shell versus the executor, deterministic ID as an idea, per-user not per-tab hooks, BreachTime on the step, ReplayUsersAtFT, and admitting OpenedAt earned their place, including the ~1s reconnect. I would keep liquidating at the breach-tick. I would make that movie the same for the person watching the goal and the person whose phone was off.

What I would do differently for a fintech

Same engine, except a washout is a legal fill. I’d keep closing at the moment they went under. I’d change five things.

1. Every mark, server-side; WebSocket is not a risk clock

For FTL, risk is scheduled off connection lifecycle: last_seen on 1→0, ReplayLastSeen on 0→1, a 10s SET NX skip for live holders, a 5-minute sweeper for users who never reconnect. That is the right split for a virtual match-day: a locked phone is a fairness story, and a missed in-game washout is a support ticket.

For cash, offline is not a state the risk engine is allowed to care about. Mark-to-market and liquidation run server-side, on every mark, whether or not any socket exists. A 10s skip after a 30% move is an intra-window hole. A 5-minute sweeper is a batch job. Using WebSocket presence as the risk clock inherits hub partitions, multi-device flaps, and clock skew as missed liquidations.

What I would do: a dedicated worker consumes the mark stream at exchange time, not time.Now(). Every account with a position in a touched instrument is evaluated on that mark, connected or not. Liquidation is attempted immediately. Circuit breakers sit in front of that loop. Reconnect is a view of already-closed positions, never the moment risk comes off. Tests may replay a recorded log through cascadeWashouts. Production does not reconstruct a window from a socket hook.

2. Live executable price + explicit gap policy

For FTL, the executor closes at the price on that historical tick. That is the whole fairness claim of this post, and it is the right claim for in-game leverage: the user’s ledger shows what would have happened at 31, and the second build books the curve gap to a virtual platform bucket.

For cash, you cannot honour an eight-minute-old AMM tick and actually unwind a live curve. The executable price is the book or the curve now. Forcing 31 while the house covers at 28 (or cannot cover) is a P&L hole, not a fairness story. Routing a forced close through the “manual close” path also destroys audit meaning: a liquidation is not a customer sell.

What I would do: a distinct liquidation order type on the live venue. The user’s fill is whatever the book or AMM actually gave. The gap goes to an insurance fund, then ADL / socialized loss with a published rule. Never rewrite the fill from a write-behind table. Do not route liquidations through “manual close.”

3. Integer money, conservative ratio

For FTL, the core is balance float64, pnl == pnl, level > 50%. That is enough for virtual shares with the same boundary rounding the series already uses.

For cash, margin level is a control loop that takes customer money. Doubles and a > around 50% are how two replicas disagree on whether to close. There is no haircut, no initial-versus-maintenance schedule. Only a single 50% line on raw unrealized PnL.

What I would do: int64 minor units or numeric for cash, PnL, and posted margin. Equity rounds down, used margin up. Maintenance is data, not a magic 50% in the loop. Haircuts as data. Never float ==.

4. Variation margin on the mark

For FTL, equity = balance + Σ unrealized PnL and washout fires when that ratio crosses 50%. Otherwise the position stays open with PnL unrealized. The 30-minute cooldown is a notification, not a debit. For a game, a toast is enough.

For cash, leaving the hole uncollected is running an uncollected receivable. A warning that cannot debit, throttled to 30 minutes, is a UX toast, not a margin call.

What I would do: write variation to the ledger on each mark, at the same durability bar as a fill. Liquidation is posted cash below maintenance after that debit. A margin-call is a demand with a clock and a legal record, not a 30-minute notification cooldown.

5. Durable decision records; isolated by default

For FTL, correctness rests on reconstructing (balance, positions, prices) for a past window. Prices are write-behind price_ticks. last_seen is a Redis key. The inputs to cascadeWashouts are not claimed persisted. One equity over all footballers; worst-PnL cascade to 50%. Reconnect replay is a rare and useful pattern for that virtual book.

For cash, “we re-ran the pure function later from Redis plus a flusher table” is not an answer to a customer dispute. Dual path (live throttled versus tick-replay versus sweeper) is two legal outcomes for the same market. Retail cash CFDs are typically isolated per position; cross / portfolio is a different product with a published close-out sequence. I would not generalize reconnect replay to any leveraged cash product.

What I would do: the mark is a sequenced event, not a flusher cache. Each evaluation writes account, seq, marks, integer equity, decision, and fill ids, committed with the money. Redis is not the system of record for last_seen or for risk. Default isolated per instrument at the live mark. Cross / portfolio is a published algorithm with negative-balance protection, sitting in the same ledger transaction as the fills.

The two designs, side by side

DecisionThis gameIf it were real money
When it runsReconnect replay (0→1), else 5-minute sweeper; live holders with 10s SET NXEvery mark, server-side, no WS dependency
PriceHistorical breach-tick (price_ticks)Live executable; slippage explicit
Who eats the gapExplicit virtual platform bucket (second build)Insurance fund, then ADL with a rule
CollateralUnrealized PnL inside equity; 30-min notification cooldownVariation posted to the ledger on each mark
Money / ratiofloat64, pnl == pnl, level > 50%int64 / numeric; conservative rounding
Marks / auditWrite-behind price_ticks + Redis last_seen; inputs not claimed persistedDurable MD log + immutable decision record committed with the money
Margin modeCross: one equity, worst-PnL cascadeIsolated by default; portfolio as a separate published algorithm

This is a fairness engine for a virtual match-day market where phones lock and goals gap footballers. The same shape, with a server-side mark consumer, live executable liquidation, integer money, variation posted on each mark, and durable decision records, is what I would take into a cash venue.

I’m open to backend/Go roles in fintech. Risk engines are where correctness matters most and is verified least. I’d love to work on yours. LinkedIn

Resources

That closes the series. All four systems (the Lua AMM, the fan-out layer, the write-behind flusher, and this engine) ran live on FTL during the 2026 World Cup. The platform is offline now.