Surviving Flash Sales: Scaling E-Commerce Ingestion and Inventory Counters with Redis Locking

For CTOs and engineering leaders running commerce platforms, a flash sale is the one day your architecture is audited in public. Traffic does not ramp; it steps. A newsletter send, an influencer post or a limited drop turns a few hundred requests per second into tens of thousands inside a minute, and every request wants the same few rows: the stock count of a handful of hot SKUs.

Three failures show up repeatedly in post-mortems:

1. Overselling. Two checkouts read stock = 1, both decrement, and the warehouse is now short one unit and the support queue is long by one angry customer. At scale this is hundreds of units, not one.
2. Row-lock collapse. The relational database serializes every UPDATE inventory SET qty = qty - 1 WHERE sku = 'X' behind a single row lock. Throughput on that SKU is capped by lock hand-off latency, and the connection pool fills with waiters until unrelated requests (login, search, cart) time out too.
3. Ingestion pile-up. Marketplace feeds, ERP stock syncs and webhook storms arrive at the same moment as shoppers. If ingestion writes through the same path as checkout, the sale starves the feeds, and stale feeds then overwrite fresh counters.

This article lays out an architecture that handles all three: Redis as the hot-path inventory authority with atomic Lua scripts, a reservation model with expiry instead of long-lived locks, a queue-backed ingestion pipeline, and a reconciliation loop that keeps the relational database as the durable system of record. Numbers below are worked examples with stated assumptions, not measurements from a specific production system.

Why the Database Is the Wrong Place for the Hot Counter#

A typical first implementation looks like this:

BEGIN;
400 font-semibold">SELECT qty 400 font-semibold">FROM inventory 400 font-semibold">WHERE sku = 400 font-semibold">class="text-emerald-300">'400 font-semibold">DROP-001' FOR 400 font-semibold">UPDATE;
-- application checks qty > 0
400 font-semibold">UPDATE inventory SET qty = qty - 1 400 font-semibold">WHERE sku = 400 font-semibold">class="text-emerald-300">'400 font-semibold">DROP-001';
400 font-semibold">INSERT INTO orders (...);
COMMIT;

SELECT ... FOR UPDATE is correct, and that is the problem: it is correct because it serializes. Each transaction holds the row lock for the full round trip including application logic and the order insert. If that window averages 8 ms, one SKU can complete roughly 125 transactions per second, regardless of how many app servers you add. A drop that attracts 5,000 buyers per second on one SKU is 40 times over that ceiling, and the excess does not fail fast; it queues, holding connections.

The optimistic alternative (UPDATE ... WHERE qty > 0 and check the affected row count) removes the explicit read and shortens the lock window, but the row is still a single serialization point. Sharding the counter across several rows (bucketed stock) helps, at the cost of complicated "last unit" logic and uneven bucket exhaustion.

The better move is to take the contention off the database entirely. Redis executes commands on a single thread, so operations on one key are inherently serialized, but each takes microseconds rather than milliseconds, and there is no connection-held transaction. A single Redis primary commonly sustains six figures of simple operations per second on modest hardware, which gives the hot path the headroom the database cannot.

The Core Primitive: Atomic Decrement With a Floor#

The simplest correct Redis operation is DECR. But a bare DECR has no floor: it happily goes to -37. Two patterns are safe.

Pattern 1: decrement and compensate. Call DECR; if the result is negative, call INCR to give it back and reject the request. It is atomic per command and works, but between the two calls the counter is transiently negative, which can confuse concurrent readers and make "sold out" display logic flicker.

Pattern 2: a Lua script. Redis runs a script atomically; no other command interleaves. This is the pattern to default to:

-- KEYS[1] = stock:{sku}
-- ARGV[1] = quantity requested
local qty = tonumber(ARGV[1])
local stock = tonumber(redis.call(400 font-semibold">class="text-emerald-300">'GET', KEYS[1]))
400 font-semibold">if stock == 400">nil then
  400 font-semibold">return -2            -- key missing: counter not warmed
end
400 font-semibold">if stock < qty then
  400 font-semibold">return -1            -- insufficient stock
end
400 font-semibold">return redis.call(400 font-semibold">class="text-emerald-300">'DECRBY', KEYS[1], qty)

Load it once with SCRIPT LOAD and invoke with EVALSHA to avoid resending the body. Return codes matter: distinguish "sold out" (-1) from "counter not initialized" (-2) so a cold cache never silently reads as zero stock, which is its own kind of outage.

Two operational constraints apply. Scripts must be short, because they block the server while they run. And on Redis Cluster, every key a script touches must hash to the same slot; use hash tags such as stock:{DROP-001} and reserve:{DROP-001} so related keys co-locate.

Reservations Instead of Long-Held Locks#

Decrementing at "add to cart" oversells nothing but strands stock in abandoned carts. Decrementing at "payment captured" lets thousands of shoppers race through checkout for units that no longer exist. The pragmatic middle is a reservation with a TTL: stock is held at checkout start and released automatically if payment does not complete.

Model it with the counter plus a per-reservation key:

-- KEYS[1] = stock:{sku}
-- KEYS[2] = reservation:{sku}:{reservationId}
-- ARGV[1] = qty, ARGV[2] = ttl seconds
local qty = tonumber(ARGV[1])
local stock = tonumber(redis.call(400 font-semibold">class="text-emerald-300">'GET', KEYS[1]) or 400 font-semibold">class="text-emerald-300">'-1')
400 font-semibold">if stock < qty then 400 font-semibold">return -1 end
400 font-semibold">if redis.call(400 font-semibold">class="text-emerald-300">'EXISTS', KEYS[2]) == 1 then 400 font-semibold">return -3 end  -- idempotent retry
redis.call(400 font-semibold">class="text-emerald-300">'DECRBY', KEYS[1], qty)
redis.call(400 font-semibold">class="text-emerald-300">'SET', KEYS[2], qty, 400 font-semibold">class="text-emerald-300">'EX', tonumber(ARGV[2]))
400 font-semibold">return stock - qty

Note the EXISTS check: checkout clients retry, and a retry must not reserve twice. The reservation id should be derived from the cart or idempotency key, not generated per attempt.

On payment success, delete the reservation key and enqueue the durable order write. On payment failure or abandonment, the key expires, but expiry alone does not return stock to the counter, because Redis TTL expiry does not run code. Two common solutions:

• Keyspace notifications (notify-keyspace-events Ex) on expiry, with a worker that re-increments. Simple, but notifications are fire-and-forget; a disconnected subscriber misses events.
• A sorted-set reaper. Add each reservation to ZADD reservations:expiry <deadline> <member>, and run a worker that atomically pops due members (ZRANGEBYSCORE plus a Lua release script) and returns the stock. This survives worker restarts and is the more robust choice at scale.

Either way, the release script must be idempotent: it should only increment if it successfully deletes the reservation key.

When You Do Need a Lock: Redlock Realities#

Not everything is a counter. Some flows genuinely need mutual exclusion: applying a one-per-customer promo, reserving a bundle across several SKUs, or running a reconciliation pass for one SKU at a time. Here the standard single-instance pattern is:

SET lock:{sku} <random-token> NX PX 5000
-- work --
-- release only 400 font-semibold">if token matches:
400 font-semibold">if redis.call(400 font-semibold">class="text-emerald-300">'GET', KEYS[1]) == ARGV[1] then
  400 font-semibold">return redis.call(400 font-semibold">class="text-emerald-300">'DEL', KEYS[1])
end
400 font-semibold">return 0

Three rules prevent the usual bugs: use a random token per holder, set a TTL so a crashed holder cannot block forever, and release with a compare-and-delete script so you never delete someone else's lock after your own expired.

The harder question is correctness under failure. A lock with a TTL is a lease, and leases can expire while the holder is paused by garbage collection or a network stall. The holder then acts on a lock it no longer owns. Multi-node Redlock reduces the chance of lock loss on node failure but does not remove this pause hazard. The practical guidance for engineering leaders:

Ingestion That Cannot Be Allowed to Hurt Checkout#

The second half of a flash-sale failure is rarely the checkout itself. It is the feed traffic: supplier stock files, marketplace order webhooks, ERP deltas and price updates arriving in bursts. The design principles are:

• Decouple with a durable queue. Accept the webhook, validate the signature, write the raw payload to Kafka, SQS or Redis Streams, and return 202 in milliseconds. Do the processing in consumers whose concurrency you control.
• Make every message idempotent. Providers redeliver. Store an event id (or a hash of source plus sequence) in a SET key NX EX 86400 guard before applying, so duplicates are dropped.
• Version, don't overwrite. An ERP snapshot that was generated 90 seconds ago must not clobber a counter that has since sold 400 units. Prefer deltas (+50 received) over absolute values, or carry a monotonic version and reject stale absolutes in Lua:

-- KEYS[1] = stock:{sku}, KEYS[2] = stockver:{sku}
-- ARGV[1] = absolute qty, ARGV[2] = source version (integer)
local cur = tonumber(redis.call(400 font-semibold">class="text-emerald-300">'GET', KEYS[2]) or 400 font-semibold">class="text-emerald-300">'0')
400 font-semibold">if tonumber(ARGV[2]) <= cur then 400 font-semibold">return 0 end   -- stale, ignore
redis.call(400 font-semibold">class="text-emerald-300">'SET', KEYS[2], ARGV[2])
redis.call(400 font-semibold">class="text-emerald-300">'SET', KEYS[1], ARGV[1])
400 font-semibold">return 1

• Separate lanes and apply back-pressure. Run feed consumers on their own worker pool and queue, with a rate limit, so a 2-million-row catalog sync cannot saturate the database connections that checkout needs. During the sale window, consider pausing non-critical bulk syncs altogether.

Keeping the Database Honest: Write-Behind and Reconciliation#

Redis is fast, but it is not where finance, fulfilment and audit should look. The relational database remains the system of record, updated asynchronously through an outbox-style path:

1. The checkout service completes the Redis reservation conversion and publishes an OrderPlaced event to a stream (or writes it to a transactional outbox).
2. A consumer persists the order and decrements durable inventory in a normal transaction, at a rate the database can sustain.
3. A reconciliation job compares stock:{sku} against durable_qty - open_reservations and alerts on drift beyond a threshold.

Redis persistence settings deserve a deliberate decision. With AOF appendfsync everysec, a crash can lose up to about one second of writes. For an inventory counter that is usually acceptable because of step 3: on recovery you rebuild the counter from the database minus unconverted reservations, rather than trusting the restarted cache. Keep a documented, tested "warm the counters" runbook. A Redis failover that returns an empty keyspace and a counter script returning -2 is the safe failure; one that returns zero stock silently is not.

For availability, run a primary with replicas under Sentinel or a managed equivalent. Remember replication is asynchronous: a failover can drop the last acknowledged writes. The WAIT command can raise durability for specific operations but does not make Redis a strongly consistent store. Design the reconciliation loop on the assumption that small drift will occur, and bias the correction toward under-selling rather than over-selling.

Hot-Key Strategy and a Worked Capacity Example#

A single hot SKU is a single key on a single shard, so a cluster does not help that SKU. Options, in order of complexity:

1. Shed load before Redis. An in-process or edge pre-check ("sold out" flag cached for a few seconds) rejects the doomed 95 percent of requests without touching the counter.
2. Split the counter. Divide 2,000 units into 8 sub-counters of 250 across shards; route each request to a random sub-counter and fall back to others on exhaustion. This scales throughput but needs "steal from neighbor" logic near the end, and can briefly show sold out while units remain elsewhere.

Worked example (illustrative assumptions, not measured): a drop of 2,000 units, 60,000 interested shoppers, peak demand of 8,000 purchase attempts per second for the first 20 seconds. With the database-lock design above, an 8 ms critical section caps the SKU near 125 orders per second, so the 2,000 units take about 16 seconds of pure serialized work while the pool backs up. With a Lua reservation script costing on the order of tens of microseconds of Redis CPU, the same 160,000 attempts are decided in a couple of seconds, the first 2,000 get reservations, and the remaining attempts receive an immediate -1. The database then absorbs 2,000 order writes over a controlled window through the queue. Your numbers will differ; the point is that the contended step moved from milliseconds to microseconds and stopped holding connections.

What to Test Before the Next Sale#

• Race tests. Fire N concurrent buyers at M units with N much larger than M and assert sold count equals M exactly, with zero negative counters.
• Kill tests. Terminate the reaper worker, a Redis replica and the primary mid-test; verify reservations are released and counters reconcile.
• Stale feed injection. Deliver an old inventory snapshot during the sale and confirm the version guard rejects it.

Decision Framework#

If you are planning a major sale or preparing a platform for one, knetwork can work with your team on an architecture review, a load-testing plan or a hands-on implementation of this pattern. Reach out through knetwork.live to book a technical consultation.