Case Study · DEFCON Demo · Cyber Resilience
Hobby Project — The Art of the Graceful Fail
A weekend project built for a conference demo turned into a working proof of cyber resilience across a realistic three-tier stack. This page captures what it is, what it proved, and where it honestly falls short.
The thesis · Assume breach
Cyber resilience is not a taller wall. It is a city that can rebuild itself while it is still on fire.
Prevention asks how to keep the attacker out. Graceful-fail assumes the attacker is already in and asks how fast the system can rebuild itself around them. If recovery is faster than the attacker's velocity through the stack, the breach stops paying — whether or not the initial intrusion was blocked.
The B-V-R equation
B · Bleed Rate
$5,000 / min
FAIR-derived from a $3B-revenue benchmark, conservatively rounded down.
V · Attacker Velocity
30 s
To full-stack compromise — the fastest breakout time in CrowdStrike's Global Threat Report.
R · Recovery
~4.6 s avg
Measured rollup across the stack, slowest DB tier included.
R < V
The win condition — a ~6.5× safety margin against the 30-second velocity ceiling.
The five-container stack
Five real Docker containers on an isolated bridge network. A watchdog polls each tier every second, detects drift, and triggers a tier-specific recovery: respawn for stateless, kill-and-restart for stateful, fence-and-resume for data. Nothing is mocked — verify it all with docker ps, curl and psql.
WAF
ModSecurity + OWASP CRS — blocks in-flight, returns real HTTP 403s
WEB
nginx:alpine (stateless) — respawns from an immutable golden image on drift
APP
Flask API on Python 3.11 — stateful: in-memory sessions, DB connections
DB primary
PostgreSQL 15 — primary
DB replica
PostgreSQL 15 — streaming WAL-replicated
Measured recovery — not estimated
~0.63 s
WEB wipeout (rm -rf web root) — respawn from golden image
~0.73 s
APP process kill — restart + session reset
~12.56 s
DB ransomware (T1486 bulk corruption) — WAL replay pause → auto-resume
13 / 15
WAF probe campaign — blocked in flight with 403, 0 exploits succeeded
The uncomfortable truths
The slide deck and the kit both carry these forward. Without them, the demo is a lie.
01 · The DB fence catches ransomware ~500 rows late
A single bulk INSERT lands inside one 1-second tick. Real defence needs sub-100 ms change-data-capture. The demo proves detection and containment, not bleed-zero.
02 · Fence-and-resume, not fail-over
pg_wal_replay_pause + auto-resume is what gets measured. pg_ctl promote is intentionally manual; the honest upgrade target is Patroni or Stolon.
03 · WAF block rate was 86%, not 100%
One probe hit a non-existent route and returned 404. Zero exploits succeeded — but it is not a 100% block claim.
04 · Single-host deployment flatters MTTR
Sub-second recovery is partly loopback networking. Across hosts and AZs the pattern survives — the numbers don't.
Immutable golden images — and their supply-chain risk
Respawn only works if the source of truth is clean. If the base image, Dockerfile, or CI pipeline is compromised, every respawn ships poisoned. Cosign, SBOMs, and immutable registries are the fix — and the next CTF challenge in the kit.
Recorded Project Demonstration
Watch the recorded B-V-R attack-and-recovery demonstration. This looping visual shows how Graceful Fail detects disruption, recovers each tier, and measures the result. It is a recorded project demo, not a live dashboard.

What to watch
Watch for the attacker entry, the tier-specific response, and the final recovery verdict. The demo makes the B-V-R model visible: recovery stays ahead of attacker velocity.
