Signal Drop: The Six-Week Decay

July 4
6 mins

View Transcript

Episode Description

Automate incident triage, and your best responders quietly lose the reps that kept them sharp, until the automation paradox meets a novel fault.I once watched one of the best incident teams I have worked with get quietly worse over about six weeks, and every change we made was, on paper, an improvement. We automated triage. The routine got handled, fewer people were woken, and the numbers looked great.Then a novel fault arrived, the kind that is in no runbook, and the team stalled. The automation had never seen it either, and the humans were rusty, because we had taken away their reps. This episode is about the hidden price ofautomating the routine, and how to keep your team's edge on purpose.The habit: run one live incident drill the automation cannot help with. A novel fault, no runbook, your best people, cold.The book is out. Metrics & Mayhem: A CTO's Guide to Observability That Actually Works. Kindle is live now; paperback and hardback launched on 1 June.Get your free chapter here: https://www.masteringobservability.com/metrics-and-mayhem/free-chapterNewsletter: https://masteringobservability.comLinkedIn: https://www.linkedin.com/in/allanmann1/

See all episodes