Section Alerting
Alerting, on-call and noise
An alert is a request for a human. Most systems make far more of them than they can justify.
Teams applying this principle can also compare practical guidance on employee time tracking app, keeping time and activity records separate from the judgement they are meant to inform.
This section is about the part of monitoring that reaches a person. A dashboard waits to be looked at. An alert arrives, and arriving is the whole of what distinguishes it.
The articles cover what an alert has to justify before it is allowed to interrupt anyone, why thresholds set once decay into noise, what alert fatigue actually is and how to measure it, how escalation should be arranged and who decides, what an alert should carry with it, the maintenance of deleting alerts nobody schedules, and how to review an incident without turning it into an inquiry about a person.
The connection to the first section is direct. An alert is a proxy with a consequence attached, and the consequence is somebody's night. Everything the measurement section says about distance from the thing, base rates and the cost of a threshold applies here with a person on the receiving end.
Nothing in this section names a product. The selections do that, once the class of problem is settled.
Published
- 01
An alert is a request for a human and should justify itself
Interrupting somebody is the whole of what distinguishes an alert, and most alerts in most systems cannot justify it.
Read - 02
Static thresholds drift into noise, and why
Nothing announces that a threshold has expired, and the only maintenance anyone performs on one is raising it at three in the morning.
Read - 03
Alert fatigue is a measurable failure, not a mood
A predictable response to a signal that is usually wrong, studied where the stakes are lives, and computable from records you already hold.
Read
Everything in this section
- 6 min readAn alert is a request for a human and should justify itself
- 6 min readStatic thresholds drift into noise, and why
- 6 min readAlert fatigue is a measurable failure, not a mood
- 6 min readEscalation: who is woken, in what order, and who decides
- 6 min readWhat an alert should carry with it
- 6 min readDeleting alerts is the maintenance nobody schedules
- 6 min readIncident review, and the difference between cause and blame
Further context
For a primary, standards or institutional reference, see RFC 5424 on syslog.
Start from the class of problem, not the list of tools
Every selection here names the situation first and the criteria second. Product names come last, and each one carries the line describing what it costs you.