A monitoring system does not observe your service, your team or your infrastructure. It observes a signal that stands in for one of those things, and the whole practice rests on how good the stand-in is.
A ping reports that a machine answered a packet. It does not report that the service works. A processor at forty per cent reports how busy a chip was, not whether anyone waiting on it was served in time. Hours logged report that a client was running, not that work happened. In each case the number is real, the measurement is honest, and the thing you actually cared about was never measured at all.
Why the substitution is unavoidable
The things worth knowing are not directly observable. "The service is healthy", "the team is productive", "the customer is having a good time" are states of the world with no sensor attached. What is observable is countable events: requests, errors, durations, keystrokes, packets. Every monitoring product is an argument that a particular set of countable events stands in acceptably for one of those unobservable states.
The question is never whether a tool uses proxies. It is how far the proxy sits from the thing, and what happens in the gap.
The gap has a shape, and it is worth naming
Three failures live in it, and they recur in both markets covered by this site.
The proxy can be satisfied without the thing being true. A health check that requests a static page passes while every database query behind it times out. An activity metric counting input passes while the person moves a mouse in a spreadsheet they are not reading.
The thing can be true without the proxy noticing. A service can be serving every user correctly while a synthetic check from one region fails. An engineer can spend a morning on the single most valuable decision of the quarter and register almost no measurable activity.
And the proxy can be optimised directly, which is the failure with a name.
When the measure becomes the target
Once a proxy is used to judge, whoever is being judged will improve the proxy, because that is the rational response and usually the instructed one. Improving the proxy and improving the thing are the same activity only while the two stay coupled, and pressure decouples them.
This is Goodhart's observation, stated for economic indicators and generalised since. It is not a cynical remark about people. It applies identically to automated systems: an autoscaler tuned on a proxy will also optimise the proxy, and will happily hold a metric steady while the experience it stood for degrades.
The practical consequence for a buyer is unglamorous and specific. A tool that measures something easy to influence directly will, once attached to a judgement, stop measuring what it measured during the trial.
Distance from the thing is a spectrum, not a verdict
Proxies are not simply good or bad, and ranking them is more useful than dismissing them. A request that returned the correct body to a real user sits very close to "the service works". The same request checked from a synthetic prober in one region sits further away. Process uptime sits further still. Processor load sits a long way out, because a great many states of the world produce the same reading.
The same ladder exists on the other side of the site. Work delivered and accepted sits close to "the work got done". A completed task in a tracker sits further away. Hours logged sits further. Input events per minute sit near the far end, because almost anything a person does at a keyboard produces them, including things nobody wants.
Tools compete mostly at the far end of that ladder, because distant proxies are cheap to collect and close ones need integration with the thing you actually care about.
What makes a proxy a good one
Four properties, all of which can be examined before purchase.
It moves when the thing moves, in the same direction, without long delay. A metric that lags an outage by ten minutes is a record rather than an alarm.
It is expensive to satisfy without doing the thing. Request success measured at the point the user receives a response is harder to fake than a process being alive.
Its failure mode is known. Everyone using it should be able to say what it looks like when the proxy is wrong.
And it supports a decision. If no answer to the measurement changes anything anyone would do, the measurement is decoration, and the strongest question to ask of any dashboard is which decision it exists to inform.
What we cannot verify
The mechanism described here is general and is not a measurement of any product. Vendors publish figures about their own accuracy and coverage, measured in conditions they rarely describe, and this site reproduces none of them. Whether a particular proxy is close enough for a particular system is answered by watching the two together through at least one real incident, which is a test no trial period is designed to provide.
The short version
- Monitoring observes a stand-in, never the state you actually care about.
- The useful question is how far the proxy sits from the thing.
- A proxy can pass while the thing fails, and fail while the thing is fine.
- Attach a proxy to a judgement and it will be optimised directly.
- A good proxy moves with the thing and is expensive to satisfy without it.
- If no result would change anyone's action, the measurement is decoration.