What no monitoring tool can measure, in either market

By Readout editorial teamMeasurement6 min read

Every monitoring product is sold on what it can see. The more useful inventory is the other one, and it is largely the same in both markets this site covers, because both are limited by the same thing: a measurement requires an event, and the most important facts are usually not events.

Nothing records what did not happen

A monitoring system observes occurrences. It has no channel at all for non-occurrences, and several of the things worth knowing are exactly that.

The user who found the page slow, gave up, and never came back generates no request, no error and no support ticket. The metric that describes them is the absence of a metric. A dashboard showing healthy traffic and zero errors is fully compatible with steadily losing the people who would have been the traffic.

The same hole exists on the other side. The incident that was prevented, the outage that did not occur because somebody checked something on a Friday afternoon, and the bad decision that was talked out of the room all leave no trace. Prevention is invisible to measurement by construction, which means the people who do it most are systematically under-recorded by any activity-based system.

Correct is not the same as available

Availability monitoring establishes that a system responded. It does not establish that the response was right. A service can return two hundred status codes, at low latency, with entirely wrong numbers in the body, and every dashboard will be green throughout.

Correctness has to be asserted explicitly, by checks that know what the answer should be, and those checks have to be written by somebody who thought of the case. Which leads to the second structural limit.

You can only ask questions you thought of

A metric exists because somebody decided to record it. An alert exists because somebody predicted a failure mode. Novel failures, by definition, are the ones nobody predicted, and they are the expensive ones.

This is the real argument for keeping wide, high-detail data that supports exploration rather than only pre-computed indicators: not because exploration is elegant, but because the question that matters during an incident is frequently one nobody had written down. It is also why the retention decisions in the previous article are not a housekeeping matter.

The workforce list is longer and sharper

Reading. Thinking. Mentoring somebody through a problem, which registers as two people producing nothing. The decision not to build something, which is often the most valuable output of a week. Judgement exercised in a meeting. Whether the work was any good.

None of these produce the events an endpoint agent can capture, and all of them are what senior people are paid for. A system that scores activity will consistently rate the most experienced person in a team below the least, and it will do so with a confident number attached, which is worse than saying nothing.

Searching where the light is

The pattern behind all of this has a name outside monitoring: looking for the keys under the streetlamp because that is where you can see. It is not stupidity. Looking where you can see is rational, and it becomes a problem only when the search under the lamp is reported as a search of the whole street.

The practical form of the error is that measurable things acquire weight simply by being measurable. A meeting about performance discusses the numbers that exist, and the effect on anything not instrumented is absent from the conversation rather than argued down.

The thing no dashboard shows is its own cost

Monitoring costs money, attention and, in the workforce case, trust. None of those three appear on any dashboard the system produces, and the system will never report that it is not worth having.

That asymmetry is worth naming because it is structural rather than anyone's fault: the benefits are instrumented and the costs are not, so the recorded evidence always favours keeping and extending the system.

The practical response

Write the list. An explicit, maintained note of what the monitoring does not cover, kept beside the dashboards, does two things: it stops the measurable crowding out the important in a review, and it tells the next person what the green screens do and do not mean.

Then route the uncovered questions somewhere else deliberately, to conversation, to sampling, to reading the work. Those are methods, not failures to automate.

What we cannot verify

This article makes an argument rather than reporting measurements, and the argument is about limits that by their nature produce no data. Claims that particular products capture qualitative work, judgement or "deep work" come from those products' vendors and rest on proxies of their own choosing. We have tested nothing and reproduce none of their figures.