Most of what an organisation runs was written by somebody else. Vendor software, an appliance, a system a contractor delivered four years ago, a service you rent and cannot see inside. The instrumentation advice in this field almost always assumes you can edit the code, and for a large part of the estate you cannot.
The options form a ladder, and it is worth working down it in order, because the cheap rungs are frequently sufficient.
Ask it what it already knows
Most serious software exposes something: a status endpoint, a management interface, an administrative interface with counters behind it, a metrics endpoint added at some version, or a command that prints statistics.
This is the cheapest option and it is regularly overlooked because finding it means reading documentation rather than installing something. What it gives is whatever the authors thought worth exposing, which is usually less than you want and more than nothing.
Read what it already writes
Applications write logs, files, database rows and queue entries. Parsing them requires no cooperation from the software at all, and it turns an opaque component into a source of events.
The cost arrives later and is worth stating plainly. Log formats are not interfaces. They change in a patch release, without notice, because nobody promised otherwise, and the parser breaks silently: it stops matching, produces no events, and the resulting graph is a flat line that looks like calm. Whoever writes a log parser owns it for as long as the system runs, and that maintenance is the real price of this rung.
Watch its environment
The host knows things the application will not say: process resident memory, file descriptors, thread counts, open sockets, disk queue, restarts. Restart counts in particular are a strong signal, because a process that is being restarted by a supervisor produces a healthy-looking service and a very unhealthy underlying situation.
Intercept the traffic
If the software speaks a protocol, something in front of it can count. A reverse proxy, a load balancer or a sidecar sees request rate, latency distribution and status codes for an application that was never instrumented and never will be.
This is the highest-value rung for closed systems, and it produces measurements that are close to what a user experiences rather than to what the code thinks it is doing, which the article on proxies argues is the better place to measure anyway.
Attach at runtime, with eyes open
Runtime agents can instrument common frameworks without source changes, by hooking known libraries as the program loads. Kernel-level observation can watch system calls and network activity for any process at all, with no cooperation whatsoever.
Both work and both carry the agent costs described in the article on polling and agents, with two additions. They are sensitive to versions, of the runtime, the library or the kernel, so an upgrade elsewhere can silently end the instrumentation. And they run with high privilege, which makes them a security consideration rather than a monitoring one.
The rung that is not technical
Some vendor agreements restrict attaching profilers, decompiling, or publishing performance measurements of the product. This is not hypothetical and it is not universal; it is a clause to read before deciding, particularly for appliances and licensed enterprise software. Where it applies, the remaining options are the outside ones, which is another reason to work down the ladder rather than reaching for the agent first.
Software you rent cannot be instrumented at all
For a service running on somebody else's infrastructure, none of the rungs above exist. What is available is their status page, their API, and whatever you can measure from your own side of the connection.
The status page is self-reported by a party with an interest in the answer, and it is generally updated after the affected customers already know. Measuring the dependency yourself, from your own systems, with your own credentials and against your own thresholds, is the only account of that service you control. It also produces something to put in front of them, which is a different kind of useful.
What we cannot verify
Capability and overhead claims for runtime and kernel-level instrumentation come from the projects and companies producing them, and depend heavily on workload; we reproduce none. Licence terms differ per product and per contract, and nothing here is legal advice. What a specific opaque system will reveal is answerable only by reading its documentation and trying, which is an afternoon that regularly saves a purchase.
The short version
- Work down the ladder: ask it, read its output, watch its host, intercept its traffic.
- Log formats are not interfaces, and a broken parser looks like a calm graph.
- Whoever writes the parser owns it for the life of the system.
- A proxy in front gives rate, latency and status without touching the code.
- Runtime and kernel agents are version-sensitive and privileged.
- For rented software the only account you control is the one you measure yourself.