A scripted check says the site is fine. The people using it say it is not. Both are reporting accurately, and the disagreement is one of the more useful signals available, provided somebody knows why it happens.
They measure different populations
A synthetic check is one path, from one place, on one device, over one network, at a fixed interval, with everything else held still. That control is the point: because the variables are fixed, a change in the result means something changed in the system.
Real user measurement observes whatever your users actually have. Old phones, congested mobile networks, distant regions, corporate proxies, extensions that interfere, and a device population weighted toward whatever your customers can afford. It is not a controlled experiment and was never meant to be.
A synthetic check answers "has our system changed". Real user data answers "what are people experiencing". Those are different questions and neither substitutes for the other.
What only the synthetic check can do
It works when nobody is there. A path exercised twice a month, a checkout at four in the morning, a feature not yet launched, and the entire period before a product has users are all invisible to real user measurement, because real user measurement requires real users.
It also produces a clean baseline. With the variables held constant, a shift in the number is attributable, which is exactly what the article on baselines argues is required before a change can be judged.
What only real user data can do
It reflects the distribution rather than a point. The median user and the ninety-ninth percentile user are having different evenings, and the synthetic check is having neither of them.
It catches what happens after delivery: third-party scripts, client rendering, a device class that struggles, a region routed badly by somebody else's network. None of that is in the response the server measured itself sending.
Real user data has a hole where the worst cases should be
This is the important limitation and it follows directly from the article on what cannot be measured. Reporting from a client requires the client to have loaded enough to report. A user whose connection failed, whose device gave up, or who closed the tab after four seconds contributes nothing.
The population that reports is therefore the population that stayed, and it is biased toward better outcomes by construction. The worse the experience, the less likely it is to appear in the data describing experiences. Privacy tooling that blocks the reporting beacon removes another slice, and it does not remove a random one.
Reading the disagreement
Because they fail in opposite directions, the combination localises problems that neither identifies alone.
Synthetic healthy and real users suffering points away from the server and toward a population: a region, a device class, a browser, a third-party dependency, or something between your infrastructure and them.
Synthetic failing and real users fine usually means the check itself is broken, or is exercising a path nobody uses, or is being served differently because it is recognisable. That is a monitoring problem rather than a service problem, and treating it as an incident wastes a night.
Both moving together is the ordinary case and is the one worth alerting on.
The check becomes a special case, and then it lies
Synthetic traffic is recognisable: same address, same agent string, same account, same interval. Over time it accumulates exemptions. It gets allow-listed past a rate limiter, excluded from an experiment, routed past a cache warm-up, given a test account with different data.
Every one of those is reasonable in isolation and each moves the check further from the thing it stands in for. A synthetic check that has been exempted from six mechanisms is measuring a system no user uses, and it will be green while the real path is not.
The same disagreement, on the other side
Timesheets and captured activity disagree constantly, and the reflex is to treat the captured number as the truth because it came from a machine. It is a different population too: it observes what a device recorded, not what work occurred, and the article on what cannot be measured lists what falls outside it. The disagreement is information about both sources, and reading it as a verdict on the person is the error the workforce section returns to.
What we cannot verify
Coverage and accuracy claims for synthetic and real user products are published by their vendors under conditions they do not describe, and we reproduce none. The biases described here are structural. How large they are in a specific system depends on that system's audience and can be estimated only by comparing reported volumes against server-side counts, which is a check worth running once.
The short version
- A synthetic check asks whether the system changed; real user data asks what people experience.
- Only synthetic works when nobody is there, and only it gives a clean baseline.
- Only real user data shows the distribution and what happens after delivery.
- Clients that failed to load report nothing, so the data is biased toward better outcomes.
- Synthetic green with users suffering points at a population, not at the server.
- An exempted synthetic check ends up measuring a system nobody uses.