Two failures this week that look unrelated and are the same failure.
The frozen number
We publish live coverage figures for our property index. The whole point is that they are fetched from production on render rather than typed into a marketing page by a person who wants them to be bigger.
They were fetched from production. Production was reading them out of a cache table that a human had to remember to refresh.
So the harvester kept working, the index kept growing, and the number the world could see stopped moving for twenty days. When we finally refreshed it against the live index, the real figure was several times what we had been showing. We were under-reporting, which is the only reason nobody complained — and exactly why nobody caught it.
The fix is not "remember to refresh it." The fix is that the refresh is part of the harvest now. A number that depends on somebody's memory is a number that is going to be wrong, and the direction it's wrong in is luck.
The app that was talking to a ghost
Separately, our Real Estate app spent six days reporting that the live property index was offline. The index was up the entire time, serving tens of millions of records to anything that asked.
An overnight install had replaced the application bundle underneath a copy that was still running and had never been relaunched. The process kept executing a binary that no longer existed on disk. It was, in every meaningful sense, a ghost: a program running from a file that had been deleted out from under it, reporting the state of a world that had moved on.
A full quit and relaunch fixed it in seconds. Both bundles carried the same version number, which is why nothing flagged it — and which is the actual finding. A version number is not an identity. If you want to know what is running, you have to ask the running process what it is, not ask the folder what it contains.
The shared lesson
In both cases every underlying system was healthy. The harvester was harvesting; the index was serving. What failed was the layer that reports on the layer that works.
That layer almost never has tests, because it feels like plumbing rather than product. It is not plumbing. It is the only thing standing between you and confidently telling people something false — on a marketing page, in a status readout, or to yourself at two in the morning while you debug a service that was never down.
We caught both of these ourselves, and we are printing both, because a company that only publishes the failures its customers already noticed isn't publishing failures at all.