What a broken listening tool looks like from the inside
3 min readMax Hamal
For four days this product collected nothing from three of its five networks. Reddit, LinkedIn and Threads returned an error on every single attempt. Nothing alerted, nothing turned red, and the app looked exactly as it had the week before.
That is the part worth writing down. A listening tool that breaks does not look broken. It looks quiet.
The shape of the failure
Here is what the scan log held when somebody finally looked:
| Source | Attempts | Failed | Mentions collected |
|---|---|---|---|
| YouTube | 145 | 0 | 841 |
| X | 180 | 125 | 55 |
| 180 | 180 | 0 | |
| 145 | 145 | 0 | |
| Threads | 144 | 144 | 0 |
469 consecutive failures across three networks. The error was the same every time: platform-feature-disabled — Monthly usage hard limit exceeded. A scraping account had hit its cap, and every request after that was refused in a few milliseconds.
Fast, consistent, and invisible. A slow failure gets noticed because somebody waits. An instant one just returns.
Why nobody noticed
Because every question the product knew how to ask had a reassuring answer.
The scan cron ran on schedule and returned HTTP 200 — it had, after all, successfully attempted every scan. The dashboard showed mentions, because YouTube was still working. The mention count went up. Nothing in the interface distinguished a week when the internet was quiet from a week when we stopped listening.
That distinction is the entire job.
The check that would have caught it
Not "did the scan run" — it did, 469 times. The useful question is narrower:
Has any source failed every attempt for a day, across at least three attempts?
Three attempts, because one failure is a blip that retries fix on their own, and a warning about a blip teaches people to ignore warnings. Every attempt, because a source that sometimes works is a different problem from one that never does.
Written as SQL against the job log, it is about six lines. It had never been written, because the product had been built to report success and nobody had asked it to report the absence of anything.
What this means if you are choosing a tool
Ask the vendor what happens when one of their sources goes down. Not whether it can — it will; every one of these products depends on scrapers and third-party APIs that rate-limit, change their markup, and revoke keys.
Ask what you see when it happens. If the answer is "the feed gets quieter", you are buying a tool that cannot tell you the difference between silence and blindness.
The honest version of this product now sends a letter when a source has failed all day, and says which one, and says your other sources are still collecting. It is a small feature. It is also the difference between monitoring and the appearance of monitoring.
The uncomfortable footnote
The customer paying for five networks was receiving two, for the entire duration of their subscription. They had not complained, because from their side there was nothing to complain about — the feed had mentions in it.
That is the strongest argument for building the alarm before the feature. A product that fails loudly is easier to trust than one that has never been seen to fail.