Recipes / Follow one story as coverage accumulates, without re-reading it
Follow one story as coverage accumulates, without re-reading it
/stories in hybrid mode. The parameters matter less
than the reasoning behind them, so that is most of what is below.
The request
GET /stories
?q=<topic>
&mode=hybrid
&from=<last poll timestamp>
# then, for the story you care about:
GET /search?story_id=<id>&from=<last poll>Why these parameters
Poll for what is new rather than re-querying the whole window, and keep the last-seen timestamp yourself. Once you have identified the story that matters, switching to its identifier is both cheaper and more precise than repeating a text query that may drift onto adjacent events as coverage evolves.
What comes back
New coverage only. Watching the outlet count grow is the useful signal: a story that keeps acquiring outlets after the first hour is behaving differently from one that was picked up widely and immediately, and the second is usually just distribution.
How this goes wrong
Re-running the full query each poll and diffing on headline. Editions rewrite headlines, so the diff reports the same story as new repeatedly, and the alert channel becomes noise. Diff on the story identifier.
Adapting it
The same approach works for tracking an ongoing situation rather than a single event: poll the topic query for new stories, then track the ones that matter by identifier. For a story that splits into several — a company announcement that becomes a regulatory story and a labour story — expect new identifiers rather than growth in the original, and treat that divergence as signal.
Running it for real
Store the last-seen timestamp per query, not globally, or a slow query will suppress a fast one. Keep the story identifier alongside whatever you persist so that a later re-poll can be reconciled without re-fetching. If coverage stops growing for a full day the story is usually over; continuing to poll it indefinitely is the most common source of wasted request budget in this pattern.
When this is the wrong tool
Story identifiers are stable but not permanent guarantees. A long-running situation will eventually produce new clusters as its subject shifts, so a tracker pinned to one identifier will slowly stop seeing the thing it was following without ever reporting an error. Re-run the topic query periodically alongside the identifier poll and reconcile, rather than trusting the identifier indefinitely. This recipe is also unsuited to measuring how a story was covered rather than what happened next — for that you want the full article set, not the increment, and the polling pattern here deliberately discards everything you have already seen.