Recipes / Stop one wire report arriving as forty results
Stop one wire report arriving as forty results
/stories in hybrid mode. The parameters matter less
than the reasoning behind them, so that is most of what is below.
The request
GET /stories
?q=<topic>
&mode=hybrid
&from=2026-07-29Why these parameters
Syndication is structural, not incidental. A wire report gets picked up by hundreds of sites, lightly re-headlined, within minutes. Every one of those is relevant by any ranking measure, so no amount of relevance tuning removes them — the fix has to happen at the response shape. Collapsing by story does that, and it is why the endpoint differs rather than a parameter.
What comes back
One row per event with a count and the outlets attached. Deduplicate downstream on the story identifier, never on the headline: headlines get rewritten between editions, so headline-based deduplication silently re-admits the same story every time an editor changes a word.
How this goes wrong
Writing your own title-similarity clustering. It handles the easy cases and fails on exactly the hard one — near-identical headlines across editions of the same story are what naive similarity handles worst — and you own it forever.
Adapting it
When you do want the duplicates — studying propagation, framing shifts between wire copy and regional rewrites, or which markets picked something up — use /search instead and keep every article. That is the one case where collapsing destroys the data you came for. Everything else, including anything feeding a model or an alert, wants the collapsed form.
Running it for real
The outlet count on a story is a usable importance signal with no model behind it, and it is more robust than article volume because it cannot be inflated by one prolific publisher. If you persist results, store the story identifier as the primary key and treat articles as children of it; retrofitting that later is painful because you will have already written deduplication logic that the schema now contradicts.
When this is the wrong tool
Story collapsing is a judgement about what constitutes one event, and judgements can be wrong at the margins. Two distinct but closely related developments occasionally collapse together, and a story that evolves substantially over days may stay under one identifier when you would rather it split. If your analysis depends on exact event boundaries — legal, compliance or academic work — verify the clusters rather than trusting them, and use /search to inspect the members of any cluster that matters. This is also the wrong recipe when the duplicates are load-bearing: propagation studies, framing comparison and anything measuring reach all need the raw articles, and collapsing them destroys the measurement rather than cleaning it up.