Recipes / Watch for regulatory action in a sector
Watch for regulatory action in a sector
/stories in hybrid mode. The parameters matter less
than the reasoning behind them, so that is most of what is below.
The request
GET /stories
?q=<sector> regulator investigation ruling fine consultation
&mode=hybrid
&from=<now - 14d>Why these parameters
Regulators publish in bursts around decisions and consultations, then go quiet, so a short rolling window returns either a flood or nothing depending on where it lands. Widen the window rather than loosening the query. Hybrid mode because regulatory language is formulaic enough for keyword matching to work on the terms of art, while the surrounding coverage describing the action is not.
What comes back
Clusters covering each action, typically the regulator statement plus trade and general coverage collapsed together.
How this goes wrong
A seven-day window and a conclusion that coverage is sparse. The publication rhythm, not the coverage, is what is sparse — this is the recipe where sampling artefacts most often get mistaken for a data problem.
Adapting it
Narrow to a specific regulator by adding its name and scoping to outlets that cover it closely, which is usually a small trade press set rather than the general press. For consultations rather than enforcement, widen the window further still — consultation periods run for weeks and coverage appears at the open and the close with nothing in between.
Running it for real
Run this weekly rather than hourly. The publication rhythm makes frequent polling wasteful, and nothing here is time-critical to the hour. Keep an eye on the outlet mix: an action covered only by the regulator's own announcement and one trade publication is at a different stage from one the general press has picked up, and that distinction usually matters more than the content of the action itself.
When this is the wrong tool
News coverage of regulatory activity is systematically incomplete. Regulators publish a great deal that no publication covers, and coverage skews heavily towards enforcement against large or newsworthy firms. If you need completeness, the regulator's own publications are the source and this recipe is a supplement that tells you which actions attracted attention. That is useful information, but it answers a different question. This is also the wrong tool for tracking a specific case through its stages: coverage clusters at announcement and resolution with nothing in between, so a case can appear dormant for months while proceeding normally.