Home / Recipes / Track what is being written about your competitors

Worked query

Track what is being written about your competitors

/stories in hybrid mode. The parameters matter less than the reasoning behind them, so that is most of what is below.

The request

HTTP
GET /stories
  ?q=<competitor name>
  &mode=hybrid
  &from=<last poll>

# one query per competitor, never an OR query

Why these parameters

One query per competitor rather than a combined OR query. You need to attribute every result to a company, and a merged query makes that ambiguous the moment two of them appear in one article — which, for competitors, is often.

The obvious approach, and why it loses

The obvious approach is one query with every competitor joined by OR, because it is a single request instead of thirty. It breaks attribution the moment two of them appear in the same article, which for direct competitors is common rather than rare — funding rounds, market-share pieces and analyst round-ups name several at once. You can attribute afterwards by string-matching the names in the result, but that reintroduces the keyword problem you avoided by using hybrid mode, and it is wrong in exactly the cases that matter, where a competitor is referred to by a short form or a product name.

When it returns too much, or too little

Too much on a single competitor usually means the name is also a common word or a product category, and the fix is a disambiguating term rather than a tighter mode. Too little is worth reading carefully: for a private company below a certain size, an empty week is genuinely uninformative, and treating it as a finding is the most common analytical error made with this recipe. Cost is the other tuning dimension nobody plans for — a list of thirty names polled hourly is over twenty thousand requests a month before anyone has run an ad hoc search. Poll the tier that matters hourly and the rest daily. A watch per company runs the same schedule server-side, and each run still counts as a request.

How you would know it is working

Grade the output the way you would grade an analyst: how much of it was worth forwarding. Sample a fortnight, mark each item as acted on, filed or ignored, and look at the proportion rather than the total. A competitive feed that nobody forwards is not informing anyone regardless of its coverage. Separately, keep a short list of things you learned about a competitor from somewhere else and check whether this feed had them first. That is the only measurement that distinguishes a monitor from a comfort blanket, and it is worth running once a quarter even when the feed looks healthy.

When this is the wrong tool

One query per competitor scales linearly with the watch list, and a list of thirty companies polled hourly is a substantial share of a month’s requests before anyone has searched for anything. Cost the list before building it. This is also the wrong tool for private companies below a certain size, where coverage is sparse enough that absence carries almost no information — a quarter with no results may mean nothing happened or may mean nobody covered what did. And it will not tell you what a competitor is planning, only what has been written about them, which is generally the part they have chosen to make public.

Running it for real

Response fields, failure modes, adaptations and scheduling cost: Track what is being written about your competitors, in the docs.

Related recipes