Home / Recipes / Monitor a market whose language nobody on the team reads

Worked query

Monitor a market whose language nobody on the team reads

/stories in semantic mode. The parameters matter less than the reasoning behind them, so that is most of what is below.

The request

HTTP
GET /stories
  ?q=<topic, in English>
  &mode=semantic
  &language=<iso639-3>
  &from=<now - 3d>

Why these parameters

Semantic mode lets you query in one language and match coverage in another, which is the only workable approach when nobody on the team can compose a query in the target language. Scope by language explicitly rather than relying on the topic to imply geography — a topic query without a language filter returns whatever the largest streams are saying about that market, which is not the same as what the market is saying about itself.

The obvious approach, and why it loses

The obvious approach is to monitor English-language coverage of the market and treat it as a proxy. It is a proxy for something, but not for what the market is reading: English coverage of a non-English-speaking country is written for outsiders, arrives later, and is selected for international interest rather than local importance. The other approach, machine-translating an English query into the target language and running it as keywords, commits to one word sense with no context to disambiguate and fails invisibly. Semantic matching from the English query avoids the commitment, and constraining by language rather than by topic is what keeps the results domestic.

When it returns too much, or too little

Too much usually means the language filter is doing less work than expected because the language spans several countries with independent news cycles. Add an outlet set for the market you actually care about; this single change does more here than any adjustment to the query. Too little is often genuine scarcity rather than a tuning problem — check the language page for the indexed outlet count before assuming the query is wrong, and widen the window rather than loosening the terms. If results are consistently about the market rather than from it, the filter being applied is probably topic rather than language.

How you would know it is working

Have twenty results read by someone who reads the language, once, and ask two questions: is this about the subject, and is this the kind of source a local reader would take seriously. The first measures the retrieval and the second measures the outlet set, and teams usually discover the second is the weaker of the two. Repeat when you change subject area, because cross-lingual matching is noticeably better on concrete technical subjects than on culturally specific ones. If nobody reads the language, buy twenty headline translations quarterly; it is a small cost against monitoring the wrong thing confidently.

When this is the wrong tool

This is not a substitute for someone who reads the language. Semantic matching across languages is good enough to surface the right material and not good enough to tell you what it says, and a translated headline loses the register, hedging and attribution that often carry the actual meaning. Teams that treat this as full coverage of a market tend to discover the gap during the one week it matters. The recipe is also weaker for languages with small streams: where a language has few outlets, a bounded query returns little and the temptation is to widen it until noise dominates. Widen the window instead, and if that still returns nothing, accept that the coverage is not there rather than loosening until something appears.

Running it for real

Response fields, failure modes, adaptations and scheduling cost: Monitor a market whose language nobody on the team reads, in the docs.

Related recipes