News data for research

The problem

A finding built on news data is only as defensible as the description of how the sample was drawn, and most news tooling is built for monitoring rather than for research. Monitoring tools optimise for recency, personalise results, rerank continuously and change their corpora without notice, all of which are reasonable product decisions and all of which make a result impossible to reproduce. The practical consequence appears at review: a reviewer asks how many articles the query returned in March, the same query is re-run, a different number comes back, and there is no way to tell whether the corpus changed, the ranking changed, or the original count was wrong. That is not a marginal methodological complaint — it is the difference between a finding and an anecdote.

How does a news API support research on news coverage?

Treat the query as part of the method and write it down with the same care as the analysis. Bound every window explicitly with timestamps rather than relative expressions, so the window is a fact rather than a function of when the code ran. Fix the outlet set where the research question concerns coverage rather than events, and hold it constant across every comparison. Decide deliberately whether you want articles or collapsed stories: article-level data is the right unit for studying framing, syndication and propagation, and story-level data is the right unit for counting events. Mixing them within one study is the most common structural error in this literature, and the two units answer genuinely different questions.

What the workflow looks like

1. Write the research question as a unit first: are you counting events, counting articles, or comparing wording? The endpoint follows from that and not the other way round.
2. Bound every window with explicit timestamps, and store them with the results rather than in the code that generated them.
3. Fix the outlet and language sets before collecting anything, and record the reasoning for each inclusion and exclusion.
4. Collect once, store the raw response, and do the analysis over the stored copy rather than re-querying — a re-run is a different sample.
5. Record the snapshot date, the query text and the parameters alongside the dataset, so a reviewer can see what was asked as well as what came back.
6. Report what the sample is not: which languages, which outlets and which formats were outside the frame.

What will a news API not do for research on news coverage?

Responses carry metadata, a headline, a snippet and a link, not article bodies, so any method that requires full text — close reading, discourse analysis, most computational text analysis of any depth — cannot be run on this alone and needs text licensed from publishers directly. The index is also not a census of the press. It covers publishers it has reached, which skews toward outlets with a stable web presence and against print-only titles, broadcast, paywalled archives and anything published primarily on a platform. Coverage depth varies by language by more than an order of magnitude, so cross-language volume comparisons measure the corpus at least as much as they measure the world.

The mistake teams make here

The recurring error is comparing volume across languages or markets without normalising for corpus size, and then interpreting the difference substantively. A language with two hundred indexed outlets will produce more articles about any subject than one with twenty, and the ratio between them is a property of the index rather than of public attention. The same error appears in time series whenever the corpus grows: a rising count across three years is partly the index adding publishers. Normalise against total volume in the same language and window, state that you have done so, and be suspicious of any finding that only survives the unnormalised version.

How would you know research on news coverage is working?

Validate the frame before you trust any number computed inside it. Pick a period and a market you can check independently — a set of events you know occurred, or a publication whose output you can enumerate — and measure what fraction of it the index actually holds. That fraction is your coverage rate, it varies by market, and quoting it in the methods section is the difference between a bounded claim and an unbounded one. Then test stability: re-run an identical bounded query a month later and compare the counts. A small difference is expected as late indexing settles; a large one means your window is too close to the present and should be pushed back before analysis.

What to build in the first week

Do not build a collection pipeline in the first week. Pick the narrowest version of your question, collect one bounded window by hand, and read a hundred of the results properly. What you are looking for is not a result but a sense of what the sample contains: how much of it is syndication, which outlets dominate, whether the retrieval is returning the concept you meant, and where the language boundary bites. Almost every methodological decision you will have to defend later — unit of analysis, outlet set, normalisation, the honest description of the frame — is easier to make after reading a hundred items than after collecting a hundred thousand.

What are the alternatives for research on news coverage?

Purpose-built research corpora offer documented sampling frames, stable identifiers and text you may analyse, which is what a peer-reviewed method usually requires, at the cost of recency and breadth. National library and publisher archives are authoritative for a single market and difficult to compare across markets. Event databases code events rather than coverage, which is the right unit if the research question is about what happened rather than about how it was reported. This is the fast, broad, multilingual layer for exploratory work, for building a sampling frame, and for questions about coverage itself — and it should be cited as a news index rather than as a press census.

The queries that implement it

• Query an exact window, down to the minute — /search in keyword mode
• Compare how two markets covered the same event — /search in semantic mode
• Follow a topic in a specific language only — /stories in semantic mode

Each recipe makes the case for its parameters. Response fields, failure modes and scheduling cost are in the technical documentation.