Public opinion and narrative analysis

The problem

Aggregate sentiment across an unfixed set of publishers measures the publisher mix, not opinion. It moves when the outlets your query happened to reach change, which looks exactly like opinion shifting.

How does a news API support public opinion and narrative analysis?

Fixed outlet sets, per-language queries and tone available per article, so comparisons are made within a stable frame rather than across a moving one.

What the workflow looks like

1. Fix the outlet set first; treat it as part of the method, not a filter.
2. Compare within that set over time rather than across sets.
3. Partition by language and market — a single figure across a language spanning several countries is not meaningful.
4. Report volume alongside tone; a tone shift on ten articles is noise.

What will a news API not do for public opinion and narrative analysis?

Tone is computed from wording, not from meaning or intent, and press coverage is not public opinion — it is what publishers chose to print. Treat this as a measure of media narrative and pair it with actual survey data before drawing conclusions about people.

The mistake teams make here

The recurring error is presenting a coverage measure as an opinion measure, usually not deliberately. A chart labelled sentiment over time, built from press coverage, will be read as what people think, and it will be quoted that way by someone who never saw the methodology. What it actually measures is what publishers chose to print and in what register, which moves with editorial decisions, news cycles and the composition of the outlet set. The second error compounds it: computing the series over an unfixed set of publishers, so the figure also moves whenever the query reaches a different mix of outlets — a change that is indistinguishable from a change in the thing being measured.

How would you know public opinion and narrative analysis is working?

Be explicit that you are measuring coverage, not opinion, and validate against something that measures opinion. Where polling exists for the same period, check whether the coverage series moves with it; where it does not, you have learned that the two diverge, which is itself a finding. A narrative measure presented as a public-opinion measure will eventually be quoted as one.

What to build in the first week

Fix the outlet set first and write down why each publication is in it, because that document is the method and it is what a reviewer will ask for. Then compute one series over a period where an independent measure of opinion exists — a published poll, an election, a referendum — and compare. Whether the two agree or diverge, you have learned something you can state, and you will have established at the outset that the two are different quantities. Starting with the comparison rather than adding it later is what keeps the labelling honest once the series is in front of people who want it to mean more than it does.

What are the alternatives for public opinion and narrative analysis?

Survey research measures what people think and this cannot. Social listening measures what people say, which is closer but skews to whoever posts. Academic media-analysis corpora offer better provenance and much worse recency. This is the fast, broad, cross-language layer for how a subject is being framed in the press, and it should be labelled as exactly that.

The queries that implement it

• Find coverage with a particular tone — /search in hybrid mode
• Compare how two markets covered the same event — /search in semantic mode
• Track a slow-moving topic without drowning in it — /stories in semantic mode

Each recipe makes the case for its parameters. Response fields, failure modes and scheduling cost are in the technical documentation.