The engine behind the answer: our updated audit of how AI models handle Iran in Persian

We learned that for some models we had been measuring a third-party search index rather than the model's own retrieval. So we rebuilt the study, automated, two-armed, and version-pinned, and ran it again. This single report replaces both earlier pieces.

The updated results are in some ways stronger than the originals: the search layer, not the model, explains much of what we previously described as model personality.

The prompt's vocabulary remains an outsized lever on every model we tested. And the sources AI systems cite about Iran are, to a degree we had not quantified before, funded by governments, just not usually the Iranian one.

A joint research project with Factnameh / ASL19.

A note on why we redid this: after publishing, we kept digging into how our testing platform actually routes web search and learned that for some models it uses its own search index rather than the model's. That changed how our results should be read, so we rebuilt the study, automated it, and ran it again.

This is how we want to work: the field is new, everyone is learning it in real time, and we would rather update openly than quietly. The full story is below.

Why we redid the study

In January and April we published an audit of how six AI models answer politically sensitive questions about Iran in Persian. We ran a fixed instrument of 48 prompts through OpenRouter with web search enabled, "left at OpenRouter defaults," and analyzed which sources each model surfaced and how each answer aligned with or challenged the state narrative.

The catch is in the word "defaults." OpenRouter routes a web-search request to a model's own native search only for a handful of providers: OpenAI, Anthropic, Google, Perplexity, and xAI. For every other provider, including DeepSeek and Mistral, it silently falls back to Exa, a US-based search index. Nothing in the interface tells you this has happened.

So when our original reports described DeepSeek routing users to "distinct, pre-packaged source libraries," or Mistral's "gravitational pull toward state-adjacent reference points," or the "remarkably similar retrieval pathways" of DeepSeek and Gemini, we were partly describing Exa wearing different model wrappers.

Our speculation about Baidu-centered indices and diverging retrieval stacks was built on data in which the Chinese model in the study never touched a Chinese index. The comparison mixed retrieval systems without knowing it.

There is some irony here that we want to name rather than bury. Our study was about retrieval opacity, about how AI systems route queries through infrastructure the user cannot see, and that same opacity reached us too.

We take that as the clearest possible demonstration of the problem: if a research team looking directly at this layer could not see it, ordinary users have no chance. The update is not a footnote to the research. It is the research.

Each prompt ran once, although models are non-deterministic; our re-run shows that between 4 and 21 percent of identical prompts change their alignment score on a second run. Rather than layer correction notes onto two published pieces, we took both down, rebuilt the pipeline, and ran the study again. This report replaces them.

The corrected design

The re-run keeps what was good about the original instrument: the same six topics (the 2022 protests, economics, the nuclear program, internet governance, hijab, the 2009 Green Movement), the same six seeker framings (neutral, future-oriented, leading-state, leading-opposition, verification-state, verification-opposition), the same 36 Persian prompts and 12 English baseline prompts, recovered verbatim from the original research file. Everything else changed.

Two arms, never mixed: The core design decision is to measure the two things the original accidentally blended, separately and on purpose:

• The native arm forces each model's own search. GPT-5.6 Sol, Claude Sonnet 5, Gemini 3.1 Pro, and Grok 4.6 run through OpenRouter with the search engine explicitly set to native, a setting that errors rather than falling back to Exa if native search is unavailable. Kimi K2.6 runs through Moonshot's own API with its built-in search. This arm answers: what does each provider's retrieval actually surface for Persian queries about Iran?
• The fixed-index arm gives all seven models, the five above plus DeepSeek V4 Pro and Qwen 3.7 Max, the identical search engine (Exa), explicitly and labeled. This is what the original did to two models by accident, done deliberately to all of them. It answers: when retrieval is held constant, what differences remain that are attributable to the model itself?

DeepSeek appears only in the fixed-index arm because its first-party API, as of August 2026, still offers no web search. That fact alone retires a piece of our original analysis: there was never a native DeepSeek retrieval pathway to study through an API.

Automation and repetition: A collector script runs all 48 prompts, twice per model per arm, in a single time window, with model versions pinned and every raw API response archived: 1,152 answers in total, collected on one day. Google's grounding redirects, which mask publisher domains behind expiring proxy URLs, are resolved to real domains at collection time. Every cited URL is captured programmatically, not transcribed by hand.

Deterministic source classification: A media matrix classifies every cited domain by explicit rules rather than judgment calls, keeping the published typology (state, semi-official, exile, international, NGO/academic, social, wiki) for comparability, and adding two things the original lacked. Domains that host different services split by URL path: bbc.com/persian is BBC Persian, exile-targeting and UK-funded; bbc.com is international media. And every domain also carries a control code recording who runs and funds it: Iranian state, Iranian para-state (IRGC-linked and ideological-organization agencies), Iranian licensed press, foreign-state-funded, opposition organization, independent, intergovernmental, or user-generated. Anything the matrix cannot classify stays unknown and is reported as such rather than guessed; the residual is 7.3 percent of citations, dominated by anonymous, recently registered Persian sites that we flag individually below.

Blinded scoring: Every answer was scored on the original 0-3 alignment rubric (0 declines, 1 aligns with the state narrative, 2 balanced, 3 challenges the state narrative) by an LLM judge, Claude Opus 4.5, at temperature zero, blinded to which model produced the answer. We had no capacity for a second human scorer this time either, and we state that plainly as a limitation rather than quietly living with it. The difference from the original is that the judging is now consistent by construction, fully logged, and re-runnable by anyone.

What we found

1. On identical retrieval, the models nearly converge

The single most important chart in this report is the simplest. When all seven models share one search index, their mean alignment scores land within about 0.08 of one another, between 2.42 and 2.50 on the 0-3 scale. When the five capable models use their own native search, the spread widens to 0.20, roughly two and a half times the fixed-index band: Claude at 2.61 and Grok at 2.58 on the state-challenging end, Gemini at 2.42 and Kimi at 2.45 below.

Much of what our original reports treated as model character was the search layer.

The differences that remain under identical retrieval are real but small. The differences under native retrieval are larger and belong substantially to the infrastructure each provider has plugged its model into, not to the model's disposition.

2. The prompt is still a lever; the index sets how long it is

The original study's central claim was that a slight shift in prompt vocabulary can flip an answer from human-rights documentation toward state-aligned framing. That finding survives the correction, on every model, in both arms.

We asked each topic in state vocabulary ("Tell me how Iran's enemies exploited...") and in opposition vocabulary ("Tell me why the case galvanized millions..."). The gap between the two mean scores is a direct measure of cue sensitivity. In the native arm it runs from 0.67 (Claude) to 1.08 (Kimi) points on the 3-point scale. Claude is the most stable across framings under native search; GPT's gap (0.91) is the only one that does not widen when the models move to the fixed index.

The new result is what the fixed index does to that gap. On Exa, cue sensitivity grows for almost every model, reaching 1.5 points for Gemini, DeepSeek, and Qwen. Held to one keyword-driven index, a state-vocabulary prompt retrieves state-vocabulary sources, and the models follow them. In other words, our original "resistance versus mirroring" split was not wrong, but it was not a binary trait of models either. It is a matter of degree, and the retrieval layer amplifies or dampens it.

State-aligned outputs remain rare overall: 88 of 1,152 answers, 7.6 percent, nearly identical to the 7.3 percent in the original manual study. Of those 88, 63 came from leading-state prompts.

No model refused to answer any question in either arm, which also replicates the original.

3. Native retrieval profiles differ sharply; the fixed index is a monoculture

With retrieval properly separated, the per-model sourcing profiles look nothing like a shared pattern.

GPT anchors overwhelmingly in institutional documentation: two-thirds of its citations are IGO, NGO, and academic sources, led by OHCHR, AP, Human Rights Watch, and the IMF.

Claude has the most exile-heavy native profile, led by Iran International, Wikipedia in both languages, and IranWire.

Grok is now the Wikipedia model: English Wikipedia is its largest single source by a wide margin, followed by think tanks. Gemini, in the minority of Persian answers where its grounding fires at all, cites a small, mixed set.

The fixed-index arm is the opposite: one diet for everyone. Its most-cited sources across all seven models are BBC Persian (686 citations), Iran International (247), Euronews Persian (138), and, notably, Iran's Ministry of ICT (111), with Radio Farda close behind.

4. Who pays for the sources models cite

The control dimension of our media matrix produces the finding we most wish we had been able to make in the original reports.

In the fixed-index arm, between 31 and 33 percent of every model's citations, with striking uniformity, come from foreign-government-funded outlets: BBC Persian, Radio Farda, VOA Persian, Euronews Persian, Deutsche Welle, Al Jazeera. Iranian-linked sources (state, para-state, and licensed domestic press combined) stay under 16 percent. Opposition-organization organs barely register.

The Persian information space that AI systems draw on is not captured by the Iranian state. On this evidence it is dominated, at least via this widely used index, by media funded by Western and Gulf governments for audiences in Iran. Whatever one thinks of those outlets' journalism, and much of it is serious, this is a different finding from "models resist state capture." The pluralism in AI answers about Iran runs substantially through state-funded channels on the other side of the contest. Our original typology, which filed BBC Persian and Radio Farda under "exile media" alongside independent diaspora newsrooms, made that structure invisible.

In the native arm the picture is thinner but consistent: foreign-state-funded sources at 7 to 10 percent, Iranian-linked sources near zero for GPT and Gemini, around 9 percent for Claude, which cites state documents largely to rebut them, a behavior we also saw in the original study.

5. For Persian, retrieval often does not fire at all

Two infrastructure findings would have been invisible without automation.

Gemini's native grounding activated on only 21 percent of Persian prompts, against 54 percent of the same prompts in English. For most Persian questions, Gemini answered from its training data alone, with no live sources at all. Kimi's native search fired on 84 of 96 answers, but its API exposes no citation URLs; the retrieval exists and is unauditable from the outside.

Under this study's inclusion rule, only models whose citations resolve to real publisher domains enter the source analysis, so Kimi's native arm counts for narrative scoring but not for sourcing.

The practical meaning for Persian-speaking users: enabling web search does not mean the answer you get was searched. Whether retrieval happens at all is itself unevenly distributed across languages, and mostly undisclosed.

6. One run is not a study

Because every prompt ran twice under identical conditions, we can now quantify what single-run designs, including our original, cannot see. Between 4 and 21 percent of prompts received a different alignment score on the second run, depending on model and arm.

Native-search citation sets barely repeat: the overlap in cited domains between two runs of the same prompt ranges from 13 to 31 percent. On the fixed index, overlap jumps to 78 to 88 percent. Native retrieval is live and volatile; a fixed index is stable but uniform.

Any audit built on one run per prompt, ours included, was reporting a coin flip as a fact for a meaningful share of its cells.

7. What the index drags in

Automated capture of every cited URL surfaced things a manual study would likely have missed or misread.

• Exa served lranintl.com (sic!), with a lowercase L masquerading as the capital I of iranintl.com, to all seven models. We initially treated it as a typosquat; verification showed it is Iran International's own anti-censorship mirror, running the identical deployment. The models were citing a real outlet through a domain designed to evade filtering, and any domain-level analysis that did not resolve this would have fragmented or mislabeled one of the most-cited sources in the study.
• Several anonymous, recently registered Persian sites were cited surprisingly often: a digital-rights watchdog registered in mid-2025 with no disclosed operator (28 citations), an unattributable repository of Iranian governance documents that blocks foreign IP addresses (23), and a site trading on Human Rights Watch's name with no connection to it. We left all of these unclassified and flagged. Models cite them without any of this context.
• Grokipedia, xAI's AI-generated encyclopedia, was cited four times, all by Claude, never by Grok. AI-generated reference content is beginning to circulate as a citable source across rival systems.
• One Qwen answer to a Persian question about IAEA inspections came back in Chinese, a reminder that a model's language routing can fail in ways that have nothing to do with politics and everything to do with training distribution.

What survives from our original reports, and what does not

Survives, and is now on firmer ground: the cue-sensitivity finding, the rarity of outright state-aligned outputs (7.3 percent then, 7.6 percent now), the concentration of state alignment in leading-state prompts, the absence of refusals, Claude's pattern of citing state sources to argue against them, and the observation that English prompts route to different sources than Persian ones.

Does not survive: every claim about DeepSeek's and Mistral's retrieval behavior, including the "jukebox" and "tide" characterizations, which described Exa's keyword response, not those models' own search; the DeepSeek-Gemini similarity, which compared a fixed index against a grounding system that rarely fired; and the framing of "Persian retrieval pathways" as a property of individual models. Grok's signature reliance on X also did not reappear: with native search, Grok 4.6's social-media citations are marginal and its largest source is English Wikipedia, so treat that original profile as tied to a specific model version and moment. We have retired the model nicknames altogether. They were memorable, and several described the plumbing rather than the model.

Our broader argument about contingency, that today's Western-indexed retrieval advantage could erode as different stacks route through different indices, stands, but the corrected study sharpens it: we could not measure a Chinese index at all, because the Chinese models reachable by API either have no search (DeepSeek) or, in Qwen's case, were tested here only on the fixed index. That measurement gap is itself a finding about auditability.

The blueprint, corrected

We are publishing the full pipeline so that anyone can replicate or extend this work; the original manual protocol took roughly 75 researcher-hours per snapshot, and this one takes about three hours of machine time.

The components:

• A prompt generator encoding the six-topic, six-seeker instrument in code, so the exact prompt set is versioned and reusable rather than living in a spreadsheet.
• A collector that runs per-provider modules (OpenRouter with the engine forced explicitly; first-party APIs where native search only exists there), retries on rate limits, resumes after interruption, pins model versions, captures every citation field programmatically, resolves Google's grounding redirects at collection time because they expire within days, and archives every raw response.
• A media matrix with deterministic domain-and-path classification, the published typology plus the control dimension, explicit flags on contested calls, and a review queue for everything unclassified. Build your own for your country; the categories that matter are who funds and who controls, not just where an outlet nominally sits.
• A blinded LLM judge applying the 0-3 rubric at temperature zero, with prompts and rationales logged. If you have two human scorers, use them on a stratified sample and report agreement; we could not, and say so.
• A dashboard generator that turns the dataset into the charts embedded in this report.

Practical notes for replicators: Decide which layer you are measuring before you collect anything, and never mix retrieval systems silently; if your platform offers a "default," find out what it actually does per provider. Force settings explicitly and prefer configurations that fail loudly over ones that fall back quietly. Run every prompt at least twice. Pin versions and collect in one window. Expect the API bill for a study this size to land in the low hundreds of dollars, dominated by search-enabled calls to flagship models; judging costs a few percent of that.

Limitations

This is one snapshot, on one day, of five native-search stacks and one fixed index. Model and index behavior changes continuously, and our own stability numbers show single measurements mislead; treat every figure here as an estimate with real variance, not a constant.

The judge is a Claude model while Claude is a study subject; it was blinded to model identity, and on identical retrieval it scored Claude's answers indistinguishably from its peers (2.50 against a 2.42 to 2.50 range), but the dependence is real, and we lacked a second human scorer to quantify judge error. Gemini was tested through a preview alias that Google can update in place. Qwen appears only in the fixed-index arm, so nothing here speaks to its native retrieval. 7.3 percent of citations remain unclassified. And Factnameh, our research partner, appeared among the cited sources (55 answers, nearly all in the fixed-index arm); we disclose the conflict and drew no conclusions from those citations.

What this means

For people who use AI to find information about Iran: the words you use still steer the answer, on every model. Verification framings ("is it true that...") continue to produce more balanced answers than leading framings that borrow anyone's political vocabulary. But this study adds two things. Whether your question is searched at all depends on your language, and you will not be told. And the sources behind a Persian answer are drawn heavily from government-funded media on both sides of Iran's information conflict, so an answer that reads as independent may be an aggregate of funded voices.

For independent and exile media: the visibility problem we described in the original reports is confirmed and now better specified. Genuinely independent diaspora outlets are cited far less than the foreign-state-funded Persian services that share their category in most typologies. The practical advice stands: publish in ways retrieval systems reward, treat state framings as claims to debunk rather than vocabulary to avoid, and study what your audience actually asks AI systems. But the competitive landscape inside AI answers is not you versus the Islamic Republic. It is you versus BBC Persian, Iran International, and Radio Farda, outlets with state budgets behind them.

For model providers: the two arms of this study separate your responsibilities. What your model does with a loaded premise is yours: cue sensitivity varies threefold across models on identical retrieval, so it is trainable. What your search layer surfaces, whether it fires at all in a given language, and whether its citations are auditable from outside is an infrastructure choice, and it is currently opaque enough that a research team specifically studying retrieval opacity was caught out by it. Disclose which engine answered, per response. Nothing in our correction would have been necessary if that one line of metadata existed.

For researchers and fact-checkers: redo-and-replace is our recommendation from experience. Audits of AI systems inherit the opacity of the systems they audit, and the best defense is instrumentation: explicit settings, raw response archives, repeated runs, deterministic classification, and the readiness to update when your understanding of the infrastructure improves. Our updated version is more interesting than the original, and it cost far less effort than the manual study did. That trade is available to anyone who automates, and to anyone willing to treat updating as part of the work.

---

This report presents joint research by Gazzetta and Factnameh / ASL19 and updates our reports from early 2026; this study re-ran the same instrument with updated methodology on 14 August 2026.

The dataset covers 1,152 model answers, 4,919 captured citations, and blinded alignment scoring of every answer. Persian prompt design and the original instrument are the work of native Persian speakers.

The collection pipeline, media matrix, judge, and dashboards are available for replication; write to hello@gazzetta.xyz. Factnameh is run by ASL19, a partner in this research; its appearance among cited sources is disclosed in the limitations section.