This newsletter has been quiet for two weeks, and I owe you the reason. I spent the first part of September in the United States with the Forum’s supporters, through the 25th anniversary of September 11, and then came home to Israel for the High Holidays, which end with Yom Kippur tomorrow night. It was not idle time. Since early summer a small group at the Forum has been building the systems this essay describes, and I wanted to show you working systems, not plans. At the center of the essay are ten comparisons between how our field works and how the Forum works now, each with a line on why the difference matters. The essay ends with a request: help us build the rest. This edition goes to every reader, free and premium alike. A second free edition follows this afternoon, on the odds of renewed American and Israeli strikes on Iran. The daily brief returns this week, after the holiday.
The Forum Is Building Something. Here Is What You Will See.
Two quiet weeks from this newsletter, and the reason for them: we have been turning the Middle East Forum into an organization that reads everything, forgets nothing, and shows its work. Ten comparisons with how our field works, and what you will see next week, next month, next quarter, and next year.
The Middle East Forum is an applied policy research organization. We were founded in 1994 to promote American interests in the Middle East and to protect Western values from Middle Eastern threats. For most of those 32 years the work ran the way it runs everywhere in this field: a scholar reads, a scholar writes, an editor publishes, and the finding waits on a shelf for someone in government to pick it up.
That model has a ceiling. The region produces more primary material in a single day than any research staff can read in a month: the press in four major languages, thousands of pages of court filings, foreign-agent registrations, university disclosures, tax returns, sanctions designations, and parliamentary records. Most think tanks respond by choosing a narrow slice and writing well about it. We decided to read all of it.
One ordinary example shows the problem. A university in the Midwest reports a gift from a Gulf government under Section 117 of the Higher Education Act, the federal law on foreign gifts. That government’s agents in Washington file a report under the Foreign Agents Registration Act listing the congressional offices they visited that quarter. A nonprofit in the same city, whose board overlaps with the university’s Middle East center, files a tax return showing a grant from a foundation that the Treasury Department later designates.
Each of those four documents is public. Each sits in a different federal database with a different format, a different filing schedule, and a different spelling of the same organization’s name. A diligent researcher can connect them in a week. Nobody connects them at scale, because tens of thousands of such filings arrive every year, and the connections are where the story lives.
Since June we have been building the systems that let about 50 staff, fellows and contributors do the reading of 500 without lowering the standard of proof. I will not describe the machinery in detail. Some of it is our advantage, and I intend to keep it that way. What I can describe is the discipline it runs on, what it does today, and what it will put in front of you over the next 12 months.
First, every claim carries its source. When one of our systems extracts a fact from a document, it stores the document, the passage, and the date alongside the fact. A reader, a journalist, or a congressional staffer who challenges a line in a Forum product will be able to see the page it came from. A finding that cannot be traced does not ship.
The rule sounds modest. Most institutions in our field cannot meet it, because their archives were built as prose, and prose loses track of its sources once an editor cuts the footnote for length. We rebuilt ours so that the footnote is the record and the prose is a summary of it.
The rule has a corollary. Our systems draw connections only from filed documents. If two organizations share an address, a board member, or a funder, we say so because a filing says so, and we label the connection “filed link.” If the machine notices a pattern that no document supports, the pattern becomes a lead for a human investigator and carries that label. It does not become a claim. The common failure in this field is the chart full of arrows that nobody can source, and we designed against it from the first line of code.
Second, a person signs everything. The systems draft, sort, translate, and flag. They do not publish. Every product that leaves the Forum passes a named editor or analyst who answers for it. Any statement that judges rather than reports carries a stated likelihood, a stated confidence, and the reason for both.
We adopted the intelligence community’s own standard for that language, Intelligence Community Directive 203, which governs how American analysts phrase a judgment. A reader deserves to know the difference between what we observed and what we assess. When a Forum product says from now on that the regime is “unlikely” to do something, the analyst chose that word from a fixed scale and has told you why.
I hold to this rule for a reason beyond quality control. Artificial intelligence is very good at producing text that sounds like analysis. The Forum’s value has never been in sounding like anything. It has been in being right, and in being accountable when we are wrong. A machine cannot be accountable. A person with a byline can.
Third, we keep score on ourselves. This month we opened a standing forecast log. It opens with five questions about Congress and the region. Each has a date by which it resolves and a base rate drawn from the filed record of how similar questions resolved before, and the Forum’s analysts record a probability before the fact. When the date arrives, the record shows whether we were right, and by how much, under the Brier score: the rule the National Weather Service uses to grade its rain forecasts and the one the intelligence community’s forecasting tournament used to grade its forecasters. I do not know of another organization in our field that has agreed to be graded this way in public. We have.
The questions are deliberately narrow: whether a particular chamber holds a recorded vote on a particular class of resolution by a particular date, or whether a particular foreign government signs a given number of new lobbying contracts in a given quarter. Narrow questions resolve cleanly, and clean resolution is what makes a score honest. Grand predictions about the fate of regimes are easy to make and impossible to grade. We will still write about the fate of regimes. We will be scored on what can be scored. This afternoon’s edition adds five questions to the log, on whether the United States and Israel strike Iran again, with the Forum’s probabilities attached.
“A machine cannot be accountable. A person with a byline can.”
Most policy institutions have added artificial intelligence as a way to draft faster. A global survey of think tanks, reported on Thursday, found that 84 percent now use AI and that many “remain in the first phase of adoption, using readily available tools to make individual tasks easier.” The output looks like what those institutions produced before, arrives sooner, and rests on the same three sources the author would have read anyway.
We have done something else. We rebuilt the organization’s reading, its memory, and its standard of proof around one fact: a machine can now read every filing in every language and hold it in one place, while a person decides what it means and stands behind the answer.
The difference is easiest to see side by side. Each of the ten comparisons below opens with how our field usually does a job, then says what the Forum does now, what is still ahead, and why the difference matters. A reader can test every one against our output this quarter.
I will not name the organizations, because the point is not to embarrass anyone. Most of them do honest work by the standard of five years ago. The standard has moved.
Editions built around what each reader follows
One newsletter goes to everyone on the list, on whatever the editor found interesting that morning.
More than 30,000 of the Forum’s readers are sorted by the nine subjects they actually read, based on what they opened, and a reader survey lets each of them correct the sorting. Editions built for each subject follow.
Congressional staff will receive something narrower: a Monday brief ranked against their own committee’s hearings over the next 14 days, with the name of the analyst who can brief them at the bottom. That pilot begins next month.
Research changes policy only when it reaches the person who can use it, in the week that person needs it. A staffer preparing for a hearing will read one page written for that hearing. A general newsletter has to compete with everything else in the inbox.
Press errors flagged the same day and followed until they are fixed
A team of analysts reads the morning’s coverage, spots an error, researches it by hand, and writes to the editor three days later.
A system reads roughly 1,200 news items a day and flags each doubtful claim together with the source that contradicts it. An editor approves every correction request before it goes out, and we record how the outlet responds: corrected, quietly edited, ignored, or refused.
A request ignored long enough goes to Forum supporters who live in the area that outlet serves, and they can write to its editors themselves. The platform is in private testing with staff and a first group of supporters. It opens to the public on October 5.
A correction made the same day reaches the people who read the error. A correction made three days later reaches almost none of them, and an error that nobody challenges becomes the version the next reporter repeats. Tracking how each outlet responds shows which ones fix their mistakes and which do not.
The region’s press, read in its own languages every morning
Foreign press coverage arrives through English-language wire summaries, or through one translator per language who reads what she has time to read.
Every morning the Forum reads the Arabic and Turkish press, and the Hebrew and Farsi press on the days our analysts call for it: the regime papers, the opposition papers, and the diaspora outlets. Systems we run ourselves, on servers in the United States, translate the articles into a single English digest with each outlet named. The translation step adds nothing to the text.
An analyst decides what matters before any of it reaches you. This afternoon’s edition on the strike question draws on this week’s Tehran, Jerusalem, and Beirut press, and names each outlet.
Governments in the region say one thing to foreign reporters in English and another to their own people in their own language. What a regime tells its own public is the better guide to what it will do, and the English wires rarely carry it.
Two filed documents behind every link, or no link at all
“Reports indicate,” followed by a footnote to another institution’s report.
A sentence linking a foreign government to a Washington lobbyist rests on two filed documents, or the Forum does not draw the link.
Behind that rule sits a single database we call the data lake. It gathers foreign-agent registrations, university disclosures of foreign gifts and contracts, nonprofit tax returns, lobbying disclosures, and sanctions lists, among others, and it holds 21 of the 25 sets of federal filings we set out to collect. It has matched more than 8,900 organizations and people to a single identity each, and it holds more than 33,000 dated records of what they gave, filed, or lobbied for.
A claim built on documents that governments, lobbyists, and nonprofits filed themselves holds up when a lawyer, a reporter, or an unfriendly committee staffer tests it. That is the standard an oversight committee needs before it acts, and the standard a newsroom needs before it prints.
Twenty years of investigations, searchable claim by claim
Two decades of investigations sit in an archive of PDFs, searched one article at a time and fully usable only by the people who wrote them.
The Islamist Watch archive, roughly 1,850 investigative articles written over two decades, has been broken down into more than 32,000 individual claims about organizations, people, events, and funding links. Each claim points to the article and passage it came from, and each gets a second check before we mark it verified.
Scholars and reporters will be able to search the verified claims directly, with sources attached. The search console opens next month, first to those already on the waiting list.
Twenty years of investigation can now be questioned as one body of evidence instead of searched one article at a time. Opening the verified claims to outside scholars and reporters lets people who do not work for us check them, which is how errors get found. The Forum took the same step on September 10, when it opened its research archive on American Islamism to researchers by application.
Predictions with numbers and dates, graded in public
Predictions are written in prose, hedged so that they can never be wrong, and never revisited.
Every judgment carries a likelihood term from a fixed scale, a percentage range, and a stated level of confidence with the reason for it. The narrow questions behind each judgment go into a log with a resolution date. Five went into the log today, in this afternoon’s edition. The first full scorecard follows in January, including the ones we got wrong.
A forecast with a number and a date can be shown to be wrong. That is what makes it useful: a reader can look at our record and decide how much weight the next judgment deserves.
Forum research in seven languages besides English
English only, with a translation of the occasional major report months later.
Translated editions of the Forum’s recent work exist today in Arabic, Farsi, Hebrew, Kurdish, French, German, and Spanish. Each article is translated the day it is published and independently checked before it is marked ready, and native speakers on staff are reading the editions now. They move to permanent addresses this quarter.
The people we write about, and the people who live under the governments we write about, will be able to read the work directly and answer it. A reader in Tehran, Erbil, or Berlin will no longer depend on someone else’s summary of what the Forum said.
A map of who holds power in Tehran, kept current
An organization chart of an adversary regime, drawn for a report three years ago and never updated.
A working map of who holds power in Tehran, from the Supreme Leader’s office through the security services to the parallel economic foundations. Every position is sourced to the document that says who holds it, and the map changes when the record changes.
It runs today as a pilot inside the Forum, and it opens to the public once every office and office-holder on it is sourced. Two more countries follow this quarter.
Sanctions, designations, and negotiations all depend on knowing who decides in Tehran, and the war has changed the answer. A chart drawn three years ago names officials who are no longer alive or no longer in office.
Warning signs tracked every day, before the crisis
Analysts learn that a crisis has started from cable television, then spend a week reading to reconstruct how it built.
An early-warning board tracks, day by day, how often the monitored press in four languages reports on mobilization, threats, closures, and talks, so that our analysts can see a crisis building before it breaks.
The board reports numbers and trends only. It will not offer a judgment until it has enough history to be checked against what actually happened, and I would rather tell you that plainly than present a guess as a finding.
Crises in this region often show in the local press before they begin: reserve call-ups, airport and school closures, a change in the tone of official threats. An organization that tracks those signals every day can warn its readers before the event instead of explaining it afterward.
Public-records requests filed in an afternoon and followed to the end
A public-records request is drafted by outside lawyers over three weeks, filed, and forgotten.
A request that cites the right law and goes to the right agency is drafted in an afternoon, filed by our own staff, tracked to its legal deadline, and appealed when the agency misses it.
Public-records laws are how citizens obtain documents a government office would prefer not to release: grant files, visitor logs, correspondence with foreign officials. Many requests fail because nobody follows up. A request that is tracked and appealed produces documents, and those documents become the filed record the rest of our work relies on.
None of what the field does in those ten comparisons is negligent. It is how the work was done when reading was the bottleneck. Reading is no longer the bottleneck. Judgment is, and judgment is the one thing we have not automated.
The ten comparisons do not cover everything that is running now.
The method is already visible in the Forum’s published work. July’s investigation of the financing behind CAIR’s Washington headquarters rested on the Islamic Development Bank’s own reports, tax returns, and property records. Hidden Handouts, published the same month, documented nearly $80 million in Health and Human Services grants to 14 organizations.
Last week we launched a Geospatial Intelligence Unit, which checks official accounts of attacks against satellite thermal data and imagery. Its first report located the damage from the September 10 drone attack on Saudi Arabia’s East-West Pipeline: seven new fires on one 110-kilometer segment southeast of Medina, and none in the Riyadh region, where the Saudi energy ministry had placed part of the attack.
Behind all of this sits the Forum’s own published record: the Middle East Quarterly, the Observer, and the project sites, indexed back to 2004 as a single searchable body. When an analyst writes about a subject today, the system shows what the Forum said about it in 2009 and in 2017, and whether the earlier judgment held.
This newsletter, which now reaches more than 80,000 readers, is edited under the same source discipline. We check every number in the brief against the outlet it came from before the brief goes out. When we get one wrong, the correction runs in the next issue with the original error quoted.
I want to be equally clear about what does not exist. No system at the Forum decides what we publish. No system writes an assessment and sends it without a person reading every line. None of the research systems described here scores individuals, uses leaked or biometric data, or draws a connection between a person and a group that a public filing does not draw first. Those are not technical limits. They are rules, and I set them.
The daily brief returns this week. The Sunday Long View keeps its weekly slot from today. Behind both, the press monitors and the morning desk briefs that feed my own reading keep running, and the Hebrew and Farsi monitors move to a daily schedule alongside the Arabic and Turkish ones.
The forecast log goes public. In October we publish the five opening questions, the base rate we drew from the filed record for each, the Forum’s probability, and the resolution date, so that you have our numbers in hand before events decide them.
The first in a series of short, citable briefs built directly from the data lake, written for congressional staff and reporters who need a documented answer in four pages, not a monograph. The first brief covers one Gulf state’s influence operation in Washington as its own agents have disclosed it, with every sentence tied to a filed document.
Three of the ten comparisons reach the public. The media corrections platform opens on October 5. The Monday committee brief for congressional staff begins its pilot. And the search console for the Islamist Watch claims opens, first to the scholars and journalists already on the waiting list.
The first scores from the forecast log. Three of the five opening questions resolve on December 31, and the first full scorecard follows in January, question by question, including the ones we got wrong.
The Iran power map extended to a second and a third country, on the same standard, and the seven translated editions at permanent addresses.
A public data portal: the Forum’s principal datasets, including the campus foreign-funding record and the Middle East foreign-agent filings, published with methodology and downloadable in full, so that other researchers can check our work or build on it without asking permission.
The second verification pass across the Islamist Watch database, complete, with the share of claims independently confirmed stated on the record.
By this time in 2027 I intend for every factual claim on a Forum page to be traceable to its source in one click, for the forecast log to carry a full year of scored predictions, and for the reading and investigative capacity described here to cover every country desk the Forum runs, not only Iran. Turkey, the Gulf, the Levant, and the Islamist networks operating in Western institutions each deserve what Iran is getting now: a sourced power map, a daily read of the domestic press, a database of the Forum’s own findings that can be queried as evidence, and a set of scored questions we are willing to be wrong about in public.
I also intend for the Forum’s research to be usable by others without asking us. A congressional staffer who needs to know what a foreign government’s agents filed last quarter should be able to find it in our portal in two minutes, with the filing attached. A reporter in Berlin should be able to read our work in German the week it is published. A scholar in Amman should be able to check our Islamist Watch claims against the sources we cite, in Arabic. The measure of the year will not be how much we produce. It will be how much of what we produce other people can verify and build on.
The scholarship will still be written by people. The people will have read everything first.
None of this makes our analysis correct. It makes it checkable, which is the precondition for being trusted by people who do not already agree with us. The Forum has spent 32 years being read by its friends. The systems we are building are designed to make us readable, and unanswerable on the facts, by everyone else.
The distinction matters beyond efficiency. The Forum’s independence has always been its asset: no government funds our research, and we can name any name. That independence is worth something only if the naming rests on a record that survives challenge. These systems exist to make that record complete and checkable. I would rather lead the organization that shows its evidence than the one that asks to be trusted.
No grant program and no government contract pays for this work. Private donors do, as they have paid for the Forum’s work for 32 years, and the build described here is carried on the Forum’s general operating budget alongside every other program. A gift to the general fund pays for the analysts who verify, the hardware the systems run on, and the editors who sign their names. If you would like to designate your support for this build specifically, reply to this email and we will record it that way.
I will be concrete about what the money does, because the rule I set for the systems applies to me as well. It pays the analysts who read the flagged material every morning and decide what matters. It pays the second reviewer on every Islamist Watch claim before it is marked verified. It pays the native speakers who check the translated editions before they go live. It pays for the servers, which we rent by the hour in the United States and shut down when idle. And it pays the editors who put their names on the product. The machines are the smallest line in that budget. The people who make them trustworthy are the largest, and they should be.
Every donor to the Forum receives premium membership in this newsletter with the Forum’s compliments. The first scored forecast report will go to donors before it goes anywhere else, along with a standing invitation to the quarterly briefing where our analysts walk through the record. You can give here.
Money is one way to help build this. There are two others. Readers who work with data, languages, or public records can tell us which filings, outlets, or datasets we are missing; those replies go to the people building the systems. And every reader can hold us to the rules above. If a Forum product makes a claim without a source, write and say so. I read the replies myself, and the ones that catch an error will be answered in this column.
Thank you for staying with me through two quiet weeks. The next ones will not be quiet.
Both of today’s editions are free for every reader. The premium Long View, the full archive, the quarterly threat briefings, and reply access are for premium members: $12 a month or $100 a year. Every MEF donor reads free; if your invitation has not arrived, write [email protected].
Go PremiumIndividual claims drawn from two decades of Islamist Watch investigations, each tied to the article and passage it came from and each checked a second time before it is marked verified. The claims come from roughly 1,850 articles. A year ago, a researcher had to search those articles one at a time.
A second free edition follows this afternoon: the odds of renewed American and Israeli strikes on Iran, with five questions entered in the Forum’s forecast log. The daily brief returns this week. The last two free Long Views are still open to every reader: The Weight of Command, on six commanders 25 years after September 11, and The Empire That Isn’t, on preemption and the lesson of Kabul. New readers can orient with the Field Guide; the MEF Dispatch, our summary of new articles and analysis from meforum.org, is one click: Subscribe to the MEF Dispatch.
Clear this morning. When it should not be, write [email protected]; corrections run here, in print.
The systems in this essay run on private gifts and on the people those gifts pay: the analysts who verify, the native speakers who check the translations, and the editors who sign. If you want a Forum that shows its evidence and publishes its scorecard, fund the build. And forward this edition to one person who reads policy research for a living.
Support the Forum →Reply question: which of the ten comparisons matters most to your own work, and what should the Forum be scored on that it is not? Replies land in my inbox; the sharpest answers run in the next mailbag.
Middle East Insider · a project of the Middle East Forum