AI for Real-World Evidence: What It Takes to Trust It

A chatbot or agent with every skill and full access to your clinical data is still not the right way to do epidemiology. Fast answers are easy. Evidence you can defend needs a different design.

Most writing about AI for real-world evidence (RWE) lists what AI can do. This piece is about something else: why the obvious setup, one smart agent connected to your data, produces results you should not trust, and what has to change before you can.

A study in a few minutes

Take a simple question: does Drug A reduce hospitalizations compared with Drug B?

You open a chatbot connected to your claims data and ask. A few minutes later you have a hazard ratio and a Kaplan-Meier curve.

But what happened in those few minutes? The agent:

• chose who is in the cohort
• defined the exposure window and the start of follow-up
• picked the confounders to adjust for
• decided how to handle missing data
• picked the diagnosis and drug codes, from memory
• ran the analysis, saw the result, and still had the whole conversation open to change its approach

Each of those is a bias decision. In a real study, an epidemiologist makes them in a protocol and a statistical analysis plan (SAP), before anyone looks at the outcome. Here they were made silently, by a model, with the answer in view.

Three problems that more tools do not fix

1. Agents are steered by expectations

Stanford researchers gave AI coding agents the same dataset and the same question, but different beliefs. Agents told they were pro-coffee reported more positive effects of coffee than agents told they were anti-coffee. The pattern held across four fields, from public health to biology (Miao, Pritchard and Zou, 2026).

The worrying part is that the analyses looked fine. 86% of the agent reports passed an independent AI methods review, and 78% passed review by a majority of human experts. The problem was not bad statistics. It was which of many defensible analyses got chosen and reported.

The gap remained even when the researchers fixed the target quantity in advance. Pinning down the estimand alone is not enough. The cohort, codes, confounders and model all have to be fixed too.

2. Agents can search for the answer you want

In a second study, researchers ran Claude and Codex on datasets from four published papers with null results, across 640 runs (Asher et al., 2026). Under normal prompts, both models gave stable estimates and refused direct requests to p-hack. But when the same request was framed as "exploring alternative approaches", both models looped over specifications and picked the most significant result. Observational designs, which is what most RWE studies are, were the most exposed.

An agent that can see the outcome and keep trying will find a result. That is the garden of forking paths, now at machine speed.

3. Long conversations rot

Chroma tested 18 frontier models and found that performance fell as the input grew, even on simple retrieval tasks (Chroma, Context Rot, 2025). A real study conversation runs to tens of thousands of tokens: the question, the papers, the code lists, the code, the errors, the fixes. By the time the agent writes the analysis, the inclusion criteria from the start are a long way back, and the model handles them less reliably.

Giving the chatbot more MCPs and skills does not touch any of these three problems. Neither does adding a second agent to review the final answer. As the Stanford study shows, the final answer usually looks fine.

What has to change

Lock the bias decisions before anyone sees the outcome

The fix is old. Write the protocol and the SAP, have a person review and sign them, and only then touch the outcome data. Cohort, exposure, comparator, follow-up, confounders, missing-data handling and the model are all fixed in that plan. The analysis then runs the plan. It does not get to revise it after seeing the hazard ratio.

That is what pre-registration and target trial emulation have always asked of human researchers. AI does not change the rule. It makes the rule easier to break, so the system has to enforce it.

Get codes and drug facts from a source, not from memory

A model working from memory will produce ICD-10 codes, RxNorm concepts and citations that look right and are not. Code lists and drug facts have to come from tools that query the real vocabularies and the real literature, and every code should trace back to the source it came from. A skill that tells the model to "be careful with codes" is still memory. A tool that searches the vocabulary is a source.

One agent per stage, holding only the approved plan

You cannot fix the first two problems inside one conversation. Split the study into stages: question, protocol, cohort, SAP, extraction, analysis, report. Give each stage its own agent, and give that agent only what it needs: the approved outputs of the earlier stages. Nothing rots in a 100,000-token chat, because no stage ever holds one.

Multiple agents alone are not the answer

A pile of agents talking to each other has the same problems as one agent, just spread out. What matters is:

• Enforced boundaries. The analysis stage cannot change the signed SAP. Outcome data is out of reach until the plan is locked.
• Traceable sources. Every code, confounder and design choice points to where it came from.
• Reproducible execution. The code is saved and reruns on the same data snapshot to give the same number.

What regulators already expect

Regulators are not banning AI in evidence generation. They are asking for the same discipline.

• FDA. The January 2025 draft guidance on AI to support regulatory decision-making for drugs sets out a 7-step credibility framework. It starts with the question of interest and the context of use, then asks for a plan, its results and any deviations, all documented.
• FDA and EMA together. In January 2026 the two agencies published ten shared principles of good AI practice. One of them is that "data source provenance, processing steps, and analytical decisions are documented in a detailed, traceable, and verifiable manner."
• EMA. The 2024 reflection paper on AI in the medicinal product lifecycle warns that AI must not undermine trust in regulatory decisions, and that bias must be actively managed. EU guidance is being built on it, and EMA's own RWE network, DARWIN EU, already publishes its study protocols.
• NICE. The 2024 position statement on AI in evidence generation asks for justified, transparent methods and says AI should "augment human involvement, not replace it."

None of this fits a chatbot that picks its own cohort after seeing the result. All of it fits a locked plan, sourced decisions and a full record.

When to trust it: like a new researcher

Think about how you come to trust an intern or a new researcher. They document their decisions. They explain their reasoning in meetings. They maintain their code. They flag what they are unsure of and ask when they do not know. They present their findings and let you review and sign off.

You do not trust them because they answer quickly. You trust them because you can see how they got there.

An AI epidemiology tool should earn trust the same way. Before you trust its results, you should be able to see:

1. The signed plan, with a date that comes before any outcome data was touched.
2. The source for every code and drug fact, not just the list.
3. The decisions and the reasons, stage by stage, including what was rejected.
4. The open questions the AI raised, and how a person answered them.
5. The code, which reruns and gives the same numbers.
6. Who signed off each stage, and what they changed.

If a tool cannot show you these, its speed does not matter.

Where AI does help

None of this means AI has no place in RWE. It is good at the slow, repetitive work: drafting a protocol from a question and the literature, searching vocabularies for code lists, reading clinical notes, running feasibility counts, writing extraction and analysis code, and drafting tables and reports. Used inside the boundaries above, it can take a study from months to days without taking the decisions away from the people accountable for them.

How Frekil is built

Frekil runs each RWE study as separate stages, each with its own agent. A person reviews and approves the protocol, cohort and analysis plan before any data is extracted, and every approval is recorded. Code lists come from vocabulary tools, not model memory. The AI writes the code. It never sees patient-level data. The code runs in a sandbox inside your environment, and every number traces back to the code and the decision behind it.

We wrote more about why a single chat is not enough in Why real-world evidence needs more than a chatbot.

So, a question for you: what is the one thing you would insist on seeing before trusting any AI tool's results?

Frequently asked questions

What does "AI for RWE" mean?

Using AI, including language models and AI agents, to do the work of a real-world evidence study: protocol drafting, code lists, cohort building, data extraction, statistical analysis and reporting. The hard part is not speed. It is making sure the AI cannot quietly make bias decisions after seeing the outcome.

Can an AI agent connected to my data run an RWE study?

It can produce a hazard ratio quickly. That is not the same as a defensible study. Without a signed protocol and SAP locked before the outcome is seen, sourced code lists and a record of every decision, the result cannot be trusted or reviewed.

Does adding more agents or a reviewer agent fix the problem?

No. Studies show that biased or selectively chosen analyses usually look fine at the end. What helps is enforced boundaries between stages, traceable sources and reproducible code.

Do the FDA and EMA accept AI in real-world evidence?

They accept AI when its role is clear, its methods are documented and its output is shown to be credible for that use. See the FDA's January 2025 draft guidance and the FDA-EMA principles from January 2026.