AI Cut Immunotherapy Chart Review by Over 40%

The three points

• Checking AI suggestions took about 40% less time. In a randomized crossover study of 17 clinical research staff, median review time fell from 428 to 242 seconds.
• The system finds events better than it grades them. F1 was 0.92 for detection and 0.66 for severity. Across 884 live notes, its grade matched human review 62% of the time.
• Reviewer agreement rose, possibly through anchoring. The trial could not separate real consistency from reviewers following the AI. External validation and patient outcomes remain untested.

Teams that track immunotherapy side effects still read clinical notes by hand. A group led from Mass General Brigham tested how much of that work an agentic AI system can take on, moving from development to a live trial, and published the results in Nature Cancer on October 29, 2026.

Side effects buried in notes

Up to 40% of patients on immune checkpoint inhibitors develop immune-related adverse events (irAEs). The authors cite ICD code sensitivity of only about 68%, even for events that led to admission. Everything else sits in free text.

The system reads one note at a time and looks for six irAEs: myocarditis, dermatitis, thyroiditis, hepatitis, colitis and pneumonitis. For each, it extracts whether the event is current or past, the CTCAE severity grade, attribution to the drug and the certainty of that attribution. It runs detection and grading three times, and a judge agent settles disagreements. The best setup used GPT-4.1-mini and cost about US$0.021 per note.

The three phases

The team first tested 263 expert-annotated notes. Detection F1, a single score that balances misses against false alarms (1.0 is perfect), reached 0.92. Dropping the three-run step lowered it to 0.78. Severity grading reached 0.66, with most errors at grades 2 and 3.

A 779-term keyword search matched or beat some LLMs on detection alone. You need the AI for what keywords cannot extract: grade, attribution and certainty.

From May to July 2025 the system ran silently on 884 live notes, with results hidden from clinicians. F1 fell to 0.72 for current events and 0.79 for past events. Its output matched human review for 62% of grades and 87% of attributions.

Time and agreement

In the final phase, 16 research coordinators and 1 research nurse reviewed 20 notes with and without AI. Median review time fell from 428 to 242 seconds, a 43.5% drop (difference 2.71 minutes, P<0.001). Fifteen of the 17 preferred working with AI.

Reviews where every current grade matched the gold standard rose from 19.4% to 25.9% (OR 1.45, P=0.045). Even with AI, roughly three in four reviews still got at least one grade wrong. Label-level grade accuracy showed no significant difference (P=0.791).

Agreement between reviewers (Krippendorff's alpha, where 1.0 is perfect) rose from 0.22 to 0.85 in one note set and from 0.51 to 0.82 in the other.

Limitations

Read the agreement numbers with care. If every reviewer follows the same wrong suggestion, alpha still rises. The authors say their study cannot separate this anchoring from real improvement.

All three phases ran inside one health system, so no phase counts as external validation. The team did not measure outcomes such as time to steroids or hospitalization.

What this means in practice

This part is our view. The closest fit today is re-reading notes for trials and safety reporting. Nobody should hand bedside severity grading to this system yet. Japanese-language notes are also untested here. If you plan to adopt something similar, first test whether your own staff catch the AI's errors on your own notes.

Study notes: Funders include the US National Cancer Institute and the American Cancer Society, and the authors state they had no role in the study. Some authors disclosed consulting fees and other industry ties. The code is available on Zenodo.

Sources

1. [1]
   論文：Nature Cancer (DOI: 10.1038/s43018-026-01264-6)
2. [2]
   論文：PubMed (PMID 42957896)

Images: generated with GPT Image 2.5 (illustrative).