Title and Abstract Screening for Systematic Reviews: Where AI Helps and Where It Contaminates

A systematic review begins with a search that returns thousands of records and a screening step that removes most of them. Screening is where AI tools are most tempting and most dangerous. A model can triage 4,000 titles in minutes, but a review's credibility rests on reproducible inclusion decisions. The useful role for automation is ordering and deduplication, not deciding inclusion on its own.

Get the record set clean first

Before any screening, export from every database with the same fields: title, authors, year, journal, DOI, abstract, and record ID. Merge with a deterministic deduplication pass on DOI first, then on normalized title plus year. Fuzzy matching catches author-name variants but also merges distinct papers with near-identical titles; require a human review of fuzzy matches above a similarity threshold.

Record the counts at each step for the PRISMA flow diagram: identified, duplicates removed, screened, excluded with reasons, included. If your tool cannot reproduce those counts from its own logs, it is not suitable for a review.

Priority ordering beats auto-exclusion

The strongest use of AI in screening is ranking, not filtering. A model that scores each record against the review question lets reviewers process likely includes first, which shortens the time to a stable inclusion set and surfaces edge cases early.

Build the score from explicit criteria: population, intervention, outcome, study design, and date range. Ask the model for a label plus which criterion failed, and store that rationale. Then have humans screen in descending score order until a block of 100 consecutive records yields no new includes. That stopping rule is defensible; "the model said no" is not.

Never let the model silently exclude records. Every exclusion needs a human-confirmed reason recorded against the record ID.

Dual screening and disagreement

Most review methods require two independent screeners at least at full-text stage. If AI participates, treat it as a third opinion with a documented prompt version, model version, and date. When the two humans disagree, the model's rationale can inform the discussion but must not cast a deciding vote.

Calibrate before scaling: have both humans screen the same 50 records and calculate agreement. If agreement is low, the criteria are ambiguous, not the screeners. Fix the protocol before touching the remaining thousands.

Choosing a screening tool around the audit trail

Reference managers handle storage and deduplication well. Dedicated review platforms such as Rayyan or Covidence handle blinding, conflict resolution, and audit trails. Browser extensions are best at the boundaries: pulling records from a database's export page, capturing full-text PDFs, and checking a record's identifiers against Crossref.

An AI extension summarizing abstracts is useful for orientation, but summaries drop exclusion-relevant details: sample size, comparator, and setting. Screen on the abstract itself, not on a generated precis.

Every automated step needs a record: the search string and date per database, the deduplication rule, the model and prompt for ranking, the screening decisions with reviewer initials, and the final counts. Export this as a methods appendix. Reviewers and readers increasingly ask for it, and a review that cannot reconstruct its own flow is rejected at the methods stage.

The line to hold: automation may organize, rank, and summarize. Inclusion decisions remain human, documented, and reproducible.

Blinding, conflict resolution, and export of decisions matter more than the interface. Before committing a protocol to a platform, check three things: whether it records per-reviewer decisions with timestamps, whether it exports exclusion reasons in a form that maps to PRISMA categories, and whether it supports changing a criterion after screening starts with a log of the change.

A spreadsheet can satisfy all three if the columns are fixed in advance and the file is version-controlled. A browser extension cannot, because its state lives inside a reviewer's browser. Use extensions to move records in and out, not as the system of record.

Exclusion reasons that survive review

"Not relevant" is not a reason. The protocol should enumerate exclusion categories tied to the research question: wrong population, wrong intervention, wrong comparator, wrong outcome, wrong design, not peer reviewed, duplicate, language, date.

Allow one primary reason per record and an optional secondary note. When reviewers disagree on the category but agree on exclusion, that disagreement is useful: it often reveals an ambiguous criterion that later affects the full-text stage.

Pilot screening, and handling the unindexed tail

Screen the first 50 records with all reviewers before the main pass. Calculate percent agreement per exclusion category. Low agreement on one category means the criterion needs rewording, not more training.

Record the pilot decisions and then reset or keep them deliberately. Mixing pilot and main-pass decisions without a marker makes the flow diagram impossible to reconcile.

Records without abstracts, conference abstracts, and grey literature are the ones where title screening fails. Route them to a separate queue and decide by hand. A model ranking them is guessing from a title fragment.

Using AI as a second reader, and the data it sees

Use the model to re-check a random 10% sample of human exclusions against the same criteria. Any record the model flags as a possible include goes back to a human. This catches fatigue errors without letting automation decide inclusion, and it gives you a number to report in the methods section: the disagreement rate between human and model screening.

De-identify if records include personal correspondence, keep the raw search exports untouched, and record the software versions used. Reviewers increasingly ask for the export files alongside the manuscript, and a missing raw file is harder to explain than a long appendix.

Label 200 records by hand, then compare the model's decisions. Report sensitivity and specificity on that set before trusting it on the rest.

If sensitivity is high but specificity is low, use the model to rank rather than to exclude. Screening everything it flags keeps recall while saving time on the obvious rejects.

Keep the screening decisions in a file you own. Whatever tool you use, export the decisions with a timestamp and the model version where one was involved. If the tool changes hands, raises prices, or disappears, the review still has its record, and you can rebuild the flow diagram from the export rather than from memory. This is the difference between a review that can be updated in two years and one that has to be repeated from the beginning.