Claim Check: AI in Education
Claim Check rates whether the claims made by causal studies of AI in education are supported by their evidence and reasoning.
Where the work stands so far
Studies on Stanford’s causal list
The starting set for Claim Check.
Studies in the tracker
Screened in and entered in the database.
Claims coded by two people
From five studies, coded by blinded coders and reconciled in calibration round one.
Final ratings published
All ratings are held during development until version 1.0.
Search and filter
Coming soonSearch and filters open with the first release of ratings.
| Study | Level | Publication status | Stage | Claim | Implications |
|---|---|---|---|---|---|
| Bastani et al. 2025 | K–12 | Peer-reviewed PNAS 0.10% retracted (169 of 165,210) | Coded and reconciled | C | I |
| Kreijkes et al. 2026 | K–12 | Peer-reviewed Computers & Education 0.08% retracted (5 of 6,554) | Coded and reconciled | C | I |
| Stadler et al. 2024 | Higher education | Peer-reviewed Computers in Human Behavior 0.03% retracted (3 of 8,673) | Coded and reconciled | C | I |
| Meyer et al. 2024 | K–12 | Peer-reviewed Computers and Education: Artificial Intelligence 0.15% retracted (1 of 664) | Coded and reconciled | C | I |
| Abbas, Jam & Khan 2024 | Higher education | Peer-reviewed International Journal of Educational Technology in Higher Education 0% retracted (0 of 598) | Coded and reconciled | C | I |
Queued studies (21)
| Study | Level | Publication status | Stage | Claim | Implications |
|---|---|---|---|---|---|
| Fütterer et al. 2026 | K–12 | Peer-reviewed | Queued | ||
| Heldreth et al. 2026 | K–12 | Peer-reviewed | Queued | ||
| Henze et al. 2026 | Higher education | Peer-reviewed | Queued | ||
| Meyer et al. 2026 | K–12 | Peer-reviewed | Queued | ||
| Ozturk et al. 2026 | Higher education | Peer-reviewed | Queued | ||
| Zhao et al. 2026 | Higher education | Peer-reviewed | Queued | ||
| Barcaui 2025 | Higher education | Peer-reviewed | Queued | ||
| Lim et al. 2026 | K–12 (teachers) | Peer-reviewed | Queued | ||
| Zhao et al. 2025 | Higher education | Peer-reviewed | Queued | ||
| Chan et al. 2026 | Higher education | Peer-reviewed | Queued | ||
| Fischer et al. 2025 | Higher education | Not yet peer-reviewed | Queued | ||
| LearnLM Team, Google & Eedi 2025 | K–12 | Not yet peer-reviewed | Queued | ||
| Blasco & Charisi 2025 | K–12 | Not yet peer-reviewed | Queued | ||
| Becker et al. 2025 | K–12 | Peer-reviewed | Queued | ||
| Chen et al. 2025 | Higher education | Not yet peer-reviewed | Queued | ||
| Degen et al. 2025 | Higher education | Not yet peer-reviewed | Queued | ||
| Ferman et al. 2021 | K–12 | Not yet peer-reviewed | Queued | ||
| Demszky et al. 2025 | K–12 (teachers) | Peer-reviewed | Queued | ||
| Demszky et al. 2023 | K–12 (instructors) | Peer-reviewed | Queued | ||
| Kim et al. 2021 | K–12 | Peer-reviewed | Queued | ||
| Holstein et al. 2018 | K–12 | Peer-reviewed | Queued |
Counts as of 2026-11-12. Publication status for queued studies is provisional until screening. Retraction rate: retracted articles and reviews as a share of all articles and reviews the journal has published, from OpenAlex on 2026-11-13. Calibration records were made under CERIC-T version 0.3 and are kept as the reliability record; they are not published as ratings, except the one preliminary example above.
Why it is needed
A study can have a sound design and still claim more than it shows. Journalists, legislators, and educators mostly need the claim, not the design.
Stanford’s review of causal studies
Asks whether a study’s design can support a causal claim using WWC criteria. Claim Check is built on Stanford’s AI Hub for Education list of causal studies and complements it.
Claim Check
Extends and complements those findings to analyze the studies’ arguments from evidence. It also records who was studied and if dependence was studied.
How a rating is made
CERIC-T is a tracker version of the CERIC Method. Its conceptual model is ROBIS (Whiting et al., 2016), a tool for assessing risk of bias in systematic reviews: signaling questions, element evaluations, and a rule-based rating.
The CERIC Method: Bjorn, G., & Burgasser, A. J. (2026). How to critically read the scientific research literature: Introducing the CERIC Method. Cambridge University Press. cambridge.org
ROBIS: Whiting, P., Savović, J., Higgins, J. P. T., et al. (2016). ROBIS: A new tool to assess risk of bias in systematic reviews was developed. Journal of Clinical Epidemiology, 69, 225–234. doi.org/10.1016/j.jclinepi.2015.06.005
CERIC-T in brief
CERIC-T reads a study as an argument with five elements: the Claim, the Evidence, the Reasoning that connects them, the Implications the authors draw, and the Context of earlier research. For each element, coders answer signaling questions and cite the page that supports each answer. Each element is then evaluated as Supported, Some concerns, or Not supported.
Fixed rules turn those evaluations into two ratings. The Claim rating comes from the Claim, Evidence, Reasoning, and Context. The Implications rating comes from the Implications alone, and is given only when the Claim holds up.
These examples are illustrations, not real studies.
Full text
Coders work from the original article and its supplement, never from an abstract or a summary.
Two coders, blinded
Each answers 41 signaling questions across Claim, Evidence, Reasoning, Implications, and Context, and evaluates each element.
Rules applied
The database applies the published decision rules to produce the two ratings. No one can override a rating, but humans can check and append it if needed.
Reconciled
Where the coders differ, one proposes with a reason and the other accepts, or a third person decides.
Released
The director approves each release. Every rating records the instrument version it was coded under.
The design rules
- A coder cannot see the other coder’s work until both have submitted.
- A submitted coding is locked. A correction needs another coder’s approval and keeps the original.
- AI agents cannot write an answer, an evaluation, or a rating. They check references, completeness, and consistency, and raise flags for people to close.
- Every change is logged with who made it.
The database code and its automated tests are public: github.com/higherlearninglab/hill-database.
What Claim Check users will see
Each main claim in a study gets one card with two ratings: one for the Claim, one for the Implications the authors draw from it. The card below is for a real study in the calibration set. Its results are preliminary.
Note-taking, alone or combined with LLM use, gave better retention and comprehension three days later than using the LLM alone.
Preliminary: this study was coded by two people in calibration round one, under version 0.3 of the instrument. The results may change before version 1.0.
Reliability and validation
Calibration round one is complete: two coders independently coded ten claims from five studies and reconciled every difference. The instrument reliability data led to version 0.4 of the instrument.
What changed in version 0.4
Every element is now evaluated against the claim as the authors state it. Questions that divided the coders were reworded, split, or dropped. Each question carries an anchor. The Claim and the Implications are rated separately.
Round two
About 30 claims, coded under version 0.4.
Version 1.0
Released when agreement between coders reaches 0.70 (kappa) on each element, reported with its interval and the other agreement statistics.
Round-one agreement figures will be published with version 1.0.
Reported beside the ratings
These answer additional questions a rating cannot.
Evidence tier
The study’s design, using the names of the ESSA tiers of evidence (U.S. Department of Education, 2016), with one added level for studies that reach none of them.
Dependence
Whether an AI tool that supports learning hands responsibility to the student over time, whether the study measured that, and whether a teacher managed its use. This item applies a framework published by HiLL’s director (Bjorn, 2026, preprint).
Durability
Whether the outcome was measured after access to the AI tool was removed, and whether the effect held.
Who was studied
Whether students in any of the 22 groups in HiLL’s definition of vulnerable students were described in the sample, and whether effects were analyzed for them.
How Dependence, Durability, and Who was studied are shown
The same four colors as the ratings, with their own meanings. These are not ratings of the study’s argument.
What the evidence does not yet cover
Gaps in the literature are emerging, and each could be the basis for future research.
- Randomizing minors to companion-AI exposure with a measure after the AI is removed.
- Which groups of vulnerable students have been sampled or analyzed: Held until human review
- Measuring outcomes more than three months after AI access is removed.
- Comparing the same tool with and without pedagogical guardrails in a companion or emotional-support context.
Adjacent evidence, not appraised
Studies of school-age youth outside a school or college setting. They fall outside Claim Check’s appraisal scope and are listed for context.
- McBain et al. 2026, JAMA Pediatrics, 180(8). Cross-sectional, nationally representative survey of US youth aged 12 to 21 (n = 1,009) on use of AI chatbots for mental-health advice and whether they told anyone.
- Namvarpour et al. 2026, CHI ’26. Computational thematic analysis of self-reported Reddit narratives about teens’ reliance on AI companion chatbots.
Open and independent
Independent
HiLL does not accept funding from AI developers or edtech vendors.
Open
The database code and tests are public now. The CERIC-T codebook will be published under CC BY-ND 4.0 with version 1.0.
Answerable
Authors and readers can query any rating at [email protected]. Rules and ratings carry their version, and changes are logged.
Who does the work
Genevieve Bjorn, Ed.D.
Director of the Higher Learning Lab; lead coder. Co-author of the CERIC Method.
Adam J. Burgasser, Ph.D.
Co-Investigator and Senior Research Advisor. Professor of Astronomy and Astrophysics, UC San Diego. Co-author of the CERIC Method; second coder for calibration.
Scientific advisory panel
A scientific advisory panel is being formed to review the instrument and its validation. Members will be named here as they join.
What funding makes possible
The instrument, the database, and the coding system are built and working. What limits the tracker now is coder time.
Versions 0.1 to 0.9
Instrument testing and development with preliminary results.
Version 1.0
Completing validation and publishing the codebook with its reliability figures.
External review
A panel review of the instrument’s content, after version 1.0.
Funders and reviewers: we are glad to walk you through the instrument, the system, and the calibration record.
[email protected]