Level 3: Controlled comparisonLearners: Higher-ed studentsOutcome: Learning or achievementPreprintStudy type: Quasi-experimental
2026-07-23 · arXiv (back catalog) · Study (preprint)
The availability of generative AI (GenAI) has raised concerns that college students substitute GenAI for cognitive effort, earning high grades without learning. If this "GenAI substitution hypothesis" is true, grades should rise disproportionately in GenAI-susceptible courses--those relying more on assessments like...
Why this label for Generative AI Availability, Grades, and Student Satisfaction at a Large Universi
- Level 3: Controlled comparison: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the difference-in-differences analysis compares courses before and after AI availability using baseline administrative data.
- Grader B: R5 design ceiling: difference-in-differences quasi-experiment on administrative data (138,386 students) with a comparison and a pre-period gives Level 3; R7 preprint cap is 3; grades are a proxy for learning, so review for R6 indirectness
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive effort'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 3: Controlled comparisonProvisional (rules only)Review of many studies (44 studies)
2026-06-11 · ERIC (peer reviewed) · Study
Integrating technology in learning has been shown to positively influence mathematics outcomes across various educational levels. However, existing literature on the effectiveness of technology integration in mathematics education tends to focus on isolated aspects of technology's role, lacking a cohesive framework...
Why this label for The Effectiveness of Technology Integration in Secondary Mathematics: A Meta-Ana
- Level 3: Controlled comparison: provisional rules-only grade, pending grader consensus and review: R5 design (synthesis): Meta-analysis; capped at 3 as preprint; included-study quality needs human check.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 2: CorrelationalStudy type: Quasi-experimental
2026-10-19 · Frontiers in Education · Study
Effective communication is fundamental to high-quality healthcare. Communication skills training is a core component of medical curricula, yet opportunities to practice with standardized patients (SPs) are limited, costly, and can elicit anxiety in novice learners.
Why this label for Use of a GenAI-patient for medical communication training for undergraduate medi
- Level 2: Quasi-experiment, n=383, groups allocated by class schedule; outcomes are self-reported nervousness, preparedness and self-efficacy, so the perception-only cap holds it at Level 2.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Level 2: CorrelationalLearners: Higher-ed studentsOutcome: Learning or achievementPreprintPreregisteredSmall or impreciseStudy type: Cohort / case-control
2026-09-21 · arXiv (back catalog) · Study (preprint)
Students increasingly study alongside generative artificial intelligence (AI), yet unguided access to fluent answers invites cognitive offloading, and there is little evidence on which configurations of AI tutoring produce learning. Two design margins are usually bundled together: pedagogical structure (how the tutor...
Why this label for When AI Tutors Speak: Evidence from a Randomized Field Experiment
- Level 2: Correlational: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 randomized 86 MBA students with a learning outcome starts at Level 3; R6 imprecision lowers it because no interval or power information is reported.
- Grader B: R5: a preregistered randomized field experiment with a learning outcome starts at Level 3; R6 imprecision (n=86, under 100, and only p-values with no interval or power information) lowers it to Level 2.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive offloading'
- Tutoring and personalization: matched 'Tutors'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: K-12 studentsOutcome: Learning or achievementPreprintShort exposureResearcher-made measure
2026-07-14 · arXiv (back catalog) · Study (preprint)
This study proposes a semi-automated system for generating dialogue-based lessons using Large Language Models (LLMs) and Text-to-Speech (TTS) technology, and exploratorily examines its educational potential via a practical quasi-experiment. The system augments rather than replaces educators through a three-stage...
Why this label for A Semi-Automated System for Generating Dialogue-Based TTS Lessons Using Large La
- Level 2: Correlational: Change to 2. Within-subject comparison of three lesson formats with 245 high school students; no baseline equivalence or counterbalancing reported, so 2.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 quasi-experiment lacks an equivalent baseline comparison because lesson formats were given in fixed order with different content.
- Grader B: R5/R6: a fixed-order within-subject quasi-experiment with 245 high school students. Format is confounded with content and there is no equivalent comparison, so the ceiling is Level 2. The measurement downgrade (researcher-made outcomes with no reliability reported) lowers it to Level 1.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive load'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryReview of many studiesStudy type: Systematic review / meta-analysis
2026-10-19 · arXiv new: cs.CY, cs.HC, cs.CL, cs.AI · Study (preprint)
Generative artificial intelligence is increasingly transforming education by enabling adaptive learning environments, personalized instructional support, and collaborative knowledge creation, while also assisting research, assessment, and academic administration. In the context of fifth generation education systems,...
Why this label for Generative artificial intelligence in fifth-generation education systems: A syst
- Level 1: Systematic review with bibliometric analysis of adoption trends and challenges; abstract reports no controlled-study inclusion, risk of bias or effect synthesis, so it is held at Level 1.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Level 1: Descriptive or exploratoryStudy type: Case study / case series
2026-10-19 · Frontiers in Education · Study
This study presents a Design-Based Research (DBR) framework integrating Digital Twin development into undergraduate engineering education through Project-Based Learning. Sixty students across eighteen teams worked in project teams toward a multi-layer Digital Twin architecture for a manufacturing process, spanning...
Why this label for Integrating multi-layer digital twins into engineering education: a design-based
- Level 1: Design-based research with 60 students and a rubric assessment, no comparison group.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-10-19 · arXiv physics.ed-ph · Study (preprint)
This chapter examines the use of connectivism as a pedagogical approach in preservice physics teacher education through a classroom experience conducted in a Contemporary Physics course at the Universidad Pedagógica y Tecnológica de Colombia (UPTC). The study involved 21 preservice teachers organized into four...
Why this label for Connectivism as an Innovative Didactic Experience in Preservice Physics Teacher
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryStudy type: Case study / case series
2026-10-09 · arXiv search: AI + education · Study (preprint)
Generative AI is increasingly studied as a tool for supporting lesson authoring; the creation and adaptation of classroom materials, yet little is known about how it should fit into secondary computing teachers' subject-specific practices. We conducted two complementary qualitative studies with eight secondary...
Why this label for From Instructional Intent to Classroom-Ready Lessons: Investigating Teacher-AI C
- Level 1: Qualitative focus groups and think-aloud with eight teachers.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Level 1: Descriptive or exploratoryLearners: TeachersOutcome: Teacher practicePreprintSmall or impreciseStudy type: Case study / case series
2026-10-09 · arXiv (back catalog) · Study (preprint)
Generative AI is increasingly studied as a tool for supporting lesson authoring; the creation and adaptation of classroom materials, yet little is known about how it should fit into secondary computing teachers' subject-specific practices. We conducted two complementary qualitative studies with eight secondary...
Why this label for From Instructional Intent to Classroom-Ready Lessons: Investigating Teacher-AI C
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: focus groups and think-aloud sessions with eight teachers are qualitative exploratory research.
- Grader B: R5 design ceiling: qualitative focus groups and think-aloud with 8 teachers, so descriptive/exploratory Level 1
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-10-03 · arXiv physics.ed-ph · Study (preprint)
In physics instruction, equations are often treated primarily as tools for obtaining numerical results. Yet physicists also use them to describe dependencies, explain why those dependencies hold, predict what happens in new situations, test expressions against physical knowledge, and explore consequences that are not...
Why this label for Sensemaking of physics equations: Rationale, conceptualization, and core strateg
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
- Preprint: not peer reviewed; capped at Level 3
- Worked examples and explicit instruction: matched 'explicit instruction'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintPerception onlyStudy type: Case study / case series
2026-10-01 · arXiv (back catalog) · Study (preprint)
Timely and specific feedback is one of the strongest influences on student learning, yet it is difficult to sustain in large electrical engineering classes where the ratio of students to demonstrators is high and a learner who is stuck may wait days to find out why an approach was wrong. Generative Artificial...
Why this label for Feedback Without the Wait: Piloting a Generative AI Practice Platform in a Large
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the abstract describes a pilot of student engagement and perceptions without a comparison design.
- Grader B: R5 design ceiling: a single-course pilot reporting engagement and perceptions, so Level 1
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintResearcher-made measureSimulated learnersStudy type: Cross-sectional
2026-09-29 · arXiv (back catalog) · Study (preprint)
When a student is stuck, a tutor faces the assistance dilemma: help given too early can hinder productive struggle, while help withheld too long leaves the student in a frustrating, persistent impasse (i.e., wheel spinning). Generative AI tutors increasingly use guardrails restricting answer-giving, yet little is...
Why this label for Examining Variation in How Guided AI Tutors Resolve Student Impasses
- Level 1: Descriptive or exploratory: Real tutoring logs analyzed descriptively, then simulated conditions; graded on the real arm as 1.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: associations in authentic student sessions support a correlational level; simulated tutor conditions do not raise it.
- Grader B: R4 mixed study graded on the real-learner arm only: correlational analysis of authentic sessions (Level 2), downgraded under R6 measurement because the impasse/recovery coding has no reported reliability
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'Tutors'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintResearcher-made measureStudy type: Case study / case series
2026-09-28 · arXiv (back catalog) · Study (preprint)
The rapid proliferation of large language models (LLMs) in the context of education has introduced significant challenges in enforcement of academic integrity, especially in programming courses. We present Argus, an automated detection system for LLM-assisted student work in undergraduate C programming assignments.
Why this label for Argus: Academic Integrity in the Era of Generative AI
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: retrospective flagged-use and exam-performance findings report correlation without statistical controls.
- Grader B: R5 design ceiling: system description plus a simple correlation between detector flags and exam scores, with no statistical controls reported, so Level 1; the detector measure is unvalidated
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Perception or attitudePeer-reviewed (journal reference listed)Perception onlySmall or impreciseStudy type: Cross-sectional
2026-09-04 · arXiv (back catalog) · Study
The study investigates students' interest and expectations in a Big Data Engineering course integrated with a Master curricula, as well as ethical implications of using Big Data. An anonymous online survey was conducted with 42 of the 67 students enrolled in the Big Data course offered to Computer Science and...
Why this label for Students' Perception of Big Data Engineering in Higher Education Curricula: Expe
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 survey ceiling: 42 students reported interests and expectations; no learning outcome was measured.
- Grader B: R5 design ceiling: a one-time anonymous survey of 42 master's students with thematic analysis is descriptive (Level 1); R7 perception-only cap also applies; the conference proceedings DOI and ISBN indicate it is peer-reviewed
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Adult learnersOutcome: Learning or achievementPeer-reviewed (journal reference listed)Small or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2026-09-01 · arXiv (back catalog) · Study
Does unrestricted AI access bypass the cognitive effort required for learning, or does it streamline knowledge acquisition? This paper reports on a study where we compare three designs for user-AI interaction in a learning context: (1) an unrestricted conversational bot like ChatGPT, (2) a pedagogically constrained...
Why this label for Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Lear
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling permits Level 2 for a pre/post comparison without stated equivalence; R6 imprecision and measurement lower it to Level 1 for 50 learners and an unvalidated immediate test.
- Grader B: R5 three-condition comparison with pre-test, capped at 3; R7 short-exposure cap applies; R6 imprecision (n=50, d and p only, no interval) lowers to 2; R6 measurement (researcher-made immediate post-test, no reliability) lowers to 1; construct flag: the authors say the chatbot's gains reflect answer retrieval rather than deeper learning
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive effort'
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Perception or attitudePreprintPerception onlySmall or impreciseStudy type: Case study / case series
2026-08-28 · arXiv (back catalog) · Study (preprint)
As generative AI (GenAI) use among students increases, educators face growing questions about how to support learning while addressing ethical and institutional concerns. This exploratory study examines a guided inquiry activity in which students co-designed a GenAI course policy.
Why this label for A Guided Inquiry Approach to Students Co-Designing Generative AI Course Policies
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: analysis of student artifacts and discussions is exploratory and has no controlled learning outcome.
- Grader B: R5 design ceiling: an exploratory qualitative analysis of student artifacts and discussions from one course activity; no comparison group
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: K-12 studentsOutcome: Behavior or engagementPreprintOpen data or materialsVendor-linkedResearcher-made measureStudy type: Case study / case series
2026-08-21 · arXiv (back catalog) · Study (preprint)
The Atom Learning Model (ALM) tokenises a school curriculum. 757 pages of GCSE and Further Mathematics material were read by machine into 1,934 atoms, each one thing a learner can do in a single step, ordered by 4,616 machine-written prerequisite links.
Why this label for Atom Learning Model (ALM): how a real classroom got tokenised
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the classroom deployment reports descriptive usage and question performance without a controlled comparison.
- Grader B: R5 design ceiling: a descriptive deployment report (373 pupils, 7 weeks) with no comparison group; it reports behavior and system metrics, not a learning effect
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseResearcher-made measureShort exposureStudy type: Case study / case series
2026-07-17 · arXiv (back catalog) · Study (preprint)
Generative AI (GenAI) is increasingly used by students for programming explanation, debugging, and assignment support. Yet unrestricted large language model (LLM) tutors can hallucinate, contradict course policy, reveal complete solutions, and foster passive dependence.
Why this label for EduGuard: A Safe RAG-Based LLM Tutor for Programming Education
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 controlled pilot with 10 learners has a Level 2 ceiling; R6 imprecision lowers it to Level 1.
- Grader B: R4 mixed study graded only on the real-learner arm. That arm is a counterbalanced pilot with 10 undergraduates and an immediate post-test only, and pilots are Level 1 under sec. 6. The benchmark part would be T.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'Tutor'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Model performancePreprintSimulated learnersSmall or imprecise
2026-07-15 · arXiv (back catalog) · Study (preprint)
This paper is an extension of a paper presented at the ICAART 2026 conference, which introduced LEA (Learning Engagement Assistant), an adaptive AI tutoring agent combining course-specific Retrieval-Augmented Generation (RAG) with structured Knowledge Component (KC) models across integrated Chat, Tutor, and Quiz...
Why this label for Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Eval
- Level 1: Descriptive or exploratory: Change to 1. First real classroom deployment with 8 students plus simulation; graded on the real-learner arm as descriptive.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: despite a classroom deployment with eight students, the reported evaluations measure system performance rather than learner outcomes.
- Grader B: R4 mixed study graded only on the real-learner arm, which is a descriptive classroom deployment with 8 students and no comparison group (Level 1). The RAGAS and simulation parts would be T.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'Tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2026-07-04 · arXiv (back catalog) · Study (preprint)
Writing is cognitively demanding and anxiety-provoking for English as a Foreign Language (EFL) learners, especially under time pressure. This paper presents The Brand War, a web-based gamified writing application combining competitive game mechanics with iterative GPT-4.1-powered formative feedback for undergraduate...
Why this label for The Brand War: A Gamified AI-Feedback System for Time-Limited EFL Writing
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the 29-student exploratory, single-session study lacks an equivalent comparison group.
- Grader B: R5 design ceiling: exploratory single-session, single-group within-student study (n=29) with AI-scored revisions; user vs non-user comparison is self-selected and not significant
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Feedback: matched 'formative feedback'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Teacher practicePreprintStudy type: Case study / case series
2026-07-02 · arXiv (back catalog) · Study (preprint)
Generative artificial intelligence (GenAI) has challenged the validity of unsupervised online assessment, especially in technical subjects where plausible answers can be produced with little effort. This paper reports lessons from designing and implementing an AI-aware, AI-testing assessment in a large second-year...
Why this label for Assessment Design in the GenAI Era: The X1-X2-X3 Assessment Pattern for Testing
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: this is an assessment-design account without measured comparative learning gains.
- Grader B: R5 design ceiling: design case report of an assessment method in one module; explicitly makes no claim of measured learning gains
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Perception or attitudePreprintSmall or impreciseStudy type: Case study / case series
2026-06-26 · arXiv (back catalog) · Study (preprint)
Large language models can complete most of the assignments in an introductory artificial intelligence course. This paper is an experience report on redesigning one such course, CSS~382 at the University of Washington Bothell, in response.
Why this label for Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a St
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the single-cohort course experience report describes student artifacts and objections without a comparison group.
- Grader B: R5 design ceiling: single-cohort experience report on a course redesign, with no outcome measurement or comparison
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintSmall or impreciseResearcher-made measureStudy type: Case study / case series
2026-04-16 · arXiv (back catalog) · Study (preprint)
Generative AI (GenAI) holds significant promise for advancing educational equity among ethnic minority students by broadening access to learning resources and mitigating linguistic barriers. However, these benefits are counterbalanced by the risk of cognitive laziness, whereby students may treat GenAI as an answer...
Why this label for Scaffolding Critical Engagement with GenAI: Transforming Ethnic Minority Prepara
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: design-based research with 78 students, discourse analysis, and paired pre/post self-efficacy measures has no comparison group.
- Grader B: R5 design ceiling: design-based research with a single group (n=78) and paired pre/post t-tests = Level 1; outcomes are discourse and self-efficacy, not achievement; accepted at AIED 2026.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-04-03 · ERIC (peer reviewed) · Study
Student performance fluctuates across learning, raising questions about what drives these changes and whether instruction equitably supports diverse learners. Although teachers likely shape these trajectories, the pathways linking student--teacher interaction to individual performance variation remain unclear.
Why this label for Student Teacher Inter Brain Coupling Forecast Cross Semester Academic Fluctuatio
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2025-07-08 · arXiv (back catalog) · Study (preprint)
Retrieval practice is a well-established pedagogical technique known to significantly enhance student learning and knowledge retention. However, generating high-quality retrieval practice questions is often time-consuming and labor intensive for instructors, especially in rapidly evolving technical subjects.
Why this label for Enhancing Student Learning with LLM-Generated Retrieval Practice Questions: An E
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 places the week-to-week comparison without equivalence data at Level 2; R6 imprecision lowers it for about 60 students with no interval or power information.
- Grader B: R5: a week with LLM-generated retrieval questions vs a week without (~60 students, no equivalence data) is Level 2, then the R6 imprecision downgrade (n<100, no interval) gives 1; one-week exposure and a researcher-made quiz
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Retrieval practice: matched 'Retrieval Practice'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-19 · arXiv new: cs.CY, cs.HC, cs.CL, cs.AI · Study (preprint)
Teachers increasingly configure educational chatbots through natural-language instructions, but how these bots fail in real-world use remains poorly understood. We introduce EduWild, a dataset of 1,726 conversations with 217 teacher-configured chatbots, each paired with the instructions and configuration behind it,...
Why this label for EduWild: How Teacher-Configured Chatbots Fail in Real-World Use and How to Fix T
- Technical test: R4: outcome is chatbot failure codes per conversation (model behavior), not student learning.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-08 · arXiv search: AI + education · Study (preprint)
It is assumed that a natural way to improve the pedagogical quality of large language model tutors is to define a metric of instructional performance and fine-tune the model against it. To test this strategy, we designed a metric of pedagogical adaptivity that scores each instructional decision in a learning sequence...
Why this label for The learner who does not learn: when optimizing a pedagogical metric degrades LL
- Technical test: R4: tutor outputs on 2,000 learner scenarios rated by educators; no learners.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-10-08 · arXiv (back catalog) · Study (preprint)
It is assumed that a natural way to improve the pedagogical quality of large language model tutors is to define a metric of instructional performance and fine-tune the model against it. To test this strategy, we designed a metric of pedagogical adaptivity that scores each instructional decision in a learning sequence...
Why this label for The learner who does not learn: when optimizing a pedagogical metric degrades LL
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: educators rate tutor outputs across learner scenarios; no real learners are assessed.
- Grader B: R4 real-learner gate: 2,000 simulated learner scenarios, with educators rating model outputs; no learners, so T
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-07 · arXiv search: AI + education · Study (preprint)
Corrective feedback is among the best-evidenced drivers of second-language acquisition, yet corrections delivered during lessons rarely accumulate into an actionable view of grammar mastery. Prompted frontier models can provide such a view from learner--tutor lesson transcripts, but they are costly at scale.
Why this label for Grammar Concept Annotation at Scale: Deployed Fine-Tuned Small Language Models O
- Technical test: Matt's decision: model benchmark; the online experiment reports engagement and business metrics, not learning.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; Technical test level decided by Matt
Graded by one AI grader from the abstract, on Matt's delegation; Technical test level decided by Matt. Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-07 · arXiv search: AI + education · Study (preprint)
Automatic Short Answer Scoring (ASAS) is central to NLP for Education. However, openly available benchmarks remain scarce, and existing datasets largely address how well students answer a question directly rather than how well they master underlying concepts (knowledge elements) such as thermal energy or epistemic...
Why this label for Alice: A Large-Scale German Benchmark for Rubric-Based Multi-Dimensional Automat
- Technical test: R4: short answer scoring benchmark of language models.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testProvisional (rules only)Study type: Technical test (no real learners)
2026-10-01 · arXiv physics.ed-ph · Study (preprint)
Earth's ionosphere is primarily generated by the Sun's ionization of atoms in the upper atmosphere. This plasma environment is highly dynamic, exhibits strong spatial and temporal, variations and significantly impacts power systems, navigation, and satellite and ground communications.
Why this label for A Student Sounding Rocket Experiment to Probe Electron Density in the Lower Iono
- Technical test: provisional rules-only grade, pending grader consensus and review: R4 real-learner gate (F1/F2): Outputs of models, simulations or audits; any humans act as raters, references or data sources, not learners.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Technical testProvisional (rules only)Study type: Technical test (no real learners)
2026-10-01 · arXiv physics.ed-ph · Study (preprint)
Multiple-choice questions are a staple of educational assessment due to their efficiency, but their diagnostic ability is inherently limited. Traditional psychometric techniques infer misconceptions by fitting models to predetermined data patterns, making them ineffective for analysing small, novel, or...
Why this label for Determining the degree of randomness in multiple-choice question distractors
- Technical test: provisional rules-only grade, pending grader consensus and review: R4 real-learner gate (F1/F2): Outputs of models, simulations or audits; any humans act as raters, references or data sources, not learners.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Technical testProvisional (rules only)Study type: Technical test (no real learners)
2026-09-28 · arXiv physics.ed-ph · Study (preprint)
The idea of engaging critically and creatively with physics theories is a stretch for many physicists and physics educators. Most physicists become enculturated into the belief that physics definitions, laws, and equations are read from the book of nature.
Why this label for Community agreements pedagogy in physics
- Technical test: provisional rules-only grade, pending grader consensus and review: R4 real-learner gate (F1/F2): Outputs of models, simulations or audits; any humans act as raters, references or data sources, not learners.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintOpen data or materialsStudy type: Technical test (no real learners)
2026-09-13 · arXiv (back catalog) · Study (preprint)
Teaching abstract theoretical computer science (TCS) concepts such as algorithm analysis and complexity theory is challenging because students must handle formal proofs and asymptotic reasoning that conventional resources rarely explain in an adaptive, on-demand way. We present AlgoRAG, a specialized...
Why this label for AlgoRAG: Retrieval-Augmented Generation for Theoretical Computer Science Educati
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: evaluates system answers to curated questions, not learner outcomes.
- Grader B: R4 real-learner gate: RAG system scored on 179 exam questions (ROUGE, pedagogical quality score); no students took part, so the claim about personalized instruction is untested
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-08-04 · arXiv (back catalog) · Study (preprint)
Large language models (LLMs) power educational applications from tutoring to essay scoring, but each is a point solution to a single task, and only recently have these point solutions been integrated into agents operating over a learning management system (LMS). Yet tutoring is long-horizon, since a learner improves...
Why this label for EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulate
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: the reported tutoring results come from simulated learners; the classroom check validates the simulation.
- Grader B: R4 real-learner gate: agent tutors are benchmarked on simulated learners; the live-classroom study only calibrates the simulator and reports no learning outcome from real learners.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintVendor-linkedStudy type: Technical test (no real learners)
2026-08-01 · arXiv (back catalog) · Study (preprint)
MagicSchool's K-12 AI product suite is used by millions of teachers and serves millions of teacher and student messages each month. Our team monitors output along four priority dimensions--student safety, tone and instructional role, pedagogical value, and structural output quality--and tracks how often it fails on...
Why this label for When Evaluators Cry Wolf: Lessons from Production LLM-as-Judge Evaluation in Edu
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: the reported outcomes evaluate model judges on synthetic cases, not student learning.
- Grader B: R4 real-learner gate: LLM-as-judge evaluators are tuned on synthetic datasets, and the only outcome is evaluator precision; the authors are from the vendor, MagicSchool.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Perception or attitudePreprintShort exposurePerception onlySmall or impreciseStudy type: Technical test (no real learners)
2026-08-01 · arXiv (back catalog) · Study (preprint)
This innovative practice full paper presents BoilerSketch, a TA-supervised, diagram-first GenAI practice and tablet interface for providing structured visual explanations in CS1 and early CS2 support settings. Large early computing courses routinely face a support bottleneck during labs and office hours because many...
Why this label for BoilerSketch: A TA-Supervised, Diagram-First GenAI Practice for Structured Diagr
- Technical test: Tool paper; no learner outcomes reported.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: instructional staff evaluated a prototype; no learner outcomes were studied.
- Grader B: R5 design ceiling: one 45-minute expert evaluation with a post-use survey of 21 instructional staff, with no comparison group, so descriptive Level 1 and no learner outcome
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Not reportedOutcome: Model performancePreprintSmall or impreciseStudy type: Technical test (no real learners)
2026-07-24 · arXiv (back catalog) · Study (preprint)
Most educational technology for children is built around visual interfaces, which excludes the many children worldwide who live with visual impairment -- an estimated 1.4 million children are blind and many more have low vision. We present Kutti AI, a voice-first learning companion designed so that audio is the...
Why this label for Kutti AI: A Voice-First, Offline-Capable Learning Companion with Real-Time Strug
- Technical test: Prototype with no evaluation with target children.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: the hackathon prototype has no reported evaluation with its intended child learners.
- Grader B: R5 design ceiling: hackathon prototype with informal qualitative observations and no evaluation with target children yet, so descriptive or pilot Level 1
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'adaptive learning'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Higher-ed studentsOutcome: Model performancePreprintOpen data or materialsStudy type: Technical test (no real learners)
2026-07-21 · arXiv (back catalog) · Study (preprint)
This full paper in the research-to-practice track presents methods for assessing student teams in tabletop exercises (TTXs). TTXs enable learner teams to prepare for workplace tasks and practice crisis responses, such as resolving cybersecurity incidents.
Why this label for Assessment in Team Problem-Solving Exercises in Computing Education
- Technical test: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: student team records are used to test assessment methods; the reported outcome is method performance.
- Grader B: R4 real-learner gate: data from 81 participants is used only to check clustering and LLM scoring against instructor scores; the only outcome is how well the methods perform, with no learner outcome (accepted at FIE 2026, peer-reviewed version not matched)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-07-14 · arXiv (back catalog) · Study (preprint)
The rise of Artificial Intelligence (AI) enables automatic analysis of large amounts of data. Previously time-consuming and labor-intensive tasks can be completed much more efficiently with the use of AI.
Why this label for Analyzing Curricular Pattern Complexity Using AI to Improve On-Time Graduation R
- Technical test: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: evaluates LLM analysis of curricula without reporting student outcomes.
- Grader B: R4 real-learner gate: LLMs analyze curriculum structures and suggest revisions. No learners took part and no graduation outcome was measured.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Model performancePreprintOpen data or materialsStudy type: Technical test (no real learners)
2026-07-09 · arXiv (back catalog) · Study (preprint)
Despite rapid AI adoption in education, rigorous evaluation of AI-powered educational (AIED) systems remains critically underdeveloped, particularly in second language (L2) education, one of the most common yet least evaluated AI applications. We introduce L2-Bench, an open-source benchmark of 1,000+ task-response...
Why this label for L2-Bench: An Evaluation Benchmark for Measuring LLM Capabilities in Second Langu
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: expert validation supports a benchmark whose measured outcome is LLM performance.
- Grader B: R4 real-learner gate: an LLM benchmark for L2 education tasks. The experts only validated the taxonomy, and no learners took part.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Higher-ed studentsOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-06-22 · arXiv (back catalog) · Study (preprint)
Large language models (LLMs) are increasingly used for estimating item difficulty in educational assessment. However, it remains unclear whether such estimates reflect how learners actually experience difficulty.
Why this label for The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: student responses serve as reference data; the evaluated outcome is LLM difficulty-rating performance.
- Grader B: R4 real-learner gate: the only outcome is how well LLM difficulty ratings match item difficulty; student responses (n=770) are only the ground truth
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-06-15 · arXiv (back catalog) · Study (preprint)
Large language models are increasingly proposed as educational tutors, yet stronger task-solving ability does not necessarily imply stronger learning support. Motivated by recent calls to measure the social impact of NLP systems in practice, we study whether public LLM tutoring benchmarks distinguish...
Why this label for Beyond Helpfulness: A Teaching-over-Solving Diagnostic for Measuring Educational
- Technical test: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: diagnostic compares model benchmark scores without learner outcomes.
- Grader B: R4 real-learner gate: secondary analysis of tutoring benchmark leaderboards; the only outcome is model performance
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'Tutors'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Higher-ed studentsOutcome: Model performancePeer-reviewed (journal reference listed)Study type: Technical test (no real learners)
2026-06-12 · arXiv (back catalog) · Study
Educational chatbots powered by large language models (LLMs) show promising effects on learning outcomes, yet most systems delegate pedagogical decisions such as content selection and didactic structuring implicitly to the LLM, making tutoring strategies difficult to trace, evaluate, and reproduce. This paper presents...
Why this label for A didactical-driven teacher assistant for a dimensional modeling course
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: authentic student questions are test inputs, while the outcome is system detection precision.
- Grader B: R4 real-learner gate: student questions are only test inputs; the only outcome is the system's detection precision
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-06-12 · arXiv (back catalog) · Study (preprint)
Large Language Model (LLM) based AI educational content generation systems are increasingly being developed, yet no standardised benchmark exists to systematically evaluate them. This study introduces LessonBench-V1, a benchmark dataset comprising 647 human-written lessons paired with LLM-based reverse-engineered...
Why this label for LessonBench-V1: A Benchmark Dataset for Evaluating AI Lesson Generation Agents
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: lesson-generation benchmark has no participating learners or learning outcome.
- Grader B: R4 real-learner gate: a benchmark dataset for lesson-generation agents with no learners
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePeer-reviewed (journal reference listed)Study type: Technical test (no real learners)
2026-05-14 · arXiv (back catalog) · Study
Generative AI increasingly supports educational design tasks, e.g., through Large Language Models (LLMs), demonstrating the capability to design assessment questions that are aligned with pedagogical frameworks (e.g., Bloom's taxonomy). However, they often rely on subjective or limited evaluation methods; focus...
Why this label for Small, Private Language Models as Teammates for Educational Assessment Design
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: the study evaluates model-generated assessment questions and judging, with no learners.
- Grader B: R4 real-learner gate: LLM/SLM question generation judged by models and expert ratings; human raters judging model outputs counts as T
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintOpen data or materialsStudy type: Technical test (no real learners)
2026-05-10 · arXiv (back catalog) · Study (preprint)
Large language models are increasingly used in K-12 education, but existing benchmarks mainly test exam question answering rather than understanding how curriculum knowledge is structured and visually presented. We call this capability curriculum cognition.
Why this label for K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training E
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: evaluates educational LLMs on curriculum benchmarks, not K-12 students.
- Grader B: R4 real-learner gate: curriculum knowledge-graph benchmark and fine-tuning corpus for LLMs; K-12 content but no learners took part
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryStudy type: Expert opinion / commentary
2026-10-19 · Frontiers in Education · Study
Artificial intelligence (AI) and learning analytics are transforming higher education by enabling personalised learning, adaptive support, and data-informed decision-making. These developments present significant opportunities for improving educational inclusion for students with learning difficulties, yet they also...
Why this label for Toward an AI- and learning-analytics-enabled inclusive support framework for stu
- Commentary: Conceptual framework paper (R3).
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
CommentaryStudy type: Expert opinion / commentary
2026-10-11 · arXiv search: AI + education · Study (preprint)
Large language models can produce fluent tutoring dialogue, but whether they actually help students learn remains unclear. We present a process-centered synthesis of LLM tutoring that follows the path from learner-state inference to pedagogical action, tutor training, evaluation, and student simulation.
Why this label for Do LLM Tutors Actually Teach? A Process-Centered Synthesis of the Tutoring Loop
- Commentary: Process-centered synthesis with a study-level audit; abstract does not state a systematic search, so R3 narrative review.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
CommentaryStudy type: Not a study
2026-09-30 · EdTech Digest · News
FETC unveils an agenda for human leadership in the age of AI. GUEST COLUMN | by Jennifer Womble These are extraordinary times in education.
Why this label for AI Cannot Lead Your District. Only People Can.
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-09-30 · arXiv physics.ed-ph · Study (preprint)
Undergraduate research experiences provide students with an opportunity to apply knowledge learned in the classroom to authentic scientific problems. In this work, we describe the development of a special-topics course in Plasma Physics and Engineering (PPE) at a small liberal arts college using a Course-Based...
Why this label for Teaching a plasma physics and engineering class at a liberal arts college using
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Not a study
2026-09-29 · EdSurge · News
As AI features flood K-12 curricula, educators are left to filter proven quality from marketing hype.
Why this label for Study: Edtech Is Rushing AI Integration Before Proving It Works
- Commentary: news summary of others' work; mentions 'Study', graded C until the cited study is itself graded
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-09-28 · arXiv physics.ed-ph · Study (preprint)
We present a traceable instructional module that maps documented learner difficulties in quantum superposition and measurement to explicit learning goals, classroom activities, and assessment evidence. Using the general two-level Hamiltonian as the organizing physics framework, the module implements this mapping...
Why this label for From Two-Level Hamiltonians to Quantum Superposition and Measurement: A Traceabl
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
- Preprint: not peer reviewed; capped at Level 3
- Assessment and grading: matched 'grading'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-09-16 · arXiv (back catalog) · Study (preprint)
Digital competence frameworks define the knowledge, skills, attitudes and values required for participation in digital society. The European Digital Competence Framework for Citizens (DigComp) and UNESCO digital literacy and artificial intelligence competency frameworks provide reference structures for policy,...
Why this label for From Digital Competence to Demonstrated Digital Capability: Positioning the Inte
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 explicitly conceptual comparison of digital-competence frameworks without participant outcomes.
- Grader B: R3 genre gate: a self-described conceptual paper comparing competency frameworks.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-09-03 · arXiv (back catalog) · Study (preprint)
Contributions: A reflection model suitable for the era of Generative Artificial Intelligence (GenAI) is introduced. The proposed model is an integrated model that extracts features from various existing models and also incorporates technological aspects of GenAI.
Why this label for The 5P Reflection Model for Education in the Generative Artificial Intelligence
- Commentary: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R3 genre gate: the paper proposes a reflection model from existing literature without reporting learner outcomes.
- Grader B: R3 genre gate: it is labelled design-based research, but it analyses existing reflection models and proposes a new one; no participants or outcome data
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-08-27 · arXiv (back catalog) · Study (preprint)
Generative artificial intelligence (GenAI) has been primarily framed as an impartial educational tool. However, this framing overlooks an even larger shift: the reassignment of epistemological authority from teachers to students to machines.
Why this label for Reclaiming Epistemic Agency: A Critical Framework for Human-Generative AI Co-Age
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 genre gate: the paper presents a conceptual evaluation and framework without an empirical study.
- Grader B: R3 genre gate: a conceptual evaluation of theories that proposes a framework; no empirical study
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryStudy type: Not a study
2026-08-17 · eSchool News · News
A year ago, many educators were still asking whether artificial intelligence belonged in the classroom. Some were cautiously experimenting with AI-generated lesson plans, while others were focused on preventing students from using it altogether.
Why this label for 5 ways AI can strengthen your teaching this school year
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-08-05 · arXiv (back catalog) · Study (preprint)
As generative AI permeates computing instruction, the emergent design challenge is to configure each tool's pedagogical role, authority, and accountability for the instructional work it performs. We argue for instructional governance by design: governance should be encoded in a teaching tool's interaction model,...
Why this label for Instructional Governance by Design: A Framework for AI in Computing Education
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 genre gate: the paper explicitly develops a position and governance framework.
- Grader B: R3 genre gate: a position paper and framework illustrated with case descriptions, with no learner outcome data.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryLearners: Higher-ed studentsOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-07-08 · arXiv (back catalog) · Study (preprint)
We present documentation of a classroom assignment that teaches runtime monitoring through a creative embedded systems build: an interactive Pachinko game. The assignment centers on a dual-core ESP32 workflow in which students write RTLola specifications for monitors, compile these monitors to C, and deploy them...
Why this label for Experiential Learning of Runtime Monitoring Using Pachinko
- Commentary: Assignment documentation with no study.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: documents a classroom assignment without reporting a measured learning outcome.
- Grader B: R3 genre gate: documentation of a classroom assignment with no data or evaluation of student outcomes.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Higher-ed studentsOutcome: Behavior or engagementPeer-reviewed (journal reference listed)Report (grey literature)Study type: Expert opinion / commentary
2026-07-06 · arXiv (back catalog) · Study
This paper examines the psychological determinants that shape academically honest and dishonest uses of generative artificial intelligence (GenAI) in higher education. Rather than treating academic misconduct as a purely technological problem, the study conceptualizes academic integrity as a psychologically mediated...
Why this label for Psychological Determinants of Academic Integrity in the Use of Generative AI in
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 genre gate: the paper is a narrative review and conceptual synthesis.
- Grader B: R3 genre gate: self-described focused narrative review and conceptual synthesis of a purposive corpus of 16 sources, not a systematic review
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryStudy type: Not a study
2026-04-15 · eSchool News · News
Last fall, during a professional development session I was running with a group of teachers in São Paulo, a fifth-grade teacher raised her hand and asked a question I have since heard in every country I work in: “I want to use AI to plan better lessons. But how do I do that without just putting kids in front of...
Why this label for The screen-time debate’s blind spot
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Not a study
2026-04-10 · eSchool News · News
Most project-based learning workshops are built around three domains: design, assessment, and implementation.
Why this label for A new need-to-know for the AI classroom
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-02-12 · ERIC (peer reviewed) · Study
The rapid proliferation of telemedicine, accelerated by global demands for decentralized healthcare, necessitates a paradigm shift in medical education. Traditional clinical training models, centered on in-person encounters, are increasingly misaligned with the realities of modern practice.
Why this label for Transforming Medical Training through Telemedicine Simulation and Remote Consult
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate (F7): Self-described conceptual, legal, policy or reflective work with no learner data.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Not a study
2026-01-27 · eSchool News · News
Picture someone sitting at a kitchen table after the kids are finally in bed, laptop open, half-drunk mug of herbal tea nearby. For years, she has had a vague idea for a business--custom curriculum design for small learning pods, for example, or a micro-studio creating bespoke art for local nonprofits.
Why this label for 2026 prediction: AI may unleash the most entrepreneurial generation we’ve ever s
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
No study citedStudy type: Not a study
2026-09-22 · EdSurge · Launch / announcement
The CZI-backed initiative helps educators and developers use state standards, curricula, and datasets to create and evaluate classroom AI, but critics ...
Why this label for Learning Commons Launches Open Platform for K-12 Edtech
- No study cited: launch or announcement; no traceable study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
No study citedLearners: Not reportedOutcome: Not reportedPreprintStudy type: Not a study
2026-06-05 · arXiv (back catalog) · Study (preprint)
Critique is central to design education and project-based learning, yet high-quality critique is often scarce, uneven, hard to document, and disconnected from evolving artefacts. We present Critsly, an artefact-aware AI critique workspace that turns AI from a detached feedback tool into a critique teammate in the...
Why this label for Critsly: An Artefact-Aware AI Critique Teammate for Design Education and Project
- No study cited: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R1 source gate: product demo describes features but cites no evaluation study.
- Grader B: R1 source gate: a demo describing a tool, with no evaluation or study of learners
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
No study citedLearners: Not reportedOutcome: Not reportedPreprintStudy type: Not a study
2026-06-05 · arXiv (back catalog) · Study (preprint)
Teachers and researchers need to adapt educational AI to local goals, but most systems remain difficult to customize or study without coding expertise. We present LearnAdapt Agentic Studio on PedOS 1.1 Lumina, a no-code authoring and governed runtime environment for educational AI plugins.
Why this label for From Tools to Teacher-Built Teammates: No-Code Pedagogical Plugin Authoring with
- No study cited: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R1 source gate: product demo describes a workflow but cites no evaluation study.
- Grader B: R1 source gate: a demo describing an authoring platform, with no evaluation or study
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Not yet classified
2026-08-28 · Computers & Education (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Computers & Education by Chen-Chung Liu, Chin-Chung Tsai, Ramazan Yilmaz et al. Available online 28 August 2026. Abstract on the publisher page.
Why this label for The design framework for pedagogical agents supporting constructivist learning w
- Not yet classified: publisher listing with no abstract yet
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Not yet classified
Listed for the February 2027 issue · Computers & Education (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Computers & Education, Volume 257 by Ari Kim. Issue: February 2027. Abstract on the publisher page.
Why this label for Designing a pipeline linking professional development to AI-supported instructio
- Not yet classified: publisher listing with no abstract yet
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Not yet classified
Listed for the January 2027 issue · Computers & Education (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Computers & Education, Volume 256 by Hongfeng Zhang, Liwei Ding, Jinqiao Zhou. Issue: January 2027. Abstract on the publisher page.
Why this label for Unpacking AI - education discourse: Topic modeling and fsQCA of Technology - Res
- Not yet classified: publisher listing with no abstract yet
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Not yet classified
Listed for the December 2026 issue · Teaching and Teacher Education (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Teaching and Teacher Education, Volume 184 by Amy Walker, Nicole Garcia Diaz. Issue: December 2026. Abstract on the publisher page.
Why this label for Walking toward transformation: Experiential pedagogy and preservice middle grade
- Not yet classified: publisher listing with no abstract yet
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
