Level 3: Controlled comparisonLearners: Higher-ed studentsOutcome: Learning or achievementPreprintStudy type: Quasi-experimental
2026-07-23 · arXiv (back catalog) · Study (preprint)
The availability of generative AI (GenAI) has raised concerns that college students substitute GenAI for cognitive effort, earning high grades without learning. If this "GenAI substitution hypothesis" is true, grades should rise disproportionately in GenAI-susceptible courses--those relying more on assessments like...
Why this label for Generative AI Availability, Grades, and Student Satisfaction at a Large Universi
- Level 3: Controlled comparison: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the difference-in-differences analysis compares courses before and after AI availability using baseline administrative data.
- Grader B: R5 design ceiling: difference-in-differences quasi-experiment on administrative data (138,386 students) with a comparison and a pre-period gives Level 3; R7 preprint cap is 3; grades are a proxy for learning, so review for R6 indirectness
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive effort'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalStudy type: Quasi-experimental
2026-10-19 · Frontiers in Education · Study
Effective communication is fundamental to high-quality healthcare. Communication skills training is a core component of medical curricula, yet opportunities to practice with standardized patients (SPs) are limited, costly, and can elicit anxiety in novice learners.
Why this label for Use of a GenAI-patient for medical communication training for undergraduate medi
- Level 2: Quasi-experiment, n=383, groups allocated by class schedule; outcomes are self-reported nervousness, preparedness and self-efficacy, so the perception-only cap holds it at Level 2.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Level 2: CorrelationalLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintPreregisteredPerception onlyStudy type: Cohort / case-control
2026-10-07 · arXiv (back catalog) · Study (preprint)
AI learning tools are rapidly entering classrooms, but evidence about whether they help students learn is mixed and rests mostly on test scores. Comparatively less research addresses whether the use of AI changes students' live learning behaviors in class.
Why this label for How assigned AI use before class shapes active student engagement in class
- Level 2: Correlational: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R7 hard cap: randomized assignment is reported, but the measured outcome is classroom engagement rather than learning.
- Grader B: R7 engagement-only cap: a preregistered randomized field experiment (n=759 MBA students) that would otherwise be Level 3, but outcomes are class contributions and self-reported perceptions, with no learning measure, so max 2
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Motivation and engagement: matched 'student engagement'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: K-12 studentsOutcome: Learning or achievementPeer-reviewed (journal reference listed)Researcher-made measureClass-assigned, no clustering adjustment
2026-07-26 · arXiv (back catalog) · Study
Climate change is a socio-scientific issue: it rests on science but cannot be settled by science, because any serious response forces people to weigh costs, values, and competing interests under uncertainty. Helping students make such decisions well is a central aim of science education, and the arrival of generative...
Why this label for Can AI-Assisted Inquiry Enhance Students' Decision-Making Skills in Socio-Scient
- Level 2: Correlational (human override): Full paper (arXiv 2608.28617, Methods and Limitations) says intact classes were assigned to conditions and outcomes were tested at the student level; clustering is discussed only as a limitation, with no clustering-adjusted analysis. Our method lowers a level for class assignment without adjustment.
- Changed from: Level 3: Controlled comparison
- Grader A: R5 design ceiling: three student groups had pretest data and comparable starting levels; random assignment is not reported.
- Grader B: R5 design ceiling: three-group pretest-posttest quasi-experiment (270 secondary students, 90 per group, comparable baseline) gives Level 3; inter-coder agreement reported, so no R6 measurement downgrade; n>=100 so no imprecision downgrade; not an RCT, so not a Level 4 candidate
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Human grade change 2026-10-21, approved by Matthew Rhoads, Ed.D. (approved in chat, 2026-10-21) Method
Level 2: CorrelationalLearners: K-12 studentsOutcome: Learning or achievementPreprintShort exposureResearcher-made measure
2026-07-14 · arXiv (back catalog) · Study (preprint)
This study proposes a semi-automated system for generating dialogue-based lessons using Large Language Models (LLMs) and Text-to-Speech (TTS) technology, and exploratorily examines its educational potential via a practical quasi-experiment. The system augments rather than replaces educators through a three-stage...
Why this label for A Semi-Automated System for Generating Dialogue-Based TTS Lessons Using Large La
- Level 2: Correlational: Change to 2. Within-subject comparison of three lesson formats with 245 high school students; no baseline equivalence or counterbalancing reported, so 2.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 quasi-experiment lacks an equivalent baseline comparison because lesson formats were given in fixed order with different content.
- Grader B: R5/R6: a fixed-order within-subject quasi-experiment with 245 high school students. Format is confounded with content and there is no equivalent comparison, so the ceiling is Level 2. The measurement downgrade (researcher-made outcomes with no reliability reported) lowers it to Level 1.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive load'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseStudy type: Cohort / case-control
2026-06-21 · arXiv (back catalog) · Study (preprint)
Pretesting - attempting problems before instruction - supports learning by activating prior knowledge and sharpening attention to subsequent instruction. Recent work suggests that adaptive AI-assisted pretesting can yield further advantages, particularly for tasks requiring higher-order reasoning, yet it remains...
Why this label for Do Gains from Generative AI-Enabled Adaptive Pretesting Persist? Evidence from a
- Level 2: Correlational: Randomized retention study, but sample size not reported; held at 2 until n is verified.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: undergraduates were randomly assigned to study conditions and assessed on posttest performance.
- Grader B: R5 RCT (undergraduates randomly assigned, seven-week retention, learning outcome) gives Level 3; R6 imprecision lowers it to 2 because n and intervals are not reported (RCT without n, needs human review)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Retrieval practice: matched 'retrieval practice'
- Spacing and interleaving: matched 'spaced retrieval'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 2: CorrelationalLearners: Higher-ed studentsOutcome: Perception or attitudePreprintPerception onlyStudy type: Cross-sectional
2026-01-02 · arXiv (back catalog) · Study (preprint)
The development of effective autograders is key for scaling assessment and feedback. While NLP based autograding systems for open-ended response questions have been found to be beneficial for providing immediate feedback, autograders are not always liked, understood, or trusted by students.
Why this label for The Effect of Transparency on Students' Perceptions of AI Graders
- Level 2: Correlational: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling and R7 outcome cap: students' attitudes are compared with a control condition, but assignment and baseline equivalence are not reported, and outcomes are perception only.
- Grader B: R7 perception-only cap: transparency vs control comparison measuring only student attitudes toward autograders = max Level 2; n not reported; accepted to SIGCSE 2027.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryReview of many studiesStudy type: Systematic review / meta-analysis
2026-10-19 · arXiv new: cs.CY, cs.HC, cs.CL, cs.AI · Study (preprint)
Generative artificial intelligence is increasingly transforming education by enabling adaptive learning environments, personalized instructional support, and collaborative knowledge creation, while also assisting research, assessment, and academic administration. In the context of fifth generation education systems,...
Why this label for Generative artificial intelligence in fifth-generation education systems: A syst
- Level 1: Systematic review with bibliometric analysis of adoption trends and challenges; abstract reports no controlled-study inclusion, risk of bias or effect synthesis, so it is held at Level 1.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Level 1: Descriptive or exploratoryStudy type: Case study / case series
2026-10-19 · Frontiers in Education · Study
This study presents a Design-Based Research (DBR) framework integrating Digital Twin development into undergraduate engineering education through Project-Based Learning. Sixty students across eighteen teams worked in project teams toward a multi-layer Digital Twin architecture for a manufacturing process, spanning...
Why this label for Integrating multi-layer digital twins into engineering education: a design-based
- Level 1: Design-based research with 60 students and a rubric assessment, no comparison group.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-10-03 · arXiv physics.ed-ph · Study (preprint)
In physics instruction, equations are often treated primarily as tools for obtaining numerical results. Yet physicists also use them to describe dependencies, explain why those dependencies hold, predict what happens in new situations, test expressions against physical knowledge, and explore consequences that are not...
Why this label for Sensemaking of physics equations: Rationale, conceptualization, and core strateg
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
- Preprint: not peer reviewed; capped at Level 3
- Worked examples and explicit instruction: matched 'explicit instruction'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-10-02 · arXiv physics.ed-ph · Study (preprint)
Public Exams are an assessment system designed in the context of Biology Education Research to increase student engagement in the exam design process. Students are shown incomplete versions of test questions ahead of time, with the format of the question revealed but some details withheld.
Why this label for Adapting Public Exams for Introductory Physics: A Mixed-Methods Investigation of
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
- Preprint: not peer reviewed; capped at Level 3
- Motivation and engagement: matched 'student engagement'
- Assessment and grading: matched 'Exams'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPeer-reviewed (journal reference listed)Study type: Cross-sectional
2026-10-01 · arXiv (back catalog) · Study
Large language models can generate plausible solutions to programming assignments, making it tempting to detect their use by matching student code against a reference bank of generated solutions. Yet similar code can also arise when an assignment admits only a few natural implementations, which leaves open what a...
Why this label for Correctness, Convergence, and AI-Generated Code Detection: A Longitudinal Study
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: retrospective comparisons of student submissions and generated code are descriptive, with no controlled learner assignment.
- Grader B: R5 design ceiling: descriptive longitudinal analysis of real student code submissions across cohorts compared with LLM code; no comparison design, so Level 1 (accepted at Koli Calling, DOI)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintPerception onlyStudy type: Case study / case series
2026-10-01 · arXiv (back catalog) · Study (preprint)
Timely and specific feedback is one of the strongest influences on student learning, yet it is difficult to sustain in large electrical engineering classes where the ratio of students to demonstrators is high and a learner who is stuck may wait days to find out why an approach was wrong. Generative Artificial...
Why this label for Feedback Without the Wait: Piloting a Generative AI Practice Platform in a Large
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the abstract describes a pilot of student engagement and perceptions without a comparison design.
- Grader B: R5 design ceiling: a single-course pilot reporting engagement and perceptions, so Level 1
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Model performancePreprintSmall or imprecise
2026-09-29 · arXiv (back catalog) · Study (preprint)
AI-assisted cheating has become an important threat to the security of online exams. This study examines whether the risk of AI-assisted cheating in the final exam can be predicted using students' digital traces in the learning management system (LMS) during the first eight weeks of the semester.
Why this label for Early Prediction of AI-Assisted Cheating Risk in Online Exams Through Learning A
- Level 1: Descriptive or exploratory: Change to 1. Predictive analysis of LMS logs from 52 students; correlational design downgraded once for imprecision (n under 100, no interval).
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: student logs train a cheating-risk predictor, and predictive accuracy is the reported outcome.
- Grader B: R5 predictive/correlational design (Level 2), downgraded under R6 imprecision: n=52 with no interval information
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintPerception onlyStudy type: Case study / case series
2026-09-22 · arXiv (back catalog) · Study (preprint)
This paper presents a multi-source classroom study conducted during a 10-week quarter in data science courses at Drexel University. We first investigate the behaviors of student engagement with large language models (LLMs) using four surveys across three data science courses.
Why this label for Student Use of LLMs and the Limits of AI-Generated Question Difficulty in Data S
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 classroom surveys describe student use and perceptions; response data also test LLM question ratings without a learner comparison.
- Grader B: R5: surveys and an item-difficulty correlational analysis in a 10-week course with no comparison group are descriptive (Level 1); the R7 perception/engagement cap also applies to learning claims.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Retrieval practice: matched 'retrieval practice'
- Motivation and engagement: matched 'student engagement'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2026-09-18 · arXiv (back catalog) · Study (preprint)
Generative AI can improve students' programming performance, but successful task completion may not reflect what they retain. We examined performance, retention, cognitive load, and ownership in a controlled between-subjects experiment with 59 undergraduate computer science students, 55 were retained for analysis.
Why this label for Your Programming Students' Cognition with ChatGPT: Higher Performance, Lower Ret
- Level 1: Descriptive or exploratory: Between-subjects experiment, 55 analyzed, no baseline equivalence and no interval; downgraded to 1.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 controlled comparison lacks explicit random assignment or baseline data; R6 imprecision lowers Level 2 for 55 analyzed students without an interval or power report.
- Grader B: R5: a controlled between-subjects experiment with a recall (learning) outcome is Level 3; R6 imprecision (n=55, no intervals) lowers it to Level 2. R7 short-exposure cap also noted.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive load'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Adult learnersOutcome: Learning or achievementPeer-reviewed (journal reference listed)Small or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2026-09-01 · arXiv (back catalog) · Study
Does unrestricted AI access bypass the cognitive effort required for learning, or does it streamline knowledge acquisition? This paper reports on a study where we compare three designs for user-AI interaction in a learning context: (1) an unrestricted conversational bot like ChatGPT, (2) a pedagogically constrained...
Why this label for Socrates went Nuclear: Comparing Interaction Strategies for AI systems in a Lear
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling permits Level 2 for a pre/post comparison without stated equivalence; R6 imprecision and measurement lower it to Level 1 for 50 learners and an unvalidated immediate test.
- Grader B: R5 three-condition comparison with pre-test, capped at 3; R7 short-exposure cap applies; R6 imprecision (n=50, d and p only, no interval) lowers to 2; R6 measurement (researcher-made immediate post-test, no reliability) lowers to 1; construct flag: the authors say the chatbot's gains reflect answer retrieval rather than deeper learning
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive effort'
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintResearcher-made measureSmall or impreciseStudy type: Case study / case series
2026-08-25 · arXiv (back catalog) · Study (preprint)
Worked examples are an important part of introductory programming, but reading their expert explanations is passive. Self explanation, students explaining the problem and its solution to themselves with subgoal level analysis, converts passive reading into an active study of worked example, yet it is hard to scale...
Why this label for Self-Explanation Tutor for Active Study of CS1 Worked Examples
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the course deployment reports student revisions and learning evidence without a stated comparison group.
- Grader B: R4 mixed study, so only the real-learner arm is graded (the LLM-vs-human-rater arm is T); that arm is a single-group course deployment, so R5 sets Level 1
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Worked examples and explicit instruction: matched 'Worked Examples'
- Tutoring and personalization: matched 'Tutor'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: K-12 studentsOutcome: Behavior or engagementPreprintOpen data or materialsVendor-linkedResearcher-made measureStudy type: Case study / case series
2026-08-21 · arXiv (back catalog) · Study (preprint)
The Atom Learning Model (ALM) tokenises a school curriculum. 757 pages of GCSE and Further Mathematics material were read by machine into 1,934 atoms, each one thing a learner can do in a single step, ordered by 4,616 machine-written prerequisite links.
Why this label for Atom Learning Model (ALM): how a real classroom got tokenised
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the classroom deployment reports descriptive usage and question performance without a controlled comparison.
- Grader B: R5 design ceiling: a descriptive deployment report (373 pupils, 7 weeks) with no comparison group; it reports behavior and system metrics, not a learning effect
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: K-12 studentsOutcome: Perception or attitudePreprintSmall or imprecisePerception onlyStudy type: Case study / case series
2026-08-19 · arXiv (back catalog) · Study (preprint)
K-12 students often possess creative engineering ideas but lack technical skills to build them, while undergraduates have coding expertise but few opportunities to lead real-world projects or mentor others. The rapid development of AI-assisted tools offers a potential bridge to connect these groups, yet the structure...
Why this label for Multi-Tier Mentorship with AI-Assisted Development: Authentic Engineering for K-
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the nine-person pilot reports surveys and thematic findings without a comparison group.
- Grader B: R5 design ceiling: a pilot with 4 high-school students and 3 undergraduates, using surveys and thematic analysis; descriptive, and R7 perception-only applies
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Perception or attitudePeer-reviewed (journal reference listed)Perception onlyStudy type: Cross-sectional
2026-07-22 · arXiv (back catalog) · Study
The rise of generative AI (GenAI) in higher education has prompted urgent debates surrounding academic integrity and ethical use. This study examines cross-cultural differences in student perceptions of GenAI use, comparing responses from students at Canadian and South Korean universities.
Why this label for Did Alice Do Wrong? Cross-Cultural Differences in Student Perceptions of Generat
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the scenario survey compares student judgments but reports no controlled intervention or statistical controls.
- Grader B: R5 design ceiling: cross-sectional scenario-based survey comparing Canadian and Korean students' perceptions, so descriptive Level 1 with a perception-only outcome
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseResearcher-made measureShort exposureStudy type: Case study / case series
2026-07-17 · arXiv (back catalog) · Study (preprint)
Generative AI (GenAI) is increasingly used by students for programming explanation, debugging, and assignment support. Yet unrestricted large language model (LLM) tutors can hallucinate, contradict course policy, reveal complete solutions, and foster passive dependence.
Why this label for EduGuard: A Safe RAG-Based LLM Tutor for Programming Education
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 controlled pilot with 10 learners has a Level 2 ceiling; R6 imprecision lowers it to Level 1.
- Grader B: R4 mixed study graded only on the real-learner arm. That arm is a counterbalanced pilot with 10 undergraduates and an immediate post-test only, and pilots are Level 1 under sec. 6. The benchmark part would be T.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'Tutor'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: TeachersOutcome: Perception or attitudePreprintSmall or impreciseStudy type: Case study / case series
2026-07-17 · arXiv (back catalog) · Study (preprint)
While generative AI tools are directly changing how undergraduate computer science is learned and taught, they are also reshaping the relationships between instructors and students. In contrast to existing tool-oriented research on how instructors view and adopt AI, this study investigates how instructors think about...
Why this label for To Police or to Guide: How Higher Education Computer Science Instructors Design
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: 13 instructor interviews are qualitative evidence.
- Grader B: R5 design ceiling: qualitative study of 13 semi-structured instructor interviews, which is descriptive (Level 1).
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Model performancePreprintSimulated learnersSmall or imprecise
2026-07-15 · arXiv (back catalog) · Study (preprint)
This paper is an extension of a paper presented at the ICAART 2026 conference, which introduced LEA (Learning Engagement Assistant), an adaptive AI tutoring agent combining course-specific Retrieval-Augmented Generation (RAG) with structured Knowledge Component (KC) models across integrated Chat, Tutor, and Quiz...
Why this label for Learning Engagement Assistant (LEA): Cross-Course Scalability and Classroom Eval
- Level 1: Descriptive or exploratory: Change to 1. First real classroom deployment with 8 students plus simulation; graded on the real-learner arm as descriptive.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: despite a classroom deployment with eight students, the reported evaluations measure system performance rather than learner outcomes.
- Grader B: R4 mixed study graded only on the real-learner arm, which is a descriptive classroom deployment with 8 students and no comparison group (Level 1). The RAGAS and simulation parts would be T.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'Tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Learning or achievementPreprintSmall or impreciseShort exposureResearcher-made measureStudy type: Case study / case series
2026-07-04 · arXiv (back catalog) · Study (preprint)
Writing is cognitively demanding and anxiety-provoking for English as a Foreign Language (EFL) learners, especially under time pressure. This paper presents The Brand War, a web-based gamified writing application combining competitive game mechanics with iterative GPT-4.1-powered formative feedback for undergraduate...
Why this label for The Brand War: A Gamified AI-Feedback System for Time-Limited EFL Writing
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the 29-student exploratory, single-session study lacks an equivalent comparison group.
- Grader B: R5 design ceiling: exploratory single-session, single-group within-student study (n=29) with AI-scored revisions; user vs non-user comparison is self-selected and not significant
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Feedback: matched 'formative feedback'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Teacher practicePreprintStudy type: Case study / case series
2026-07-02 · arXiv (back catalog) · Study (preprint)
Generative artificial intelligence (GenAI) has challenged the validity of unsupervised online assessment, especially in technical subjects where plausible answers can be produced with little effort. This paper reports lessons from designing and implementing an AI-aware, AI-testing assessment in a large second-year...
Why this label for Assessment Design in the GenAI Era: The X1-X2-X3 Assessment Pattern for Testing
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: this is an assessment-design account without measured comparative learning gains.
- Grader B: R5 design ceiling: design case report of an assessment method in one module; explicitly makes no claim of measured learning gains
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: K-12 studentsOutcome: Learning or achievementPreprintSmall or impreciseShort exposureStudy type: Case study / case series
2026-06-26 · arXiv (back catalog) · Study (preprint)
Generative AI systems (GenAI) are increasingly used by students as learning companions, yet little is known about how they regulate these tools in open-ended learning settings, where the goal is to improve understanding for later independent performance. This study examined Grade-9 students' use of a general-purpose...
Why this label for Changes in Help-Seeking Strategies Predict unaided Performance during AI-based M
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R6 imprecision: the controlled association uses 89 learners with computable trajectories and reports no interval or power information.
- Grader B: R5 correlational design with pre-test control (Level 2 ceiling) on Grade-9 students, lowered one level by R6 imprecision (analytic n=95/89, no interval reported)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Metacognition and self-regulation: matched 'self-regulated'
- Tutoring and personalization: matched 'tutor'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Perception or attitudePreprintSmall or impreciseStudy type: Case study / case series
2026-06-26 · arXiv (back catalog) · Study (preprint)
Large language models can complete most of the assignments in an introductory artificial intelligence course. This paper is an experience report on redesigning one such course, CSS~382 at the University of Washington Bothell, in response.
Why this label for Teaching Intro AI When the Tools Can Do the Homework: A Course Redesign and a St
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the single-cohort course experience report describes student artifacts and objections without a comparison group.
- Grader B: R5 design ceiling: single-cohort experience report on a course redesign, with no outcome measurement or comparison
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: K-12 studentsOutcome: Learning or achievementPreprintResearcher-made measureStudy type: Case study / case series
2026-06-26 · arXiv (back catalog) · Study (preprint)
Research indicates students require customized written composition feedback to enhance critical thinking and writing competence, yet teachers face barriers to delivering timely personalized guidance due to heavy workloads. To address this gap, this study developed the Writing Improvement and Smart Evaluation Agent...
Why this label for Using AI-Generated Feedback to Improve Critical Thinking and Writing Proficiency
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: three-month single-group intervention; no comparison group reported.
- Grader B: R5 design ceiling: a three-month intervention with 260 sixth graders, but no comparison group is described, so it is a single-group pre/post (Level 1)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Feedback: matched 'AI-Generated Feedback'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Perception or attitudePreprintPerception onlyStudy type: Cross-sectional
2026-06-18 · arXiv (back catalog) · Study (preprint)
The rapid integration of generative artificial intelligence (AI) has reshaped the landscape of higher education. Students have embraced tools such as ChatGPT with striking speed, while teaching staff and institutions have responded with greater caution.
Why this label for From Novelty to Normalisation: Tracking Changing Perceptions of AI in Higher Edu
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: longitudinal survey tracks reported attitudes and use without an intervention comparison.
- Grader B: R5 design: a repeated survey (n=1,665) of perceptions and reported use, which is descriptive (Level 1)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: K-12 studentsOutcome: Model performancePreprint
2026-06-17 · arXiv (back catalog) · Study (preprint)
Personalised learning systems often assume that mathematical ability is combined of discrete abilities, acquired sequentially and dependent upon first acquiring foundational abilities, and students often report different strengths. In this work, we explore the validity of these assumptions by applying clustering...
Why this label for Archetypes or ability? Clustering for modelling student mathematical competence
- Level 1: Descriptive or exploratory: Change to 1. Secondary analysis of exam data from 119,034 real students; descriptive and modelling study, not a model benchmark.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: exam results train and benchmark predictive models; no learner outcome is evaluated.
- Grader B: R5 design: exploratory clustering of exam data from 119,034 students with no comparison or intervention, so it is descriptive (Level 1)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'Personalised learning'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Perception or attitudePreprintPerception onlyStudy type: Cross-sectional
2026-06-13 · arXiv (back catalog) · Study (preprint)
Peer grading is widely used in education, yet it elicits mixed reactions from educators and students. Although many studies have examined students' views of peer grading, their findings are scattered, and no clear overall picture has emerged.
Why this label for Students' Perceptions of Peer Grading
- Level 1: Descriptive or exploratory: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: thematic analysis of papers and student discussions is descriptive and reports perceptions, not measured learning.
- Grader B: R5 design ceiling: qualitative thematic analysis of 107 papers plus Reddit posts about students' views of peer grading, with no stated systematic search method; perception only (accepted at ECTEL 2026, not matched)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryLearners: Adult learnersOutcome: Learning or achievementPreprintSmall or impreciseShort exposureStudy type: Cross-sectional
2026-05-07 · arXiv (back catalog) · Study (preprint)
Advance in technology offer the potential for future adoption of a combination of virtual reality (VR) and real-time adaptivity to enhance training and education. Providing a valid neuro-ergonomic measure of cognitive load can enable an adaptive training regime to continuously adjust tas difficulty to an optimal level...
Why this label for Leveraging fNIRS to Evaluate Workload for Adaptive Training in Virtual Reality
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: 36 people complete one training task and retention test, with no comparison group reported.
- Grader B: R5 design ceiling: single-session study validating an fNIRS workload measure (n=36); the outcome is the cognitive-load measure, not a learning comparison
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Cognitive load: matched 'cognitive load'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-05-05 · ERIC (peer reviewed) · Study
Background: Academic resilience, defined as the capacity of socio-economically disadvantaged students to achieve at high academic levels, is a key indicator in international education policy. Large-scale assessments like the Programme for International Student Assessment (PISA) routinely report the prevalence of...
Why this label for Longitudinal Insights into Academic Resilience in PISA in Italy: The Role of Pri
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
- Assessment and grading: matched 'Student Assessment'
- Policy and governance: matched 'education policy'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryLearners: Higher-ed studentsOutcome: Behavior or engagementPreprintSmall or impreciseResearcher-made measureStudy type: Case study / case series
2026-04-16 · arXiv (back catalog) · Study (preprint)
Generative AI (GenAI) holds significant promise for advancing educational equity among ethnic minority students by broadening access to learning resources and mitigating linguistic barriers. However, these benefits are counterbalanced by the risk of cognitive laziness, whereby students may treat GenAI as an answer...
Why this label for Scaffolding Critical Engagement with GenAI: Transforming Ethnic Minority Prepara
- Level 1: Descriptive or exploratory: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: design-based research with 78 students, discourse analysis, and paired pre/post self-efficacy measures has no comparison group.
- Grader B: R5 design ceiling: design-based research with a single group (n=78) and paired pre/post t-tests = Level 1; outcomes are discourse and self-efficacy, not achievement; accepted at AIED 2026.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-04-15 · ERIC (peer reviewed) · Study
This study examined the associations between sedentary behavior (SB) at age 11 and mathematics ability at age 17, and assessed the mediating effect of internalizing and externalizing problems at age 14, using UK Millennium Cohort Study data (N = 3622; 53% male). Self-reported frequency of SB (listening to music,...
Why this label for Longitudinal Associations between Sedentary Behavior Types and Mathematics Abili
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-04-03 · ERIC (peer reviewed) · Study
Student performance fluctuates across learning, raising questions about what drives these changes and whether instruction equitably supports diverse learners. Although teachers likely shape these trajectories, the pathways linking student--teacher interaction to individual performance variation remain unclear.
Why this label for Student Teacher Inter Brain Coupling Forecast Cross Semester Academic Fluctuatio
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-03-30 · ERIC (peer reviewed) · Study
Background: Children with attention deficit hyperactivity disorder (ADHD) are vulnerable to the psychological impact of traumatic events due to inherent difficulties in emotion regulation and executive functioning. This study examines the predictive roles of attachment styles and family functioning on depressive,...
Why this label for Attachment Style and Family Functioning as Predictors of Trauma Symptoms in Chil
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Level 1: Descriptive or exploratoryProvisional (rules only)
2026-02-17 · ERIC (peer reviewed) · Study
Question-asking is central to human communication, cognitive development, and learning. While prior research has examined questioning in controlled, non-longitudinal settings, little is known about how the originality and complexity of the questions people ask change over time or how these abilities respond to...
Why this label for Knowledge Reshapes Inquiry by Changing Question Asking Ability and Impacting Aca
- Level 1: Descriptive or exploratory: provisional rules-only grade, pending grader consensus and review: R5 design: descriptive (F3): Descriptive, qualitative, pilot or deployment study with real participants.
- Assessment and grading: matched 'exam'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-10-15 · arXiv (back catalog) · Study (preprint)
Adaptive assessment systems personalize learning by estimating learner proficiency and recommending suitable questions. However, many existing approaches rely on fixed difficulty labels, static rating updates, or black-box prediction models that provide limited control over rating stability, interpretability, and...
Why this label for MARS: A Momentum-Aware Adaptive Rating System for Personalized Assessment
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: evaluation measures prediction and routing performance on logged and simulated data, not learner outcomes.
- Grader B: R4 real-learner gate: evaluated on a log dataset (ASSISTments) and a simulated exam (MARS-SimExam); the only outcomes are prediction, rating stability and routing scores, so it measures the model
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-14 · arXiv search: AI + education · Study (preprint)
Phonetic error detection (PED) aims to identify phoneme-level substitutions, deletions, and insertions and is essential for educational and clinical speech assessment and intervention. Despite recent advances in speech models for phoneme recognition, their ability to reliably detect fine-grained phonetic errors...
Why this label for PEDBench: Can Speech Models Detect Phonetic Errors?
- Technical test: R4: speech model benchmark; outcome is model performance.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-14 · arXiv search: AI + education · Study (preprint)
Law students are examined through open-text answers: case solutions, essays and memos that apply the law to a set of facts. Such answers are expensive to grade, human graders disagree widely on them, and large language models (LLMs) now offer to grade them at scale, both in teaching and in the benchmarks that test...
Why this label for SoK: Grading Open-Text Legal Answers, from Law School Exams to Trustworthy Rubri
- Technical test: Systematization of LLM grading paradigms; no learner outcome.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-13 · arXiv search: AI + education · Study (preprint)
The rapid advancement of image-to-text extraction systems and large language models (LLMs) poses new challenges for educational assessment by making assignment materials easy to transcribe and solve with minimal human effort. To address this problem, we present AIRRI (Artificial Intelligence Resilient Reading...
Why this label for AIRRI: Towards a Human-Readable, Machine-Resistant Reading Interface
- Technical test: R4: OCR degradation tests; readability check is not a learning outcome.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-11 · arXiv search: AI + education · Study (preprint)
AI-text detectors are already applied to Kazakh writing, yet no study reports how often publicly available detectors flag Kazakh text whose human authorship is dated before ChatGPT. We present AINA-KK: 3,811 human passages from encyclopedic, news, web and educational pages that provably existed before ChatGPT, and...
Why this label for Unchecked or Falsely Accused? Evaluating AI-Generated Text Detectors on Kazakh
- Technical test: R4: AI-text detector benchmark.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-08 · arXiv search: AI + education · Study (preprint)
It is assumed that a natural way to improve the pedagogical quality of large language model tutors is to define a metric of instructional performance and fine-tune the model against it. To test this strategy, we designed a metric of pedagogical adaptivity that scores each instructional decision in a learning sequence...
Why this label for The learner who does not learn: when optimizing a pedagogical metric degrades LL
- Technical test: R4: tutor outputs on 2,000 learner scenarios rated by educators; no learners.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-10-08 · arXiv (back catalog) · Study (preprint)
It is assumed that a natural way to improve the pedagogical quality of large language model tutors is to define a metric of instructional performance and fine-tune the model against it. To test this strategy, we designed a metric of pedagogical adaptivity that scores each instructional decision in a learning sequence...
Why this label for The learner who does not learn: when optimizing a pedagogical metric degrades LL
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: educators rate tutor outputs across learner scenarios; no real learners are assessed.
- Grader B: R4 real-learner gate: 2,000 simulated learner scenarios, with educators rating model outputs; no learners, so T
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-07 · arXiv search: AI + education · Study (preprint)
Large language models (LLMs) have already been shown to match or exceed human experts in persuasion. While their persuasive capabilities hold promise for beneficial uses such as education and health communication, they can also be used to manipulate and misinform, making their evaluation a growing priority for...
Why this label for LLM Persuasion Is in the Eye of the Evaluation
- Technical test: R4: automated persuasion evaluations across 15 LLMs.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-07 · arXiv search: AI + education · Study (preprint)
Automatic Short Answer Scoring (ASAS) is central to NLP for Education. However, openly available benchmarks remain scarce, and existing datasets largely address how well students answer a question directly rather than how well they master underlying concepts (knowledge elements) such as thermal energy or epistemic...
Why this label for Alice: A Large-Scale German Benchmark for Rubric-Based Multi-Dimensional Automat
- Technical test: R4: short answer scoring benchmark of language models.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-07 · arXiv search: AI + education · Study (preprint)
As Large Language Models (LLMs) are increasingly adopted for automated grading and feedback in higher education, their structured outputs, including multi-dimensional rubric scores, detailed feedback comments, and improvement suggestions, create an appearance of thorough analytic evaluation. This study examines...
Why this label for Beyond Score Accuracy: Examining the Diagnostic Quality of LLM-Generated Structu
- Technical test: R4: LLM grading vs instructor scores on an existing dataset; outcome is model performance.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Higher-ed studentsOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-10-07 · arXiv (back catalog) · Study (preprint)
As Large Language Models (LLMs) are increasingly adopted for automated grading and feedback in higher education, their structured outputs, including multi-dimensional rubric scores, detailed feedback comments, and improvement suggestions, create an appearance of thorough analytic evaluation. This study examines...
Why this label for Beyond Score Accuracy: Examining the Diagnostic Quality of LLM-Generated Structu
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: student responses provide test data, while the outcomes measure LLM grading and feedback quality.
- Grader B: R4 real-learner gate: compares LLM grades and feedback with instructors on existing student responses; the only outcome is model output quality
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Not a learning studyOutcome: Model performanceStudy type: Technical test (no real learners)
2026-10-06 · arXiv search: AI + education · Study (preprint)
In K-12 mathematics tutoring, student-tutor dialogue provides rich evidence of learners' problem-solving processes and sources of difficulty. Learning analytics research increasingly relies on large language models (LLMs) to extract such information from dialogue for a variety of downstream tasks, including knowledge...
Why this label for Agreement Is Not Validity: Cross-Model LLM Consensus in Diagnosing Student Failu
- Technical test: R4: validity of LLM classifications vs human coding; outcome is model performance.
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-10-02 · arXiv (back catalog) · Study (preprint)
AI can help adults produce plausible workplace artefacts while leaving their independent reasoning difficult to inspect. LearnAdapt Praxis addresses this design problem through a five-stage learning workspace: Frame, Learn, Build, Validate and Transfer.
Why this label for LearnAdapt Praxis: Controlled AI Assistance and Evidence Traces for Adult Workpl
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: synthetic project episodes test engineering properties; no adult learner outcomes are measured.
- Grader B: R4 real-learner gate: system checks run on a synthetic rehearsal with fictional contexts; the abstract says it does not establish learning gains
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testProvisional (rules only)Study type: Technical test (no real learners)
2026-10-01 · arXiv physics.ed-ph · Study (preprint)
Multiple-choice questions are a staple of educational assessment due to their efficiency, but their diagnostic ability is inherently limited. Traditional psychometric techniques infer misconceptions by fitting models to predetermined data patterns, making them ineffective for analysing small, novel, or...
Why this label for Determining the degree of randomness in multiple-choice question distractors
- Technical test: provisional rules-only grade, pending grader consensus and review: R4 real-learner gate (F1/F2): Outputs of models, simulations or audits; any humans act as raters, references or data sources, not learners.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-09-25 · arXiv (back catalog) · Study (preprint)
Oral assessment with generative AI requires more than a conversational interface: educators must connect a spoken response to its source material, scoring criteria, model outputs and subsequent human judgement. This technical report presents CALLIOPE, a source-grounded oral assessment system integrating versioned...
Why this label for CALLIOPE: A Source-Grounded Oral Assessment System and Synthetic Readiness Evalu
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 synthetic spoken fixtures test the assessment system; no human-participant outcomes are reported.
- Grader B: R4 real-learner gate: synthetic software verification with spoken fixtures; the abstract says no human-participant outcomes are reported.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Model performancePreprintResearcher-made measureStudy type: Technical test (no real learners)
2026-09-22 · arXiv (back catalog) · Study (preprint)
This study aimed to automate hierarchical diagnostic reasoning (HDR), a constructive method for evaluating practical judgment skills, by developing and testing an evaluation process using a large language model. HDR is a descriptive task that measures higher-order cognitive skills by requiring students to identify and...
Why this label for Automating Constructive Assessment with Large Language Models: Toward Scalable a
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 the reported results validate automated problem generation, scoring, and feedback against ratings, not learner outcomes.
- Grader B: R4 real-learner gate: the outcomes are LLM item-generation validity, human-AI scoring agreement and rater judgments of feedback, which is model performance only; no student learning outcome.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintOpen data or materialsStudy type: Technical test (no real learners)
2026-09-13 · arXiv (back catalog) · Study (preprint)
Teaching abstract theoretical computer science (TCS) concepts such as algorithm analysis and complexity theory is challenging because students must handle formal proofs and asymptotic reasoning that conventional resources rarely explain in an adaptive, on-demand way. We present AlgoRAG, a specialized...
Why this label for AlgoRAG: Retrieval-Augmented Generation for Theoretical Computer Science Educati
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: evaluates system answers to curated questions, not learner outcomes.
- Grader B: R4 real-learner gate: RAG system scored on 179 exam questions (ROUGE, pedagogical quality score); no students took part, so the claim about personalized instruction is untested
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-08-27 · arXiv (back catalog) · Study (preprint)
Reliable assessment of causal research designs in the social sciences is critical for evidence-based policy-making, yet has so far relied entirely on manual expert analysis. We introduce Automated Research Design Tracking and Assessment (ARDTrA), a task that involves detecting the research design used in a paper and...
Why this label for Research Design Tracking and Assessment for the Social Sciences
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: expert-annotated papers serve as a benchmark for an LLM research-design assessment pipeline.
- Grader B: R4 real-learner gate: RAG/LLM pipeline evaluated on an expert-annotated paper dataset; outcome is model performance (EMNLP 2026).
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-08-17 · arXiv (back catalog) · Study (preprint)
Radiology reports describe findings and locations in free text, but downstream search and analysis require these relations in a fixed schema. Learning this extraction requires labels that are unevenly distributed across institutions: smaller hospitals have less local evidence, and pooling data may be infeasible.
Why this label for FedPref: Federated Preference Learning for Structured Radiology Report Extractio
- Technical test: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: simulated hospitals and report extraction are evaluated by model F1.
- Grader B: R4 real-learner gate: federated model training on simulated hospitals, and the only outcome is extraction F1, which is model performance.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-08-04 · arXiv (back catalog) · Study (preprint)
Large language models (LLMs) power educational applications from tutoring to essay scoring, but each is a point solution to a single task, and only recently have these point solutions been integrated into agents operating over a learning management system (LMS). Yet tutoring is long-horizon, since a learner improves...
Why this label for EduClaw-Bench: A Long-Horizon Benchmark for Pedagogical LLM Agents with Simulate
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: the reported tutoring results come from simulated learners; the classroom check validates the simulation.
- Grader B: R4 real-learner gate: agent tutors are benchmarked on simulated learners; the live-classroom study only calibrates the simulator and reports no learning outcome from real learners.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'tutoring'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-08-04 · arXiv (back catalog) · Study (preprint)
An emerging literature examines the critical question of when and how prediction can be useful in allocating scarce societal resources. We examine a novel variant of this question: What happens when predictive uncertainty differs systematically across the population?
Why this label for Scarcity and Predictive Uncertainty: Implications for Societal Resource Allocati
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 technical gate: a mathematical allocation model is illustrated with a dataset, without learner outcomes.
- Grader B: R4 real-learner gate: a mathematical allocation model illustrated on the PISA dataset, with no learner intervention or learning outcome.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintVendor-linkedStudy type: Technical test (no real learners)
2026-08-01 · arXiv (back catalog) · Study (preprint)
MagicSchool's K-12 AI product suite is used by millions of teachers and serves millions of teacher and student messages each month. Our team monitors output along four priority dimensions--student safety, tone and instructional role, pedagogical value, and structural output quality--and tracks how often it fails on...
Why this label for When Evaluators Cry Wolf: Lessons from Production LLM-as-Judge Evaluation in Edu
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: the reported outcomes evaluate model judges on synthetic cases, not student learning.
- Grader B: R4 real-learner gate: LLM-as-judge evaluators are tuned on synthetic datasets, and the only outcome is evaluator precision; the authors are from the vendor, MagicSchool.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSmall or impreciseResearcher-made measureStudy type: Technical test (no real learners)
2026-08-01 · arXiv (back catalog) · Study (preprint)
Generative AI tools increasingly produce educational materials: documents, slides, images, audio, and video, yet little is known about whether this content meets accessibility requirements. This paper presents a protocol for evaluating the accessibility of AI-generated educational materials against the Web Content...
Why this label for A Protocol for Evaluating the Accessibility of AI-Generated Educational Material
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 technical gate: an exploratory test scores AI-generated artifacts, with no learner outcome.
- Grader B: R4 real-learner gate: a protocol scoring the accessibility of AI-generated materials, using one model and one evaluator, with no learners.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Higher-ed studentsOutcome: Model performancePreprintOpen data or materialsStudy type: Technical test (no real learners)
2026-07-21 · arXiv (back catalog) · Study (preprint)
This full paper in the research-to-practice track presents methods for assessing student teams in tabletop exercises (TTXs). TTXs enable learner teams to prepare for workplace tasks and practice crisis responses, such as resolving cybersecurity incidents.
Why this label for Assessment in Team Problem-Solving Exercises in Computing Education
- Technical test: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: student team records are used to test assessment methods; the reported outcome is method performance.
- Grader B: R4 real-learner gate: data from 81 participants is used only to check clustering and LLM scoring against instructor scores; the only outcome is how well the methods perform, with no learner outcome (accepted at FIE 2026, peer-reviewed version not matched)
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintSimulated learnersStudy type: Technical test (no real learners)
2026-07-10 · arXiv (back catalog) · Study (preprint)
Social desirability and impression management are pervasive sources of response distortion in human personality assessment, yet their effects on Large Language Models (LLMs) remain underexplored. This study investigates whether contemporary LLMs systematically modulate the expression of Dark Triad traits...
Why this label for Faking Good and Faking Bad in LLMs: Response Distortion Across Dark Triad Person
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: all experimental responses come from LLMs.
- Grader B: R4 real-learner gate: psychometric prompting of seven LLMs, with no human participants.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Adults not in a learning role (raters)Outcome: Model performancePreprintOpen data or materialsStudy type: Technical test (no real learners)
2026-07-09 · arXiv (back catalog) · Study (preprint)
Despite rapid AI adoption in education, rigorous evaluation of AI-powered educational (AIED) systems remains critically underdeveloped, particularly in second language (L2) education, one of the most common yet least evaluated AI applications. We introduce L2-Bench, an open-source benchmark of 1,000+ task-response...
Why this label for L2-Bench: An Evaluation Benchmark for Measuring LLM Capabilities in Second Langu
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: expert validation supports a benchmark whose measured outcome is LLM performance.
- Grader B: R4 real-learner gate: an LLM benchmark for L2 education tasks. The experts only validated the taxonomy, and no learners took part.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Higher-ed studentsOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-06-22 · arXiv (back catalog) · Study (preprint)
Large language models (LLMs) are increasingly used for estimating item difficulty in educational assessment. However, it remains unclear whether such estimates reflect how learners actually experience difficulty.
Why this label for The Easy Trap: Why LLMs Underestimate Misconception-Driven Difficulty
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: student responses serve as reference data; the evaluated outcome is LLM difficulty-rating performance.
- Grader B: R4 real-learner gate: the only outcome is how well LLM difficulty ratings match item difficulty; student responses (n=770) are only the ground truth
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-06-15 · arXiv (back catalog) · Study (preprint)
Large language models are increasingly proposed as educational tutors, yet stronger task-solving ability does not necessarily imply stronger learning support. Motivated by recent calls to measure the social impact of NLP systems in practice, we study whether public LLM tutoring benchmarks distinguish...
Why this label for Beyond Helpfulness: A Teaching-over-Solving Diagnostic for Measuring Educational
- Technical test: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R4 real-learner gate: diagnostic compares model benchmark scores without learner outcomes.
- Grader B: R4 real-learner gate: secondary analysis of tutoring benchmark leaderboards; the only outcome is model performance
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Tutoring and personalization: matched 'Tutors'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePeer-reviewed (journal reference listed)Study type: Technical test (no real learners)
2026-05-14 · arXiv (back catalog) · Study
Generative AI increasingly supports educational design tasks, e.g., through Large Language Models (LLMs), demonstrating the capability to design assessment questions that are aligned with pedagogical frameworks (e.g., Bloom's taxonomy). However, they often rely on subjective or limited evaluation methods; focus...
Why this label for Small, Private Language Models as Teammates for Educational Assessment Design
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: the study evaluates model-generated assessment questions and judging, with no learners.
- Grader B: R4 real-learner gate: LLM/SLM question generation judged by models and expert ratings; human raters judging model outputs counts as T
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintStudy type: Technical test (no real learners)
2026-05-12 · arXiv (back catalog) · Study (preprint)
Automatic generation of educational materials using large language models (LLMs) is becoming increasingly common, but assigning difficulty levels to such materials still requires substantial human effort. LLM-as-a-Judge has therefore attracted attention, yet disagreement with human raters remains a major challenge.
Why this label for Predicting Disagreement with Human Raters in LLM-as-a-Judge Difficulty Assessmen
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: experiments assess model difficulty ratings against human raters, with no learners.
- Grader B: R4 real-learner gate: predicts LLM-judge disagreement with human difficulty ratings; the outcome is model AUC only; EDM poster acceptance not matched
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
Technical testLearners: Simulated or AI-onlyOutcome: Model performancePreprintOpen data or materialsStudy type: Technical test (no real learners)
2026-05-10 · arXiv (back catalog) · Study (preprint)
Large language models are increasingly used in K-12 education, but existing benchmarks mainly test exam question answering rather than understanding how curriculum knowledge is structured and visually presented. We call this capability curriculum cognition.
Why this label for K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training E
- Technical test: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R4 real-learner gate: evaluates educational LLMs on curriculum benchmarks, not K-12 students.
- Grader B: R4 real-learner gate: curriculum knowledge-graph benchmark and fine-tuning corpus for LLMs; K-12 content but no learners took part
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryStudy type: Expert opinion / commentary
2026-10-19 · Frontiers in Education · Study
Artificial intelligence (AI) and learning analytics are transforming higher education by enabling personalised learning, adaptive support, and data-informed decision-making. These developments present significant opportunities for improving educational inclusion for students with learning difficulties, yet they also...
Why this label for Toward an AI- and learning-analytics-enabled inclusive support framework for stu
- Commentary: Conceptual framework paper (R3).
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
CommentaryStudy type: Expert opinion / commentary
2026-10-19 · Frontiers in Education · Study
Over the past decade, fairness research in classroom assessment has progressively repositioned students from passive objects toward partial subjects of assessment. By advocating for student active engagement in assessment processes via discourses such as student voice and rights, student agency, student assessment...
Why this label for Beyond passive rights-holders: toward student moral agency in assessment fairnes
- Commentary: Perspective article (R3).
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
CommentaryProvisional (rules only)
2026-10-15 · arXiv physics.ed-ph · Study (preprint)
This study investigates integrative thinking in undergraduate computational physics design essays (CPDEs) within an engineering design-based introductory physics laboratory. Integrative thinking refers to the interaction of science, mathematics, computation, design, and metacognitive reflection during problem solving.
Why this label for Integrative Thinking in Computational Physics Design Essays
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate (F7): Self-described conceptual, legal, policy or reflective work with no learner data.
- Preprint: not peer reviewed; capped at Level 3
- Metacognition and self-regulation: matched 'metacognitive'
- Assessment and grading: matched 'rubric'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Expert opinion / commentary
2026-10-13 · arXiv search: AI + education · Report / framework
Human-AI Interaction is commonly evaluated by what an artificial agent enables during use, yet interaction can also alter the human who acts next. We introduce Residual Human States, a trajectory-based framework for representing the potentially multidimensional human state carried forward after interaction with AI,...
Why this label for What Human-AI Interaction Leaves Behind: Residual Human States - A Trajectory-Ba
- Commentary (human override): The paper introduces a conceptual framework (Residual Human States) and illustrates it with four cases, most drawn from previously published or separate studies, rather than reporting one new graded study. Our genre rule places framework work in Commentary.
- Changed from: Level 1: Descriptive or exploratory
Human grade change 2026-10-21, approved by Matthew Rhoads, Ed.D. (approved in chat, 2026-10-21). Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-10-13 · arXiv (back catalog) · Study (preprint)
Compute cost in automated essay scoring (AES) is usually treated as a technical efficiency issue rather than a measurement design concern. Yet deployment decisions require more than reporting latency, model size, or inference expense.
Why this label for From Efficiency to Sufficiency: The Cost-Aware Measurement Design Framework for
- Commentary: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R3 genre gate: proposes a measurement design framework without an empirical evaluation.
- Grader B: R3 genre gate: conceptual cost-measurement framework for automated essay scoring with no empirical study
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryProvisional (rules only)
2026-10-12 · Educational Administration Quarterly (SAGE) · Study
This article promotes estimation thinking as both a methodological stance and a logic of theorizing aimed at enhancing theoretical precision. Estimation thinking is defined as an inferential approach that moves beyond null hypothesis significance ...
Why this label for Test Less, Estimate More: Apply Estimation Thinking to Enhance Theoretical Preci
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-10-12 · Educational Administration Quarterly (SAGE) · Study
The purpose of this article was to evaluate the extent to which research in educational leadership and management reflects the core principles of estimation thinking - a statistical thinking that prioritizes estimation over binary hypothesis testing and ...
Why this label for Current State of Estimation Thinking in Educational Leadership Research: A Revie
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-10-08 · arXiv physics.ed-ph · Study (preprint)
Students often perceive scientific experiments as clean, straightforward procedures with predictable outcomes. Students' beliefs about the nature of experimental physics typically remain inconsistent through the initial stages of undergraduate education.
Why this label for Measuring the Speed of Light with a Time-of-Flight Setup: An Authentic Experimen
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-10-06 · arXiv physics.ed-ph · Study (preprint)
The moving-coil meter provides a simple context in which to model deterministic electromechanical dynamics, thermal fluctuations, and electronic readout. The movement is a linear electromechanical system in which electrical resistance and intrinsic mechanical damping act as two independent dissipative channels, each...
Why this label for Thermal fluctuations and electronic readout using a moving-coil movement
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-09-30 · arXiv physics.ed-ph · Study (preprint)
Undergraduate research experiences provide students with an opportunity to apply knowledge learned in the classroom to authentic scientific problems. In this work, we describe the development of a special-topics course in Plasma Physics and Engineering (PPE) at a small liberal arts college using a Course-Based...
Why this label for Teaching a plasma physics and engineering class at a liberal arts college using
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
- Preprint: not peer reviewed; capped at Level 3
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Not a study
2026-09-29 · EdTech Magazine K-12 · News
Fueled by factors such as chronic absenteeism and pandemic learning challenges, students’ reading scores have been trending downward for years. A biennial National Assessment of Education Progress test found that approximately a third of eighth grade students lack basic reading skills, the largest percentage on record.
Why this label for How K - 12 Districts Are Boosting Student Literacy With Technology
- Commentary: news, commentary or policy with no study
- Reading and writing: matched 'Literacy'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-09-28 · arXiv physics.ed-ph · Study (preprint)
We present a traceable instructional module that maps documented learner difficulties in quantum superposition and measurement to explicit learning goals, classroom activities, and assessment evidence. Using the general two-level Hamiltonian as the organizing physics framework, the module implements this mapping...
Why this label for From Two-Level Hamiltonians to Quantum Superposition and Measurement: A Traceabl
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
- Preprint: not peer reviewed; capped at Level 3
- Assessment and grading: matched 'grading'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Systematic review / meta-analysis
2026-09-24 · arXiv (back catalog) · Study (preprint)
Generative artificial intelligence has moved from a disruptive novelty to a recurring part of software-development and computing-education workflows, while software agents are beginning to act across repositories, command lines, browsers, tests, and other tools. The educational problem is no longer whether students...
Why this label for Judgment-Centred Software Engineering Education: A Post-Hype Review and Framewor
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 integrative narrative review and proposed education framework, with no systematic study synthesis described.
- Grader B: R3 genre gate: a structured integrative (narrative) review plus a conceptual framework, with no stated systematic search criteria or controlled-study synthesis.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-09-21 · arXiv (back catalog) · Study (preprint)
Third-party evaluations for frontier AI have mostly tested models through external interfaces before deployment. But the risks from frontier AI models depend on how their developers use and govern them internally.
Why this label for Embedded Assessments for Frontier AI
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 position paper argues for embedded assessments and recommends their design without reporting an empirical study.
- Grader B: R3 genre gate: a policy and position paper recommending embedded assessments for frontier AI.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryProvisional (rules only)
2026-09-21 · Educational Administration Quarterly (SAGE) · Study
Estimation thinking is increasingly recommended across disciplines, including educational leadership, as an alternative to traditional significance testing. Unlike the binary logic ofp-values and arbitrary cutoffs, such asp< .05, estimation emphasizes ...
Why this label for Getting Estimation Thinking into Educational Leadership: Challenges and Next Ste
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Not a study
2026-09-18 · eSchool News · News
In 2023, Catherine Hartmann, a professor at the University of Wyoming, gave students in a history of meditation course a deeply personal assignment: try a contemplative practice and write a reflective analysis on the experience.
Why this label for The oral exam is back, and AI may be the reason
- Commentary: news, commentary or policy with no study
- Assessment and grading: matched 'exam'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-09-16 · arXiv (back catalog) · Study (preprint)
Digital competence frameworks define the knowledge, skills, attitudes and values required for participation in digital society. The European Digital Competence Framework for Citizens (DigComp) and UNESCO digital literacy and artificial intelligence competency frameworks provide reference structures for policy,...
Why this label for From Digital Competence to Demonstrated Digital Capability: Positioning the Inte
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 explicitly conceptual comparison of digital-competence frameworks without participant outcomes.
- Grader B: R3 genre gate: a self-described conceptual paper comparing competency frameworks.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryProvisional (rules only)
2026-09-15 · Educational Administration Quarterly (SAGE) · Study
For several decades, scholars across statistics, psychology, and research methodology have questioned the widespread reliance on null hypothesis significance testing (NHST) and dichotomous decision making based on fixed significance thresholds. ...
Why this label for From Ritual to Reasoning: Estimation Thinking as an Inferential Framework for Ed
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-08-27 · arXiv (back catalog) · Study (preprint)
Companies that deploy AI agents and make them available to others are creating the sociotechnical circumstances under which this technology integrates into existing social and economic structures. AI-deploying companies are institutional actors that actively shape society's capacity to withstand and govern the...
Why this label for Assessing Company Contributions to Societal Resilience: Extending the Societal C
- Commentary: Consensus level confirmed against the methodology.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: the structured assessment of company documents is a descriptive application of a framework.
- Grader B: R3 genre gate: adapts a conceptual framework and applies it to public company documents; no participants or learning outcome
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: TeachersOutcome: Perception or attitudePreprintReport (grey literature)Study type: Expert opinion / commentary
2026-08-19 · arXiv (back catalog) · Study (preprint)
On 28 July 2026, the SIGCSE Virtual 2026 Working Group on computing assessment and generative AI held an open online workshop attended by 73 computing educators. The first hour ran seven strategy rooms, one per approach to adapting (or deliberately preserving) computing assessment in the AI era: open-ended and...
Why this label for AI Can Do Your Homework. Now What? Report from an Online Workshop on Computing A
- Commentary: Workshop report, not a study.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 descriptive ceiling: workshop report records 73 educators' accounts without a controlled learning study.
- Grader B: R3 genre gate: a workshop report recording what 73 computing educators discussed; it has no study design and no measured outcome.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Motivation and engagement: matched 'student motivation'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-08-05 · arXiv (back catalog) · Study (preprint)
As generative AI permeates computing instruction, the emergent design challenge is to configure each tool's pedagogical role, authority, and accountability for the instructional work it performs. We argue for instructional governance by design: governance should be encoded in a teaching tool's interaction model,...
Why this label for Instructional Governance by Design: A Framework for AI in Computing Education
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 genre gate: the paper explicitly develops a position and governance framework.
- Grader B: R3 genre gate: a position paper and framework illustrated with case descriptions, with no learner outcome data.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-07-27 · arXiv (back catalog) · Study (preprint)
Spaced repetition systems (SRS) have demonstrated robust effects on long-term retention, yet existing tools reduce the flashcard interaction to a single binary gesture: flip and self-rate. This impoverished interaction model fails to leverage decades of cognitive science evidence on retrieval practice, and requires...
Why this label for Memdora: Designing Cognitively-Grounded Flashcard Interactions for AI-Powered Sp
- Commentary: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R3 genre gate: the paper presents a system and design rationale but reports no evaluation of learners.
- Grader B: R3 genre gate: system design and rationale paper for a flashcard app, with no evaluation with learners reported
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
- Retrieval practice: matched 'Flashcard'
- Spacing and interleaving: matched 'Spaced Repetition'
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-07-22 · arXiv (back catalog) · Study (preprint)
Generative AI is changing a basic premise of educational assessment: that submitted work can reliably evidence the human capacities a credential claims to certify. The challenge is not simply whether students use AI, but what remains inferable about learning when some cognitive work has been delegated to a system.
Why this label for What Does the Credential Still Certify? Cognitive Stewardship for AI-Mediated Ed
- Commentary: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R3 genre gate: the paper develops an assessment framework and audits university policies rather than studying learners.
- Grader B: R3 genre gate: conceptual framework plus an LLM-coded audit of 30 university policy documents; no learners, so no learner-level evidence
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-07-18 · arXiv (back catalog) · Study (preprint)
Academic peer review is under mounting strain: NeurIPS 2025 received 21,575 submissions, ICLR 2025 received 11,603, and ICML 2025 received 12,107. This volume has outpaced the supply of qualified reviewers, and large language models (LLMs) are already filling the gap, largely undisclosed.
Why this label for Local AI pre-screening for human triple-blind peer review in health sciences
- Commentary: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R3 genre gate: proposes a peer-review framework without evaluating it.
- Grader B: R3 genre gate: proposes a peer-review pre-screening framework with no study; it only cites someone else's benchmark.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-07-09 · arXiv (back catalog) · Study (preprint)
Teaching computer science at universities is often structured rather classically and theory oriented. The former refers to "transmission"-style lectures accompanied by exercises which are submitted and graded manually, providing delayed feedback (if any).
Why this label for DOMtutor: Automated Autograding for Logic in Computer Science
- Commentary: Both graders agree; rules version missed it. Consensus level confirmed.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R3 genre gate: proposes an autograding approach without reporting an evaluation.
- Grader B: R3 genre gate: proposes using DOMjudge autograders for theory courses, with no evaluation reported.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Higher-ed studentsOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-07-08 · arXiv (back catalog) · Study (preprint)
We present documentation of a classroom assignment that teaches runtime monitoring through a creative embedded systems build: an interactive Pachinko game. The assignment centers on a dual-core ESP32 workflow in which students write RTLola specifications for monitors, compile these monitors to C, and deploy them...
Why this label for Experiential Learning of Runtime Monitoring Using Pachinko
- Commentary: Assignment documentation with no study.
- Review status: Confirmed against the published method (AI-assisted review)
- Grader A: R5 design ceiling: documents a classroom assignment without reporting a measured learning outcome.
- Grader B: R3 genre gate: documentation of a classroom assignment with no data or evaluation of student outcomes.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review) Method
CommentaryLearners: Higher-ed studentsOutcome: Behavior or engagementPeer-reviewed (journal reference listed)Report (grey literature)Study type: Expert opinion / commentary
2026-07-06 · arXiv (back catalog) · Study
This paper examines the psychological determinants that shape academically honest and dishonest uses of generative artificial intelligence (GenAI) in higher education. Rather than treating academic misconduct as a purely technological problem, the study conceptualizes academic integrity as a psychologically mediated...
Why this label for Psychological Determinants of Academic Integrity in the Use of Generative AI in
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 genre gate: the paper is a narrative review and conceptual synthesis.
- Grader B: R3 genre gate: self-described focused narrative review and conceptual synthesis of a purposive corpus of 16 sources, not a systematic review
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryStudy type: Not a study
2026-05-27 · eSchool News · News
The overreliance on AI is a widely discussed topic for teachers, administrators, and families alike. The last thing we want is for technology to stifle the creativity, expertise, and human connection that educators bring to the classroom or hinder our students’ ability to think critically.
Why this label for How AI helps teachers spend less time on assessments and more time on impactful
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Expert opinion / commentary
2026-05-06 · Carl Hendrick (learning science) · Commentary
New evidence on the limits of retrieval practice, phone bans that don't move test scores, ChatGPT as cognitive crutch, and why early vocabulary growth tells us surprisingly little about later reading
Why this label for The Monthly Dispatch - What’s New in Learning Science - May 2026
- Commentary: news, commentary or policy with no study
- Retrieval practice: matched 'retrieval practice'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Not a study
2026-04-28 · eSchool News · News
AI is rapidly reshaping education, but not always in ways that support learning. A growing number of AI tools promise to “help” students by doing assignments, writing papers, solving problem sets, or even completing exams automatically.
Why this label for When AI does the work, who does the learning?
- Commentary: news, commentary or policy with no study
- Assessment and grading: matched 'exams'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-04-26 · arXiv (back catalog) · Study (preprint)
Generative Artificial Intelligence (GenAI) presents a governance challenge for STEM assessment. Unrestricted access can enable task outsourcing that undermines the validity of traditional assessments, while blanket prohibitions are difficult to enforce, may drive use underground, and do little to prepare students for...
Why this label for Positioning Generative Artificial Intelligence in STEM Assessment: When to Requi
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 genre gate: the proposed STEM assessment framework is illustrated with examples, not tested with learners.
- Grader B: R3 genre gate: conceptual ECD-based governance framework for GenAI in STEM assessment, illustrated with physics examples, with no learner data.
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryLearners: Not reportedOutcome: Not reportedPreprintStudy type: Expert opinion / commentary
2026-04-14 · arXiv (back catalog) · Study (preprint)
We identify a structured confound in Reinforcement Learning from Human Feedback (RLHF). Pairwise preference labels are intended to reflect the compared outputs, but they may also reflect the rater's state during annotation.
Why this label for Rater State Bias in RLHF Preference Data: An Audit Framework
- Commentary: Two independent AI graders gave the same level.
- Review status: Confirmed against the published method (AI-assisted review): both graders agreed
- Grader A: R3 genre gate: develops a hypothesis and audit framework, with no completed empirical study reported.
- Grader B: R3 genre gate: conceptual hypothesis and proposed audit framework with a pilot plan and no collected data
- Rule version: METHODOLOGY v0.3, pipeline v1, rules rules-v0.4
Graded blind by two AI graders against our published method. Confirmed against the published method (AI-assisted review): both graders agreed Method
CommentaryStudy type: Not a study
2026-04-10 · eSchool News · News
Most project-based learning workshops are built around three domains: design, assessment, and implementation.
Why this label for A new need-to-know for the AI classroom
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryStudy type: Not a study
2026-03-12 · eSchool News · News
Artificial intelligence is no longer approaching the classroom--it is already embedded in it. Students are using generative tools to brainstorm, summarize, translate, draft, and revise.
Why this label for The AI-resistant classroom is a myth: Designing assessments that assume AI is pr
- Commentary: news, commentary or policy with no study
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-02-25 · ERIC (peer reviewed) · Study
Who is more creative: highor low-socioeconomic status (SES) individuals? This question is the focus of intense debates within both the scientific community and society at large, since creativity has been linked to innovation, productivity, and wealth generation.
Why this label for Reading Ability Conflates SES Creativity Gaps
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate: No empirical learner data described; conceptual, framework, policy or argument paper.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
CommentaryProvisional (rules only)
2026-02-12 · ERIC (peer reviewed) · Study
The rapid proliferation of telemedicine, accelerated by global demands for decentralized healthcare, necessitates a paradigm shift in medical education. Traditional clinical training models, centered on in-person encounters, are increasingly misaligned with the realities of modern practice.
Why this label for Transforming Medical Training through Telemedicine Simulation and Remote Consult
- Commentary: provisional rules-only grade, pending grader consensus and review: R3 genre gate (F7): Self-described conceptual, legal, policy or reflective work with no learner data.
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
No study citedStudy type: Not a study
2026-10-05 · arXiv search: AI + education · Study (preprint)
We present an AI-based web-oriented tool designed to support formative feedback for oral presentation slides in higher education: AISSA. It allows students to upload their slides before presentation and automatically receive rubric-based quantitative scores and qualitative feedback generated by large language models...
Why this label for AISSA Demo: AI-based Student Slides Analysis Tool for Automated Grading and Feed
- No study cited: Tool demo with no evaluation reported (R1).
- Review status: Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified
Graded by one AI grader from the abstract, on Matt's delegation; not yet human-verified. Method
Not yet classified
Listed for the January 2027 issue · Computers & Education (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Computers & Education, Volume 256 by Chengchen Li, Ye Han, Enhao Feng. Issue: January 2027. Abstract on the publisher page.
Why this label for Engagement with GenAI-Assisted L2 writing: Conceptualization, measurement, and a
- Not yet classified: publisher listing with no abstract yet
- Reading and writing: matched 'L2 writing'
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
Not yet classified
Listed for the December 2026 issue · Teaching and Teacher Education (Elsevier) · Study
Publisher listing (abstract not yet available): Journal article in Teaching and Teacher Education, Volume 184 by Taina Gabriel, Jochanan Veerbeek, Moritz Börnert-Ringleb. Issue: December 2026. Abstract on the publisher page.
Why this label for Comparing the preference of diagnostic information from variations of dynamic te
- Not yet classified: publisher listing with no abstract yet
Provisional label from fixed rules applied to the collected text; research items are re-graded by our review pipeline. Method
