Journal article
Predicting the difficulty and response time of multiple choice questions using transfer learning
2020
15th Workshop on Innovative Use of NLP for Building Educational Applications
Abstract
This paper reports on whether transfer learning can improve the prediction of the difficulty and response time parameters for ā 18,000 multiple-choice questions from a high-stakes medical exam. The type of the signal that bestpredicts difficulty and response time is also explored, both in terms of representation abstraction and item component used as input (e.g., whole item, answer options only, etc.). Theresults indicate that, for our sample, transferlearning can improve the prediction of item difficulty when response time is used as an auxiliary task but not the other way around. In addition, difficulty was best predicted using signal from the item stem (the description of theclinical case), while all parts of the item were important for predicting the response time.
See MoreThis article was published outside of NWEA. The full text can be found at the link above.
Related Topics
MAP Reading Fluency with Coach Evidence Base
This document provides an overview of the research underlying MAP Reading Fluency with Coachās AI-powered intelligent reading tutor and the research on key elements of early literacy instruction. It describes the components of the MAP Reading Fluency with Coach pedagogy and the research base supporting each component.
Products: MAP Reading Fluency
Topics: Early learning, Empowering educators, Innovations in reporting & assessment, Reading & language arts
Exploring the educational impacts of COVID-19
This visualization was developed to provide state-level insights into how students performed on MAP Growth in the 2020ā2021 school year. Assessments are one indicator, among many, of the student impact from COVID-19. Our goal with this tool is to create visible data that informs academic recovery efforts that will be necessary in the 2022 school year and beyond.
Topics: COVID-19 & schools, Innovations in reporting & assessment
Executive Summary: Content proximity spring 2022 pilot study
This executive summary outlines results from the Content Proximity spring 2022 pilot study, including information on the validity, reliability, and test score comparability of MAP Growth assessments that leverage this new item-selection algorithm.
Products: MAP Growth
Topics: Computer adaptive testing, Innovations in reporting & assessment, Test design
Content Proximity Spring 2022 Pilot Study Research Report
The purpose of this research report is to provide detailed information about updates to the MAP Growth item-selection algorithm. This brief includes results from the Content Proximity pilot study, including information on the validity, reliability, and test score comparability of MAP Growth assessments that leverage this new item-selection algorithm.
Products: MAP Growth
Topics: Computer adaptive testing, Innovations in reporting & assessment, Test design
This study compared the test taking disengagement of students taking a remotely administered an adaptive interim assessment in spring 2020 with their disengagement on the assessment administered in-school during fall 2019.
Topics: Equity, Innovations in reporting & assessment, School & test engagement
This study evaluates the effects of asking items throughout the passage (i.e., embedding items) to achieve a more precise measure of reading comprehension by removing barriers for students to demonstrate their understanding. Results showed a significant impact of embedding comprehension items within reading passages on the measurement of student achievement in comparison to answering items at the end of the passage.
Topics: Equity, Innovations in reporting & assessment, Reading & language arts
Variation in respondent speed and its implications: Evidence from an adaptive testing scenario
The more frequent collection of response time data is leading to an increased need for an understanding of how such data can be included in measurement models. Models for response time have been advanced, but relatively limited large-scale empirical investigations have been conducted. We take advantage of a large data set from the adaptive NWEA MAP Growth Reading Assessment to shed light on emergent features of response time behavior.
Topics: School & test engagement, Innovations in reporting & assessment
