Journal article
Response time as an indicator of test taker speed: assumptions meet reality
2015
Measurement: Interdisciplinary Research and Perspectives, 13(3–4), 186–188.
Abstract
The growing presence of computer-based testing has brought with it the capability to routinely capture the time that test takers spend on individual test items. This, in turn, has led to an increased interest in potential applications of response time in measuring intellectual ability and achievement. Goldhammer (this issue) provides a very useful overview of much of the research in this area, and he provides a thoughtful analysis of the speed-ability trade-off and its impact on measurement. This author does not claim to understand why test takers tend to increase their time efficiency during MAP test events, although he notes that, having reanalyzed some of his own data, he had observed this phenomenon in data from 2 very different testing programs. How commonly this effect is present in test data is, for the time being, unclear. Whatever its nature, however, this increasing efficiency effect appears to (at least sometimes) represent an additional systematic factor influencing response time. And whenever this effect is present to a material degree, it threatens the validity of models that presume that item response time is systematically influenced only by item time intensity and the test taker’s ability and working speed. It confounds attempts to exploit a speed-ability trade-off, which is central to many of the models that Goldhammer discussed. If exhibited speed can change without a commensurate change in exhibited ability, the usefulness of response time in estimating a test taker’s ability is undermined. Hence, whenever an increased efficiency effect is present, it represents an inconvenient reality for response-time-based psychometric models.
See MoreThis article was published outside of NWEA. The full text can be found at the link above.
Related Topics
MAP Reading Fluency with Coach Evidence Base
This document provides an overview of the research underlying MAP Reading Fluency with Coach’s AI-powered intelligent reading tutor and the research on key elements of early literacy instruction. It describes the components of the MAP Reading Fluency with Coach pedagogy and the research base supporting each component.
Products: MAP Reading Fluency
Topics: Early learning, Empowering educators, Innovations in reporting & assessment, Reading & language arts
Predicting Amira Reading Mastery Based on NWEA MAP Reading Fluency Benchmark Assessment Scores
This document presents results from a linking study conducted by NWEA in May 2024 to statistically connect the grades 1–5 English Amira Reading Mastery (ARM) scores with the Scaled-Words-Correct-Per-Minute (SWCPM) scores from the MAP Reading Fluency benchmark assessment taken during Fall and Winter 2023–2024.
Products: MAP Reading Fluency
Topics: Computer adaptive testing, Early learning, Measurement & scaling, Reading & language arts
Exploring the educational impacts of COVID-19
This visualization was developed to provide state-level insights into how students performed on MAP Growth in the 2020–2021 school year. Assessments are one indicator, among many, of the student impact from COVID-19. Our goal with this tool is to create visible data that informs academic recovery efforts that will be necessary in the 2022 school year and beyond.
Topics: COVID-19 & schools, Innovations in reporting & assessment
Executive Summary: Content proximity spring 2022 pilot study
This executive summary outlines results from the Content Proximity spring 2022 pilot study, including information on the validity, reliability, and test score comparability of MAP Growth assessments that leverage this new item-selection algorithm.
Products: MAP Growth
Topics: Computer adaptive testing, Innovations in reporting & assessment, Test design
Content Proximity Spring 2022 Pilot Study Research Report
The purpose of this research report is to provide detailed information about updates to the MAP Growth item-selection algorithm. This brief includes results from the Content Proximity pilot study, including information on the validity, reliability, and test score comparability of MAP Growth assessments that leverage this new item-selection algorithm.
Products: MAP Growth
Topics: Computer adaptive testing, Innovations in reporting & assessment, Test design
Achievement and Growth Norms for Course-Specific MAP Growth Tests
This report documents the procedure used to produce the achievement and growth user norms for a series of the course-specific MAP® Growth™ subject tests, including Algebra 1, Geometry, Algebra 2, Integrated Math I, Integrated Math II, Integrated Math III, and Biology/Life Science. Among these tests, Integrated Math I, Integrated Math II, Integrated Math III, and Biology/Life Science were the first time to have their norms available. The remaining tests, i.e., Algebra 1, Geometry, and Algebra 2, had their norms updated including receiving more between-term growth norms by using more recent test events. Procedure for norm sample selection and a model-based approach using the multivariate true score model (Thum & He, 2019) that factors out known imprecision of scores to generate the norms are also provided in detail, along with the snapshots of the achievement and growth norms for each test.
Products: MAP Growth
Topics: Measurement & scaling
Longitudinal models of reading and mathematics achievement in deaf and hard of hearing students
New research using longitudinal data provides evidence that deaf and hard of hearing (DHH) students continue to build skills in math and reading throughout grades 2 to 8, challenging assumptions that DHH students’ skills plataeu in elementary grades.
Topics: Equity, Measurement & scaling
