Journal article
Do response styles affect estimates of growth on social-emotional constructs? Evidence from four years of longitudinal survey scores
July 2020
Multivariate Behavioral Research
Abstract
Survey respondents employ different response styles when they use the categories of the Likert scale differently despite having the same true score on the construct of interest. For example, respondents may be more likely to use the extremes of the response scale independent of their true score. Research already shows that differing response styles can create a construct-irrelevant source of bias that distorts fundamental inferences made based on survey data. While some initial studies examine the effect of response styles on survey scores in longitudinal analyses, the issue of how response styles affect estimates of growth is underexamined. In this study, we conducted empirical and simulation analyses in which we scored surveys using item response theory (IRT) models that do and do not account for response styles, and then used those different scores in growth models and compared results. Generally, we found that response styles can affect estimates of growth parameters including the slope, but that the effects vary by psychological construct, response style, and IRT model used.
See MoreThis article was published outside of NWEA. The full text can be found at the link above.
Related Topics
Predicting Amira Reading Mastery Based on NWEA MAP Reading Fluency Benchmark Assessment Scores
This document presents results from a linking study conducted by NWEA in May 2024 to statistically connect the grades 1–5 English Amira Reading Mastery (ARM) scores with the Scaled-Words-Correct-Per-Minute (SWCPM) scores from the MAP Reading Fluency benchmark assessment taken during Fall and Winter 2023–2024.
Products: MAP Reading Fluency
Topics: Computer adaptive testing, Early learning, Measurement & scaling, Reading & language arts
Typical learning for whom? Guidelines for selecting benchmarks to calculate months of learning
To describe the impact of the COVID-19 pandemic on students, researchers have translated test scores into months of learning to claim how many months/years students are behind in school. Despite its perceived accessibility, there are major downsides to this translation. To inform future uses by researchers and media, we discuss in this brief how to calculate this metric as well as its trade-offs.
Topics: COVID-19 & schools, Equity, Growth, Growth modeling, Seasonal learning patterns & summer loss
Achievement and Growth Norms for Course-Specific MAP Growth Tests
This report documents the procedure used to produce the achievement and growth user norms for a series of the course-specific MAP® Growth™ subject tests, including Algebra 1, Geometry, Algebra 2, Integrated Math I, Integrated Math II, Integrated Math III, and Biology/Life Science. Among these tests, Integrated Math I, Integrated Math II, Integrated Math III, and Biology/Life Science were the first time to have their norms available. The remaining tests, i.e., Algebra 1, Geometry, and Algebra 2, had their norms updated including receiving more between-term growth norms by using more recent test events. Procedure for norm sample selection and a model-based approach using the multivariate true score model (Thum & He, 2019) that factors out known imprecision of scores to generate the norms are also provided in detail, along with the snapshots of the achievement and growth norms for each test.
Products: MAP Growth
Topics: Measurement & scaling
Longitudinal models of reading and mathematics achievement in deaf and hard of hearing students
New research using longitudinal data provides evidence that deaf and hard of hearing (DHH) students continue to build skills in math and reading throughout grades 2 to 8, challenging assumptions that DHH students’ skills plataeu in elementary grades.
Topics: Equity, Measurement & scaling
Achievement and growth norms for English MAP Reading Fluency Foundational Skills
This report documents the norming study procedure used to produce the achievement and growth user norms for English MAP Reading Fluency Foundational Skills.
Products: MAP Reading Fluency
Topics: Measurement & scaling
The MAP Growth theory of action describes key features of MAP Growth and its position in a comprehensive assessment system.
Products: MAP Growth
Topics: Equity, Measurement & scaling, Test design
Bayesian uncertainty estimation for Gaussian graphical models and centrality indices
This study compares estimation of symptom networks with Bayesian GLASSO- and Horseshoe priors to estimation using the frequentist GLASSO using extensive simulations.
Topics: Measurement & scaling
