Working paper
Estimating student growth on psychological and social-emotional constructs: A comparison of multiple scoring approaches
2020
Description
A huge portion of what we know about how humans develop, learn, behave, and interact is based on survey data. Researchers use longitudinal growth modeling to understand the development of students on psychological and social-emotional learning constructs across elementary and middle school. In these designs, students are typically administered a consistent set of self-report survey items across multiple school years, and growth is measured either based on sum scores or scale scores produced based on item response theory (IRT) methods. While there is great deal of guidance on scaling and linking IRT-based large-scale educational assessment to facilitate the estimation of examinee growth, little of this expertise is brought to bear in the scaling of psychological and social-emotional constructs. Through a series of simulation and empirical studies, we produce scores in a single-cohort repeated measure design using sum scores as well as multiple IRT approaches and compare the recovery of growth estimates from longitudinal growth models using each set of scores. Results indicate that using scores from multidimensional IRT approaches that account for latent variable covariances over time in growth models leads to better recovery of growth parameters relative to models using sum scores and other IRT approaches.
The final version of this paper is available at:
Kuhfeld, M., & Soland, J. (2020). Avoiding bias from sum scores in growth estimates: An examination of IRT-based approaches to scoring longitudinal survey responses. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000367
See MoreTopics: Growth modeling, Social-emotional learning
Associated Research
Related Topics
Typical learning for whom? Guidelines for selecting benchmarks to calculate months of learning
To describe the impact of the COVID-19 pandemic on students, researchers have translated test scores into months of learning to claim how many months/years students are behind in school. Despite its perceived accessibility, there are major downsides to this translation. To inform future uses by researchers and media, we discuss in this brief how to calculate this metric as well as its trade-offs.
Topics: COVID-19 & schools, Equity, Growth, Growth modeling, Seasonal learning patterns & summer loss
The purpose of this technical appendix is to share more detailed results and to describe more fully the sample and methods used in the research included in the brief, Learning during COVID-19: An update on student achievement and growth at the start of the 2021-22 school year. We investigated two research questions:
- How does student achievement in fall 2021 compare to pre-pandemic levels (namely fall 2019)?
- How did academic gains between fall 2019 and fall 2021 compare to normative growth expectations?
Topics: COVID-19 & schools, Equity, Growth modeling
To what extent has the COVID-19 pandemic affected student achievement and growth in reading and math, and which students have been most affected? Using data from 6 million students in grades 3-8 who took MAP Growth assessments in reading and math, this brief examines how gains across the pandemic (fall 2019 to fall 2021) and student achievement in fall 2021 compare to pre-pandemic trends. This research provides insight to leaders working to support recovery.
Topics: COVID-19 & schools, Equity, Growth modeling
This study examined the stability of social-emotional learning (SEL) skills and the extent to which students’ initial level in SEL skills in 6th grade and growth in SEL skills from 6th to 8th grade are related to students’ successful transition to secondary school. Findings suggest that understanding how a student develops social-emotionally can improve identification of students not on track to succeed in high school.
Topics: Social-emotional learning, High school, Middle school
The forgotten 20 percent: Achievement and growth in rural schools across the nation
Using achievement data from fall and spring of grades K-8 for 840,000 students in 8,800 public schools, this study provides novel evidence on how achievement and growth differ between rural and nonrural schools. Rural students start kindergarten slightly ahead of nonrural students but fall behind by middle school. The divergence is driven by larger summer losses for rural students. In both rural and nonrural schools, Black–White achievement gaps widen during the school year.
Topics: Equity, Growth modeling, Seasonal learning patterns & summer loss
Predictors and consequences of school mobility in middle childhood
This study examined family factors associated with school mobility and if either a move during the previous year or cumulative moves across elementary school were related to child functioning. School mobility during elementary school did not appear to be a pervasive risk although the authors were unable to study very high rates of school mobility because of very small sample sizes.
Topics: Middle school, Social-emotional learning
Understanding differential growth during school years and summers for students in special education
New research examining academic achievement and growth of students in special education and their peers who were never in special education during each school year and summer in grades K-4 shows that students with disabilities grow as much or more academically during the school year than their peers without disabilities during some years, but that steeper summer learning losses for students with disabilities contribute to widening disparities.
Topics: Equity, Growth modeling, Seasonal learning patterns & summer loss
