As my colleagues have discussed in earlier Teach. Learn. Grow. posts, there are common misconceptions holding state summative assessments back from innovating at scale. I’d like to look at the use of computer adaptive assessment and obstacles to using it for state summative tests. These kinds of tests are widely respected, yet many states still don’t have a computer adaptive summative assessment.
About computer adaptive assessment
Computer adaptive tests aren’t new and have been around since the 1980s, when the first adaptive assessment, the Armed Services Vocational Aptitude Battery (ASVAB), came into use.
Computer adaptive tests tailor the difficulty of test items to student performance as they take the assessment. Questions generally get easier if a student is struggling and harder if a student is excelling.
There are different types of computer adaptive assessment, too. Computer adaptive assessment used for federal accountability can be configured to adapt on and off grade level, providing grade-level performance information by constraining the amount of off-grade level adapting. Federal policy only allows state tests to adapt one grade above the tested level and one grade below, but I don’t see that as a constraint because computer adaptive assessments can adapt their difficulty levels within a grade level, asking deeper and more complex questions as is appropriate.
Due to a large item bank with items covering the whole ability continuum, adaptive tests more accurately measure student ability than fixed-form assessments, in which all students see the same test items. They also tend to be quicker to administer, requiring fewer questions to measure student achievement. Since the questions are tailored toward student levels, they are generally considered more engaging for students.
The resistance to computer adaptive state tests
Obstacles holding states back from shifting to computer adaptive assessment generally include misguided fears about the degree to which they provide comparable results between students, since kids aren’t seeing the same questions. The test blueprint ensures the items selected by the adaptive engine sufficiently cover the reporting categories for all students; however, fixed-form assessments normally also have multiple test forms with different questions, to address concerns of cheating.
Computer adaptive assessment does require a large item bank to provide students with the right item based on their previous responses. It is costly for states to build up an item bank for computer adaptive summative tests due to the amount of time and number of reviews it takes to develop a high-quality item, but there are ways to economize. States can pool resources or have educators in a state review and approve test items from another state’s test, rather than creating entirely new ones. There is also potential, with more research, for automated item generation to help create items, with teachers in the state reviewing all items.
A worthwhile change
If a state is looking to improve their state assessment system, they should consider taking a close look at computer adaptive assessment, which has advantages over traditional, fixed-form assessments, such as fewer items, which leads to shorter tests, and more precise information on student knowledge. Many of the perceived barriers to using computer adaptive assessments can easily be addressed if those barriers are stopping states from moving to computer adaptive assessments.
What are your ideas on how we can use computer adaptive assessments to improve statewide assessments? Let us know. We’re @NWEAPolicy on Twitter.
