Assessment Literacy involves understanding how assessments are made, what type of assessments answer what questions, and how the data from assessments can be used to help teachers, students, parents, and other stakeholders make decisions about teaching and learning. Assessment designers strive to create assessments that show a high degree of fidelity to the following five traits:
1. Content Validity
2. Reliability
3. Fairness
4. Student Engagement and Motivation
5. Consequential Relevance
In this third blog post in our Five Characteristics of Quality Educational Assessments series, we discuss the third of five characteristics of quality educational assessments – Fairness.
Since the seventies there have been great strides in educational assessment development practices to ensure an assessment experience that is as fair as possible to the largest possible population of students. Why fairness in testing is important is self-evident: every student deserves an equal opportunity to demonstrate what he or she understands, knows, and can do. It is an ethical imperative, and in the case of summative assessment providers, a legal imperative, that assessments are culturally inclusive, accommodating to students with special physical or cognitive needs, and accessible to students for whom English is not a first language.
The issue of fairness in testing can be subdivided into three distinct categories: cultural sensitivity, bias, and accessibility to special populations, such as English Language Learners and special education students.
An educational assessment that demonstrates cultural sensitivity respects diversity, strives to fairly represent gender in non-stereotypical ways, and contains content that a student from anywhere in the country from any socio-economic strata would have access to understanding.
Cultural sensitivity is more about including content, scenarios, and contexts that are relevant to people from all sorts of different backgrounds and perspectives than it is about policing content and bowdlerizing, or sanitizing, it. Cultural sensitivity is a qualitative trait that is ensured through rigorous reviews during large-scale assessment development, often including the use of rubrics and checklists.
The concept of assessment bias is when a group of students has an unfair advantage on an item or group of items that is statistically observable. Unfair advantages do not include things like better preparation, higher aptitude, or ease with test taking. Unfair advantages can come from many different directions. They can occur when an item’s content matter privileges a certain kind of background knowledge or experience.
Bias is not a trait that classroom or informal assessment tracks since it is revealed through analyzing the response patterns of various testing populations and looking for statistically meaningful deviation from the general spread of response patterns. Ferreting out bias is another service provided by psychometricians when they are parsing and making sense of student response data.
Accessibility for special education students and English Language Learners is a fundamentally different issue than sensitivity and bias, but it relates to the same organizing concept: fairness in assessment. Accessibility for state summative assessments is legislated because it directly addresses the rights of individuals based on the legal premise that every American student has the right to a quality public education.
Accessibility in educational assessment translates into the tools, assists, devices, and accommodations that are allowed so that students can either take the same test as their peers, or have an equivalent assessment experience. At a classroom level, teachers are acutely aware when issues of accessibility due to linguistic, physical, cognitive, or emotional capabilities arise. In a school ecosystem, there are teams of support providers, including classroom and special education teachers, tutors, school psychologists, case workers and social services personnel focused on ensuring that students have equal access to the same educational opportunities as their peers.
Our fourth and final post on characteristics of quality educational assessments will cover the final two traits – student engagement and motivation, and consequential relevance. In the meantime, please feel free to share your thoughts on what qualities a good educational assessment should have by dropping a comment below.
