If all students are going to be college and career ready, many have a lot of catching up to do. And one question NWEA researchers are frequently asked is “What is a reasonable growth target for schools serving students who have fallen behind?” Recently, we had a guest speaker in our offices who offered his opinion on this question, stating that students who are behind may require one-and-one half years of growth each year. I’ve heard that particular target bandied about frequently, and know that a number of schools even set that target for their students.
Let’s start by defining what 1.5 years of growth actually is. The most common definition is that one year of growth is the equivalent to the student’s growth norm (which is based on their starting scale score and grade in school). Thus, if the fall-to-spring growth norm for a student is 8, then 8 points would constitute a year of growth. By extension, 1.5 years of growth would equate to 1.5 times the growth norm. That would be 12 points if the growth norm were 8. We believe it is more appropriate to use the straighter definition, so we will reference 1.5 years of growth henceforth as 1.5 times the growth norm, since that’s what this metric actually represents.
It’s true that students who are behind academically are not going match their peers if they make average growth, so above-average growth is necessary for them to catch up. The fact that something is necessary, however, doesn’t always make it reasonable. For example, assume Rex is a bowler who aspires to the Professional Bowlers Association Tour. He averages 150 pins a game. He needs to improve his average to 220 pins a game to make the tour, or improve by 70. The 70 pin improvement is necessary for him to make the tour, but it’s not reasonable to hold Rex accountable for that improvement, because the evidence shows that very few experienced bowlers improve by that amount. There’s nothing wrong for this to be Rex’s aspirational goal, it’s good to shoot high, and if Rex gets great coaching and commitment, perhaps he may be one of the few that makes it, but let’s not label Rex a failure if he doesn’t.
That’s why it’s important to distinguish between a goal that’s an aspiration and a goal that’s realistic. Is it an admirable aspiration for students behind grade level to make 1.5 years of growth each year? Sure, as long as we don’t label students who fail to achieve this ambitious goal as failures. Is that something I can hold schools accountable for achieving? The answer to that question should be based on evidence.
You determine the reasonableness of a goal by acquiring information about the past performance of the school and, more importantly, the past performance of schools collectively relative to the 1.5 years of growth goal. In other words, a good test of reasonableness is to ask the question “How frequently have schools in the nation have met this goal in the past?”
NWEA publishes growth norms for both students and schools that can be used to empirically test this question. We can use the growth norms to estimate the likelihood that both individual students and schools would meet the goal of 1.5 times the growth norm. Using the subject of mathematics as an example, and considering the fact that we are interested in students who are starting out behind the goal of college readiness, let’s address this particular question: What is the likelihood that a low performing student, in this case a student starting with a below-average score, would reach 1.5 times the average growth in a school year? [1]
Figure 1 presents the answer using the 2015 student norms, and it reveals two important facts. First, at every grade, the majority of students do not meet the year and one half threshold. That’s not particularly surprising since above average growth was needed to meet it. Second, the goal is more commonly achieved in the middle grades and high school than it is in elementary grades. Why is that the case?
Figure 1 – Proportion of low-performing students meeting 1.5 years of growth in mathematics
Without getting overly technical, the reason is that the growth norm for students in the early grades is higher, so what constitutes 1.5 times the growth norm in grade 1 is much greater (in this example 19 RIT according to the 2015 norms) than what constitutes 1.5 times the growth norm at grade 9 (5 RIT). Thus it is much rarer for first graders to show 19 points of growth than it is for ninth graders to show 5 points of growth. That’s why only 10% of first graders meet this target, while 42% of ninth graders achieve it. That is also one of the reasons we do not report years of growth, as people tend to assume that a year of growth in a subject at one grade is the same as another, and this is clearly not the case. Further this misunderstanding can cause people to make bad decisions. If the data in Table 1 represented a school, an uninformed observer would ask why students in the early grades are failing so badly and why upper grade students were so much more successful.
What happens when we extend the criterion to schools? Figure 2 shows this information from the 2015 school norms for mathematics.
Figure 2 – Proportion of schools with low performing students averaging 1.5 years of growth in mathematics
As you can see, the proportion of schools that meet the 1.5 times the norm criterion is much smaller than the proportion of students that reach the goal. Why is that the case? The answer is that it’s much less common to get a group of students to meet a goal than it is to get a single student to meet the goal. Think of it this way—let’s say a teacher picked a 5th grade student from a low-performing school at random and tutored her in mathematics to try and get 1.5 years of growth. Based on the student norms in Table 1, there’s a 23% chance our student will meet that goal. Now imagine trying to get an entire school full of low-performing fifth graders to reach the same target. It’s considerably harder, because the goal has to be achieved with a group of students rather than one, and the law of averages works against us. That’s why only 4% of the schools depicted in Table 2 reach this goal at the fifth grade level.
So is the 1.5 times the growth norm a reasonable goal for most schools? No. We have empirical data across a large population of schools that indicates that the vast majority of schools do not reach this target. The empirical data also show that it is far less common for elementary students to reach this target than middle and high school students.
So what’s the harm in setting unrealistic goals for schools? The harm comes when we make high-stakes decisions about the performance of teachers and schools based on a standard that is rarely reached. Of course, average growth is not enough for low performing students to catch up to new and higher standards. It’s necessary. But that doesn’t mean it’s a good idea to hold teachers and schools accountable for a goal that has been shown to be rarely reached.
In baseball, the standard for hitting excellence is 3 hits per 10 at bats or a batting average of .300. Over 90% of major leaguers fail to reach this standard. That failure doesn’t mean that teams can afford to cut all their sub .300 hitters. They must evaluate players by a different standard, which is WAR (in baseball parlance that’s Wins Above Replacement, or the wins above the likely replacement player). To put it simply, a baseball team doesn’t compare a player’s performance against a .300 batting average. They compare the player’s performance to the group of players, usually minor leaguers and rookies, available to replace him, most of whom actually hit around .220. Think of it this way, if my shortstop is a .275 hitter, he may not be reaching the team’s aspiration for him, but I’m not going to fire him if the odds are that his replacement will hit around .220. Put in education terms, you wouldn’t want to fire a teacher or a school without knowing if the replacement you put in place would be better. The data here show that if you use 1.5 years of growth as the standard for dismissing a teacher (or closing a school), you are almost certain to replace that teacher with someone worse.
So what constitutes a reasonable growth goal? Unfortunately there is not a single number that answers the question. One factor that can be considered is whether the students’ growth is above average, which is whether their growth exceeds that of a representative peer group. If growth is well below average, one can make an argument that students might do better if the school or leadership changed. A second factor is the prior track record of the school. One may not expect a .200 hitter to improve to a .300 level within a year, but they should show evidence of improvement. The third factor to consider is the direction and pace of change. If the school is improving and improvement is accelerating, that should get more credit than a school that has improvement but improvement is decelerating.
—
[1] For our purposes, we used the 20th percentile score from the school norms, as that score more reasonably reflects the average in a low performing school than the 20th percentile score from the student norms. This also ensured that we were evaluating students and schools using the same starting scale score, so that the ensuing comparisons would be direct. Students were evaluated relative to the student norms from this starting score, schools were evaluated relative to the school norms. To illustrate, in grade 6 math, the starting point for both student and school is the 20th percentile of the school norms, or a score of 205. At that score average student growth is about 8 points and 1.5 years of growth would be 12. The average school growth at that same score is 7 points which would round to 10 (wouldn’t this round to 11?). We evaluated student growth by estimating the percentage of sixth grade students in the norming sample who reached 12 points, and we evaluated school growth by estimating the percentage of schools in the sample that reached 11 points of growth.
