Texas-Style School Accountability: Rewarding the “Best” Schools?

Posted on April 26, 2011

2



Texas is considered a leader in state accountability systems in the nation. Indeed, George W. Bush touted school accountability as the primary driver of the Texas Miracle of the 1990s. Ultimately, this fondness for school accountability Texas-style led to the basic framework of the No Child Left Behind Act (NCLB)–President Bush’s signature education reform effort. While many have critiqued the very existence of the Texas Miracle and make a number of very valid points, this post focuses on the current version of Texas style school accountability which is not much different from the original version implemented in 1993.

While there are a large number of problems with the general notion of school accountability and the Texas school accountability system in particular, this post assumes that accountability systems will not disappear in the near future, thus such systems must be as fair as possible. Again, this post is making a very narrow critique. There are many other valid critique such as an incentive to teach to the test that I will explore in a future post.

MAJOR CRITIQUE

The major critique of the Texas accountability system is that it includes only status measures and ignores growth measures. In other words, the system does not look at how much a school improves student outcomes, but only about the current level of achievement of students.

Now, some would argue that the Required Improvement component of the system assesses growth to some degree, but assessing growth using proficiency measures (percent meeting some standard) results in an inaccurate assessment of of growth.

I will use the case of high schools to provide examples of why looking solely at status indicators rather than both status and growth measures dooms some schools to a low accountability rating no matter how well they do and rewards other schools with a high accountability rating no matter how terrible they do.

As shown in Figure 1 below, the proportion of incoming 9th grade students not passing the 8th grade TAKS mathematics test differ substantially across schools with different accountability ratings. Almost none of the schools rated Exemplary in 2010 had more than 10% of incoming 9th grade students who had failed the 8th grade TAKS math test. Thus, the incoming ability level of 9th grade students appears to predict the school’s accountability rating. At the other end of the spectrum, 75% of the schools rated low-performing in 2010 had at least 20% of 9th grade students who had failed the 8th grade TAKS math test. The median percentage of 9th grade students failing the 8th grade math test was 25%. If 25% or more of the incoming 9th grade students have not passed the 8th grade math test, do we really expect that school to be able to have enough students meet the commended standard to be acceptable or exemplary? And this same pattern happens every year. So there are high schools in Texas where every single cohort of 9th grade students have a substantial percentage of students who have failed the 8th grade math TAKS or not even taken the test. Such schools realistically have no chance to meet the exemplary standards on commended status.

FIGURE 1:

SOURCE: Student-Level TAKS Scores and School Aggregate TAKS scores, TEA; ANALYSIS: Dr. Ed Fuller

As shown in FIGURE 2, the disparity is somewhat greater when comparing the three-year average percentage of incoming 9th grade students not passing the 8th grade TAKS math test. Indeed, the three-year average percentage of incoming 9th grade students not passing the 8th grade math TAKS (2007, 2008, and 2009) was 33% for low-performing schools and only 8% for schools rated exemplary.

FIGURE 2:SOURCE: Student-Level TAKS Scores and School Aggregate TAKS scores, TEA; ANALYSIS: Dr. Ed Fuller

FIGURE 3 takes a slightly different look by examining the percentage of students entering the 9th grade who met the commended level (scale score of 2400 or greater) on the 8th grade math TAKS. More than one-half of schools rated exemplary in 2010 had at least 25% of incoming 9th grade students who had met the commended standard on the 8th grade TAKS math test. If this happens every year–and it does for many schools–then the school would be meet the exemplary standard for commended status even if they elicited no growth from the students. So, essentially, a substantial number of schools would meet the new exemplary or recognized standard without doing much at all except for having high-performing feeder schools!

FIGURE 3:SOURCE: Student-Level TAKS Scores and School Aggregate TAKS scores, TEA; ANALYSIS: Dr. Ed Fuller

In fact, the mean scale score on the 8th grade TAKS math test for incoming 9th grade students is highly correlated ( .646) with the percentage of students achieving commended status in grades 9, 10, and 11 in 2010. Thus, the average scale scores of the incoming cohorts are strongly associated with the percentage of students at the commended level. Again, the best strategy to have a high percentage of students at the commended level. Why are we rewarding high schools for having high percentages of students at the commended level when many schools would meet the standard based on the performance of their feeder schools? Why are we rewarding schools for having high-performing feeder schools instead of rewarding schools that actually IMPROVE the performance of students?

FIGURE 4

SOURCE: Student-Level TAKS Scores and School Aggregate TAKS scores, TEA; ANALYSIS: Dr. Ed Fuller

Let’s look at this from another perspective. Table 1 shows some of the”best” high schools in the state. I have redacted the names because using the actual school names does not serve much good. But these are actual schools in Texas. Note that all these schools were rated exemplary in 2010 and most were rated recognized or exemplary for the previous years.

TABLE 1:


SOURCE: School accountability ratings from AEIS, TEA

Yet, as we see in Table 2, many of these schools had decreases in student achievement (as measured by math and reading TAKS scale scores converted into Z-scores). Yes–the schools had DECREASES in student achievement relative to all other schools. Yet, they were still rated exemplary. Look at the Small Suburban School #2. The school had staggeringly large decreases of around one standard deviation every year in math and reading, yet maintained its recognized rating.

TABLE 2:

SOURCE: School aggregate TAKS scores, TEA; ANALYSIS: Dr. Ed Fuller

Let’s look at this from the opposite direction–the “worst “schools in Texas. Table 3 includes some of the “worst” high schools in the state–schools with consistently low accountability ratings. Pay particular attention to Urban 5. That school was rated low-performing (AU) for 4 of the 6 years. This school would be threatened with closure.

TABLE 3:

SOURCE: School accountability ratings from AEIS, TEA

But wait!!!! When we look at the actual student achievement of teh students in the school, the students actually have fairly large growth. On average, the students had math achievement .24 standard deviations GREATER than the average school and reading achievement .32 standard deviations GREATER than the average school. At no point in time did the school have achievement lower than the average school in Texas. Yet, the accountability system blames the educators in these schools. In fact, many would argue the educators should be fired. But the educators improved student achievement. Shouldn’t we be REWARDING these educators rather than DEMONIZING them? Or, at the very least, providing them with support to improve achievement even further?

TABLE 4:

SOURCE: School aggregate TAKS scores, TEA; ANALYSIS: Dr. Ed Fuller

Thus, our accountability system does not do a terrific job at identifying effective and ineffective schools because our system focuses on status measures (measures taken at one point in time) rather than GROWTH measures! No wonder we have schools that have great difficulty in attracting and retaining well-qualified teachers and administrators! Why would effective educators go to and stay at a school where they know the accountability system and the media punish them even though they have been successful???

If we are going to have an accountability system for Texas public schools, then we need to have a system that accurately identifies effective and ineffective schools. Our current system does not do this, primarily because we do not have a growth measure. And, while the Texas Projection Measure (TPM) helped adjust for this partially, a projection measure simply does not measure growth. Indeed, even when using the TPM in the accountability system, I found that:

Let’s push for a system that is FAIR to educators so that there will be no disincentive for teachers and administrators to work in schools in which sudents enter the school behind grade level! Let’s do accountability right or don;t do it at all!!

Advertisements
Posted in: Uncategorized