What You Don’t Know About TPM and Why You Should Demand a Growth Measure

Posted on April 27, 2011

3



Recently, TEA has removed TPM from use as one component in school accountability ratings. Many superintendents are upset because they felt TPM gave them credit for improvement on TAKS with students who had not yet cleared the passing hurdle. I understand this sentiment. Before TPM, there was absolutely no component of the accountability system that recognized any improvements in student achievement.

For one superintendent’s perspective, read the following letter to TEA:

https://docs.google.com/document/d/1XldiWf9-8lIjAvd95VMv5EVKrckknBfVPs-P790xA64/edit?hl=en&pli=1#

The problem, whoever, is that TPM is NOT a growth measure and did NOT accurately assess student growth. Rather, TPM was a statistical prediction of where a student might score in the future based on past performance. I am a researcher and statistician and do this type of work for a living as an education consultant. I have run such models before. Having done such work in the pass, I can tell you that predicting future performance is incredibly inaccurate.

While TEA claimed that the TPM was a fairly accurate predictor of future student performance, this was true only if you examined all students. Well, most students score either far above or far below the passing standard. You don;t need a statistical procedure to tell you that a student who has passed TAKS every year and passed TAKS by more than 3 or 4 questions on the last assessment will pass TAKS on the next assessment. Similarly, you don’t need a statistical measure to tell you that a student who has never passed TAKS and is still 3 or 4 questions below the passing standard is unlikely to pass TAKS in the future. TPM was very accurate in predicting future performance for these kids. But my 10 year old daughter would be fairly accurate in making the same predictions.

The difficulty came in predicting kids 3 questions above and below the passing standard. It turns out that TPM was about as accurate as a coin flip in predicting future performance for these kids.

In fact, TEA was told that TPM was initially less accurate than flipping a coin in predicting future performance of kids scoring around the cut score. They went ahead with the measure anyway knowing full well it did not work very well for kids scoring around the cut score.

There is no doubt that TPM benefited school districts. But it did not accurately assess how well a district improved a student’s performance. The only way to do that is with a growth measure. A growth measure does not predict future performance, but looks backward in time and assesses how much growth a student made. While growth measures are not 100% accurate either, they are far, far more accurate than projection measures.

Why Superintendents and Principals Should Care

As I showed in my previous post entitled Texas-Style School Accountability: Rewarding the “Best” Schools? found at https://fullerlook.wordpress.com/2011/04/26/texas-school-accountability/, the incoming ability level of students strongly predicts high school outcomes, even with TPM. The same is true at the middle school level. If we could assess the incoming ability level of kids before they entered elementary school, we would find the same result. Thus, even with TPM, schools with incoming students scoring 0.5 standard deviations below average were 7 times more likely to be low-performing than other schools. TPM does not help in this regard because it does not give you credit for the growth of students except those around the passing standard and those estimates are wrong as often as they are right.

So, you should push strongly for school growth measures as a primary component of the accountability system. The accountability system should examine both the level of achievement and the GROWTH in achievement. As I show in the other post, some schools have low achievement and great growth, but are rated low-performing or acceptable. Other schools have high levels of achievement and negative growth, yet are rated exemplary. Which school do YOU think should be designated as a failure or a success?

The reliance on the current accountability system rewards some schools and districts who are not worthy of recognition and punishes other schools and districts that are worthy of recognition. This only serves to exacerbate glaring inequities in educator quality and retention across schools and districts! In short, the current accountability system makes it more difficult to recruit and retain the high-quality staff we need at schools serving the students who need the most help. Why work at a school or district that you know will never be rated more than acceptable under the current system regardless of how much growth you elicit from your students???

If you support accurately identifying and rewarding schools and districts that are helping students improve, then you should support a growth measure. Adoption of a growth measure, if done correctly, could be used to increase or modify an accountability rating. Doesn’t TPM do this? No, it does not! See my previous post entitled Texas-Style School Accountability: Rewarding the “Best” Schools? found at https://fullerlook.wordpress.com/2011/04/26/texas-school-accountability/.

So, for example, if a school or district barely missed the recognized standard, but students made significant growth, then TEA could increase the rating to Recognized from Acceptable. Or, if a school district had low levels of achievement, but relatively large growth, TEA could add a designation such as “Academically Unacceptable but Strong Growth” or replace Academically Unacceptable with a new designation such as “Improving” to differentiate the school or district who is moving student achievement forward from the school or district that is not. On the other hand, a school or district rated exemplary that has students falling behind relatively to their peers should NOT receive an Exemplary rating. The rating should be decreased to Recognized or a new rating of “Exemplary, No Improvement” be added. This will make the system slightly more complicated, but it would be far, far more accurate and fair to the many hard-working educators who traditionally are never recognized for a job well done under our current system.

At the student level, I understand that the TPM gives hope and encouragement to students and teachers. Yet, the measure is often wrong. A growth measure–if constructed properly and presented properly–could serve the same purpose and be far more accurate. This greater accuracy would help in identifying the students who truly needed the most help and those who need less help.

Standards Should Not Be Changed Mid-Stream

As I state in my other post (https://fullerlook.wordpress.com/2011/04/24/moving-target/), TEA and the legislature should NOT move the standards or targets for schools and districts at the last second. This goes against everything we know about how targets and goals work to improve performance. If we had an accountability system for state policy makers, the system would label them low-performing in this area. You and the public deserve better.

Call your legislator and DEMAND that a fairer accountability system be put in place that accurately assess the quality of schools and districts! Right now, we reward people more for being in the right place rather than for their hard work, effort, and success.
Please read the comment below. It is an EXCELLENT comment and should be taken to heart by all of us who care about children and the future of Texas. One note is that a growth measure would provide an incentive for everyone to keep pushing all students forward regardless of where they are on the state test continuum.

Advertisements
Posted in: Uncategorized