Recently, TEA has removed TPM from use as one component in school accountability ratings. Many superintendents are upset because they felt TPM gave them credit for improvement on TAKS with students who had not yet cleared the passing hurdle. I understand this sentiment. Before TPM, there was absolutely no component of the accountability system that recognized any improvements in student achievement.
For one superintendent’s perspective, read the following letter to TEA:
https://docs.google.com/document/d/1XldiWf9-8lIjAvd95VMv5EVKrckknBfVPs-P790xA64/edit?hl=en&pli=1#
The problem, whoever, is that TPM is NOT a growth measure and did NOT accurately assess student growth. Rather, TPM was a statistical prediction of where a student might score in the future based on past performance. I am a researcher and statistician and do this type of work for a living as an education consultant. I have run such models before. Having done such work in the pass, I can tell you that predicting future performance is incredibly inaccurate.
While TEA claimed that the TPM was a fairly accurate predictor of future student performance, this was true only if you examined all students. Well, most students score either far above or far below the passing standard. You don;t need a statistical procedure to tell you that a student who has passed TAKS every year and passed TAKS by more than 3 or 4 questions on the last assessment will pass TAKS on the next assessment. Similarly, you don’t need a statistical measure to tell you that a student who has never passed TAKS and is still 3 or 4 questions below the passing standard is unlikely to pass TAKS in the future. TPM was very accurate in predicting future performance for these kids. But my 10 year old daughter would be fairly accurate in making the same predictions.
The difficulty came in predicting kids 3 questions above and below the passing standard. It turns out that TPM was about as accurate as a coin flip in predicting future performance for these kids.
In fact, TEA was told that TPM was initially less accurate than flipping a coin in predicting future performance of kids scoring around the cut score. They went ahead with the measure anyway knowing full well it did not work very well for kids scoring around the cut score.
There is no doubt that TPM benefited school districts. But it did not accurately assess how well a district improved a student’s performance. The only way to do that is with a growth measure. A growth measure does not predict future performance, but looks backward in time and assesses how much growth a student made. While growth measures are not 100% accurate either, they are far, far more accurate than projection measures.
Why Superintendents and Principals Should Care
As I showed in my previous post entitled Texas-Style School Accountability: Rewarding the “Best” Schools? found at https://fullerlook.wordpress.com/2011/04/26/texas-school-accountability/, the incoming ability level of students strongly predicts high school outcomes, even with TPM. The same is true at the middle school level. If we could assess the incoming ability level of kids before they entered elementary school, we would find the same result. Thus, even with TPM, schools with incoming students scoring 0.5 standard deviations below average were 7 times more likely to be low-performing than other schools. TPM does not help in this regard because it does not give you credit for the growth of students except those around the passing standard and those estimates are wrong as often as they are right.
So, you should push strongly for school growth measures as a primary component of the accountability system. The accountability system should examine both the level of achievement and the GROWTH in achievement. As I show in the other post, some schools have low achievement and great growth, but are rated low-performing or acceptable. Other schools have high levels of achievement and negative growth, yet are rated exemplary. Which school do YOU think should be designated as a failure or a success?
The reliance on the current accountability system rewards some schools and districts who are not worthy of recognition and punishes other schools and districts that are worthy of recognition. This only serves to exacerbate glaring inequities in educator quality and retention across schools and districts! In short, the current accountability system makes it more difficult to recruit and retain the high-quality staff we need at schools serving the students who need the most help. Why work at a school or district that you know will never be rated more than acceptable under the current system regardless of how much growth you elicit from your students???
If you support accurately identifying and rewarding schools and districts that are helping students improve, then you should support a growth measure. Adoption of a growth measure, if done correctly, could be used to increase or modify an accountability rating. Doesn’t TPM do this? No, it does not! See my previous post entitled Texas-Style School Accountability: Rewarding the “Best” Schools? found at https://fullerlook.wordpress.com/2011/04/26/texas-school-accountability/.
So, for example, if a school or district barely missed the recognized standard, but students made significant growth, then TEA could increase the rating to Recognized from Acceptable. Or, if a school district had low levels of achievement, but relatively large growth, TEA could add a designation such as “Academically Unacceptable but Strong Growth” or replace Academically Unacceptable with a new designation such as “Improving” to differentiate the school or district who is moving student achievement forward from the school or district that is not. On the other hand, a school or district rated exemplary that has students falling behind relatively to their peers should NOT receive an Exemplary rating. The rating should be decreased to Recognized or a new rating of “Exemplary, No Improvement” be added. This will make the system slightly more complicated, but it would be far, far more accurate and fair to the many hard-working educators who traditionally are never recognized for a job well done under our current system.
At the student level, I understand that the TPM gives hope and encouragement to students and teachers. Yet, the measure is often wrong. A growth measure–if constructed properly and presented properly–could serve the same purpose and be far more accurate. This greater accuracy would help in identifying the students who truly needed the most help and those who need less help.
Standards Should Not Be Changed Mid-Stream
As I state in my other post (https://fullerlook.wordpress.com/2011/04/24/moving-target/), TEA and the legislature should NOT move the standards or targets for schools and districts at the last second. This goes against everything we know about how targets and goals work to improve performance. If we had an accountability system for state policy makers, the system would label them low-performing in this area. You and the public deserve better.
Call your legislator and DEMAND that a fairer accountability system be put in place that accurately assess the quality of schools and districts! Right now, we reward people more for being in the right place rather than for their hard work, effort, and success.
Please read the comment below. It is an EXCELLENT comment and should be taken to heart by all of us who care about children and the future of Texas. One note is that a growth measure would provide an incentive for everyone to keep pushing all students forward regardless of where they are on the state test continuum.
A "Fuller" Look at Education Issues 
Rick Archer
April 27, 2011
Dr. Fuller,
I appreciate your analysis on all topics. This one is especially important to all of us who are educators and also parents.
I am tired of the status quo in regard to student learning. Student success should be an ongoing process, regardless of a student’s level of achievement. Because of the systemic make up of the TAKS test (soon to be the Staar) students who are below, or on the bubble receive much more help from the districts than do students who are high achievers. What happens to the student who receives high marks or commended on the test? They are put on the back burner and all but forgotten.
Goals for these students, and the districts that serve them, should not be to pass the test and be ignored. The goal should be to excel and then excel some more. This is the flaw of high stakes testing. TPM was just a way for the state to make itself look better in the eyes of not only Texans, but the United States as a whole. Student achievement is not a race to a finish line; there should be no such thing in education. Until the powers that be in this state see student success as an ongoing process, and not a means to an end, our students will never reach their full potential. The TAKS test, and anything resembling it, should be a “benchmark” and nothing else.
Student academic success should be based on an incremental scale, with continued completion of each new academic goal as the measurement of success. This assessment of achievement should be done every six weeks or less, not once a year. We put too much pressure on our children when we make the state test itself the end all, be all of our academic system. It is time to stop this madness and return to a common sense approach in educating our children. It is time to rid ourselves of high stakes testing. Please join me in saying no to the TAKS/Staar test.
Rick Archer
Professional Educator
Marshall
April 27, 2011
This is pretty spot on.
I’ve never really understood the TPM. It seems as though the interest should not be in the projection but in the residual produced from the forecast–that is, whether the student underperformed or outperformed the predicted score.
Bring on the the Colorado Growth Model, I say.
Dr. Ed Fuller
April 27, 2011
From a research perspective, I never understood the rational behind TPM either. If it worked, it would be incredibly valuable to educators in determining which students to spend scarce resources on. Its like the vertical scale score–if it worked it would be great. But it does not. The CO model would be a HUGE improvement over what we have now. I don’t know why we are not moving in that direction.