This study uses data from the State of Texas to compare three year averages in student characteristics and performance between the one IDEA Charter elementary school with data available over the three year time frame from 2007-08 through 2009-10. The data are all available from either the Texas Comptroller’s FAST website (http://fastexas.org/) or the Academic Excellence Indicator System (http://ritter.tea.state.tx.us/perfreport/aeis/) website from the Texas Education Agency.
Before getting to achievement, let’s examine the student characteristics of East Austin schools and IDEA Academy. Unfortunately, we cannot examine prior academic ability since there is no measure prior to the third grade.
First, let’s compare East Side elementary schools to IDEA Academy in terms of the percentage of economically disadvantaged students.
FIGURE 1: Average* Percentage of Economically Disadvantaged Students
for IDEA Academy and East Austin Elementary Schools
* Three year avg (07-08, 08-09, 09-10); SOURCE: FAST, Comptroller’s Office and AEIS Data, TEA
IDEA Academy has a lower percentage of economically disadvantaged students than all but two of the 32 East Austin elementary schools. In fact, when compared to all schools in Austin, there were 47 schools with a greater percentage of economically disadvantaged students.
Let’s turn to the percentage of Hispanic students (Figure 2). IDEA Academy has a greater percentage of Hispanic students than East Austin elementary schools and all AISD elementary schools.
FIGURE 2: Average* Percentage of Hispanic Students
for IDEA Academy and East Austin Elementary Schools
* Three year avg (07-08, 08-09, 09-10); SOURCE: FAST, Comptroller’s Office and AEIS Data, TEA
While IDEA Academy had a greater percentage of Hispanic students, IDEA Academy enrolled no African American students. The geographic area from which IDEA will recruit students definitely has African American students. How do we know IDEA has, err, any idea how to effectively educate African American students? State and Austin data shows schools struggle more with African American students than with Hispanic students.
FIGURE 3: Average* Percentage of African American Students
for IDEA Academy and East Austin Elementary Schools
* Three year avg (07-08, 08-09, 09-10); SOURCE: FAST, Comptroller’s Office and AEIS Data, TEA
Although IDEA Academy has a greater percentage of Hispanic students, it has a lower percentage of bilingual students than most East Austin schools. IDEA leaders claim that this is abacus IDEA transitions students out of bilingual education earlier than other schools. Yet, in my analysis of characteristics entering IDEA schools from 5th grade to 6th grade, students taking the Spanish-language TAKS test were far less likely to leave the public schools and enter IDEA Charter Schools. Will IDEA actively recruit bilingual students? Are they prepared to promise–on paper–that they will hire bilingual recruiters to search out the families whose parents do not speak English?
FIGURE 4: Average* Percentage of Bilingual Education Students
for IDEA Academy and East Austin Elementary Schools
* Three year avg (07-08, 08-09, 09-10); SOURCE: FAST, Comptroller’s Office and AEIS Data, TEA
Figure 5 examines student mobility rates. The student mobility rate is defined as the percentage of students not enrolled in the school for at least 6 weeks of the 36 week school year. If IDEA Academy was an East Austin school, it would have the lowest mobility rate of all schools. This makes it far, far easier to elicit gains from students. Unfortunately, regular public schools cannot remove students for poor attendance, nor can they refuse entry to a student after the start of the school year. I don’t know if IDEA removes students for poor attendance. I know that many charters do, such as KIPP. Like many charters, IDEA only accepts students at the beginning of the school year. No students are allowed to transfer into the school once the school year has started. Public schools, on the other hand, must allow any child to enter at any date. This puts IDEA Charter Schools in a very advantageous position since we know mobility has a tremendous negative effect on achievement and other outcomes.
FIGURE 5: Average* Percentage of Mobile Students
for IDEA Academy and East Austin Elementary Schools
* Three year avg (07-08, 08-09, 09-10); SOURCE: FAST, Comptroller’s Office and AEIS Data, TEA
Finally, Figure 6 portrays the percentage of special education students in East Austin schools and IDEA Academy. Only a few East Austin schools had a lower percentage of special education students than IDEA Academy. IDEA leaders again claim that they are less likely than other schools to identify students as special education students. Yet, again, my previous analysis found that students taking TAKS tests that indicated the student had some type of disability in the 5th grade were far less likely to enter IDEA Charter Schools than students without any disability.
FIGURE 6: Average* Percentage of Special Education Students
for IDEA Academy and East Austin Elementary Schools
* Three year avg (07-08, 08-09, 09-10); SOURCE: FAST, Comptroller’s Office and AEIS Data, TEA
As shown in the graphs above, IDEA Academy serves a student population radically different from the average East Austin elementary school. IDEA Academy has fewer low-income students, special education students, African American students, bilingual education students, and mobile students than the average East Austin elementary school. With respect to Allan elementary, IDEA Academy had substantially lower percentages of low-income students, special education students, and bilingual students. Any contract with IDEA should require IDEA to enroll students that match the existing Allan population with respect to the above characteristics.
If, in fact, IDEA Austin ends up enrolling fewer special education, low-income, and bilingual education students (or mobile students), then the students that research suggests are harder to teach will simply be moved into neighborhood elementary schools while more advantaged students enter into IDEA Austin. This will likely decrease the achievement and, potentially, the accountability ratings of surrounding schools. Will surrounding schools be compensated for this? Will they receive additional funding and support? Austin ISD is silent on these issues.
ACHIEVEMENT
Finally, let’s get to achievement levels.
While Austin ISD and IDEA Charter Schools continue to use percent passing as the metric to determine whether schools are doing well, the far more accurate manner to measure school effectiveness is to use student growth. Moreover, to use student growth over a three-year time span in order to ensure consistency over time is preferred by researchers.
To do this, I rely on the Financial Allocation Study for Texas (FAST) data system on the Texas State Comptroller’s website. The site can be found at: http://fastexas.org/
The FAST study used student-level data to identify student-, school-, and district-level growth for every public and charter school in Texas. The results are reported in Z-scores. The study uses z-scores for a number of reasons, but primarily because using z-scores allows for a fairer comparison across years and across grade levels. Using z-scores also makes accurate comparisons between schools somewhat easier to make since z-scores are aligned on a normal curve.
Below is a picture of a z-score distribution. Note that the average, or mean, is 0. In the FAST analysis, 0 indicates average growth for all schools in the state. Positive z-scores indicate positive growth and negative z-scores indicate negative growth. The further the z-score is from 0, the more positive or negative the growth measure is.
FIGURE 6: Z-Score Distribution
In comparing IDEA Charter elementary schools to Austin ISD elementary schools, only one IDEA Charter elementary schools had three years of data available for the FAST study (the academic years required for the study were 07-08,08-09,09-10). This was the IDEA Academy.
In Table 1, the FAST z-scores are presented for IDEA Academy and all East Austin Elementary Schools.
MATHEMATICS & READING COMBINED: IDEA Academy had a three-year average growth score for mathematics and reading combined (0.063) that was only marginally greater than the average for ALL EAST AUSTIN SCHOOLS (0.017). This means IDEA Academy achieves at about the same level as the average for ALL East Austin elementary schools.
MATHEMATICS: IDEA Academy performed 0.3 standard deviations below the average for ALL East Austin schools and actually had NEGATIVE growth over a three-year time frame. Thus, IDEA Academy had math growth that was SUBSTANTIALLY lower than other schools in East Austin.
READING: In reading, IDEA Academy performs above the East Austin average, but still below 10 East Austin schools and dramatically lower than five of them.
Further, look at how many East Austin schools have far, FAR greater growth than IDEA Academy. Graham, Blanton, Metz, Overton, and Norman all have MUCH SUPERIOR STUDENT GROWTH PROFILES than IDEA Academy.
Instead of wasting precious dollars on an outside entity that will NOT be accountable to the East Austin community, why doesn’t the Superintendent and School Board simply ask the leaders and teachers of these five OUTSTANDING EAST AUSTIN schools how to transform Allan and other lower-performing elementary schools in East Austin? The leaders and teachers in these five schools KNOW the East Austin students and community well and understand how to be successful in that community.
***And if you don’t believe me, go to the FAST and AEIS data sites yourself. All the data is there. Download it and check it out yourself. And let me caution not to use one year growth scores–they are highly unreliable which is why the FAST researchers responsibly relied on the three-year growth profiles in their analyses.
A "Fuller" Look at Education Issues 
Jim
December 11, 2011
Important analysis to make, but the exclamation marks and colors reduce the persuasiveness level. You can make the same points more persuasively without resorting to these things.
Jim
December 11, 2011
Saw that you changed this later. I hope the school board considers your points. A big step like this shouldn’t be entered into lightly. Children’s futures are at stake. However, I note that Allan Elementary, which the proposal is for IDEA to take over in the first year, is actually relatively similar demographically to the IDEA school. It is pretty close on most of the graphs. It has also done far worse than IDEA in the z-score comparison.
Dr. Ed Fuller
December 11, 2011
You are correct. But why not ask the educators in the high-growth austin elementary schools how to improve Allan? Out of all the people in the world, they would know best on how to improve the situation.
Marshall
December 12, 2011
I was curious as to whether these findings hold at the secondary level, since you present some analysis at the secondary level in your next blog-post, but don’t incorporate the FAST data.
The results are here: http://goo.gl/aGSCS. I’ve compiled comparable FAST results for both the elementary and secondary grades in separate tabs in the spread sheet, just to be sure we’re both using the same variables and FAST .csv file. The results suggest that East/West distinctions are essentially useless for this grade span, since the only campuses that are performing as strongly as IDEA schools (using the three-year average measures) are magnets in AISD (Ann Richards and LASA). Interestingly, although they exceed the growth measures for other AISD schools in reading and the composite metric, they performed less well in mathematics. But, they still did better than all other East Austin schools on this measure.
Dr. Ed Fuller
December 12, 2011
I did look at the secondary FAST results, but the grade configuration of IDEA (and KIPP and YES, for that matter) make comparisons difficult because almost no other regular schools have those grade configs–only alternative schools. Now, the FAST researchers may have controlled for this in some way when calculating the results, but I don’t see how they could. More likely, they simply took the student-level standardized residuals and aggregated them at the school level. If this is the case, then maybe the middle school kids do extremely well and the high school kids do poorly, but because there are more MS kids, the z-score ends up being high. My gut just says it is not a fair comparison to make.
Marshall
December 12, 2011
That’s a reasonable concern. But, yes, they adjusted for student-level grade (see page 5 here http://goo.gl/UOqib) which should address your weighting worry (this is just like an AISD fixed effect in a model predicting some outcome for Central Texas. If you don’t include it, your results may be driven by an AISD effect). So, unless the assumption is that the grade configuration is a confound, and not the student grade level, then this should not bias the residuals used to compute the campus and district effects. They did not include grade configuration in the campus-level portion of the equation, which would be another line of objection. But, I probably wouldn’t want to argue the technical merits of their modeling choices, given the people on their technical advisory committee 🙂
Dr. Ed Fuller
December 13, 2011
Thanks. My mind must have have been numb by the time I got to that point in the explanation. I understand why they wouldn’t include it given the myriad grade configurations that shift from one year to the next. I do think it is a factor influencing achievement, however.
Note, too, that the results are still correlated with % eco dis. I spoke with one of the financial researchers and s/he said that the model simply could not control for all the effects of poverty.
I certainly wouldn’t want to get in a technical argument with the FAST crew. But, I do see some possible ways to improve it.
Do you know of any studies that estimate the effect of kids disappearing from a cohort? That is the one factor that seems consistent across many high-performing charters (and public schools as well)–disappearance of lower-performing students. That may be the “take away” policy of such schools–have such high expectations with minimal support to drive out lower-performing kids. Not saying that is what always goes on, but I have seen it in schools I am familiar with.