Algorithms are Opinions Embedded in Code
I have been very vocal about the destructive Teacher Excellence Initiative (TEI) in Dallas ISD. In order to formally rebut the system I have wanted to run an independent analysis and have been trying to get TEI raw data from DISD for almost a year now. But DISD won’t give it up. As a taxpayer, parent, and concerned community member, their refusal is both troubling and telling....very.
Luckily, we have someone in the metroplex with the expertise needed to do just such an analysis, Dr. Michael Dryden. Dr. Dryden has taught middle and high school math and science in the South Pacific, Indonesia, Australia, and the U.S., and has a Doctorate in Research and Evaluation in Science Education. He has analyzed TEI extensively and you can read part 1 of his series here, which focuses on the TEI bias towards choice schools. Continue reading this blog for my layman’s interpretation of Dr. Dryden’s analysis. (Sources are referenced in Dr. Dryden’s full report).
TEI was piloted in DISD in 2013 and implemented district wide in 2014. It is a combination evaluation and compensation system. It is commonly referred to as a merit pay system and claims to get rid of “bad teachers”. Here is an explanation of how it works.
Dr. Dryden has explained specifically why Dallas ISD’s TEI model is very problematic. TEI is biased because it doesn’t properly adjust for the emphasis placed on education in the home nor does TEI adequately control for prior student achievement. It is well known that students from affluent homes, homes where education is emphasized, and those who have previously exceeded in education are the very students who are known to perform well on standardized tests. TEI deems teachers of these exact students as the most “effective” teachers in the district, thus confirming inherent bias in the TEI system. DISD’s own research department has been reporting this finding every year for the past three years, here’s a statement from the 2017 district TEI report summary:
Majority white campuses, choice campuses, collegiate academies, and magnet campuses have the highest Summative Performance Scores, the highest student achievement scores and the highest percent teachers rated Proficient I or higher.
Figure 3 below, shows the higher “choice” campus salaries are probably due in part to a grandfathering of salaries but also the TEI process is more likely to assign choice teachers to higher TEI performance categories.
Percent of teachers within each TEI category by choice and non-choice (neighborhood campus).
[Figure 3 in Dr. Dryden's report]
Take a look at Figures 4 and 5, which show the difference in median pay in DISD between the top 25 and bottom 25 campuses. It is heavily biased towards campuses of choice and against neighborhood campuses especially middle schools.
Median Salary of the top 25 paying campuses in Dallas ISD, 2017-18. Student choice campuses
are in red. [Figure 4 in Dr. Dryden's report]
Median salary of the bottom 25 campuses in Dallas ISD, 2017-2018. [Figure 5 in Dr. Dryden's report]
The other fatal flaw in TEI is that it is not a true pay for performance model, it is simply a pay by distribution model. In other words, the system is fixed to only allow for 20% of teachers to actually receive the higher effectiveness ratings. The remaining 80%, regardless of their performance evaluation, will be relegated into the lower effectiveness ratings. Here is the actual DISD TEI targeted distribution model:
[Figure 1 in Dr. Dryden's report]
The Dallas business community presents this to you and me as DISD retaining its most effective teachers, when in fact it is simply a fixed distribution. Said another way, the more your pay someone the more likely they are to stay in the system. The longer you keep someone’s pay the same, deny earned raises, the more likely they are to exit the system. Trustee Audrey Pinkerton reported this earlier this year. Because TEI is biased towards teachers of higher achieving students, it is penalizing teachers of lower achieving students, students without strong educational emphasis in the home.
Dr. Dryden sums it up nicely:
The results below do not mean many teachers in schools of student choice are not effective, nor does it mean many neighborhood campus teachers are ineffective. It just means the current TEI system metrics and algorithms are insensitive to the differences in these campuses and comparisons across these different campuses are biased and lead to false conclusions and dubious fixed pay distributions.
This education Reform model is wrong. Read Dr. Dryden’s conclusion for possible solutions. We as a community must demand better from our elected officials, in this case our Board of Trustees. We should come together for the sake of the future of education in Dallas.