In Dallas ISD teachers are evaluated and compensated annually using the Teacher Excellence Initiative (TEI). It is an education Reform merit pay model in which teachers pay is based in part on student's standardized test scores.
The student's test scores are calculated three ways in the TEI - pass rate (raw score) and two growth measurements, one a value-added model (VAM) and the other a student growth percentile (SGP).
The final TEI score incorporates the best of the three. The problem is that academic research considers all three calculations flawed or invalid measures of student achievement. Here I'll look at the VAM growth measurement.
The District purports that their VAM algorithm (named CEI) accounts for all of the variables that could effect a student’s ability to learn (or show growth), thus the calculated score is solely the result of the teacher’s effect.
The scatter plots below show the actual percentile rankings of 1,224 math teachers in Dallas ISD from 2011-2013 using their VAM scores. The center of the cube is the 50th percentile for all three years. If the VAM algorithm used is reliable you would expect a cigar shaped cloud along the green diagonal, indicating reasonably consistent rankings. Slight gradual variations (positive or negative) are expected in teacher's effectiveness, huge annual fluctuations are not.
But the math teacher's VAM scores show wild fluctuations, creating predominantly random distribution patterns on the scatter plots. This level of volatility from one year to the next makes it clear that these scores aren’t justified for use. They are too unstable, in other words unreliable.
Given this instability, the District reportedly changed the VAM algorithm. Unfortunately I have been unable to obtain current data as the District refuses to release the scores claiming both the data is not available and protected under Texas Education Code, Section 21.355.
2012 vs. 2013 There is some minor clustering at the high CEI levels.
2011 vs. 2013 It is almost purely random.
This view looks directly down the diagonal of the cube and clearly shows there is some information and it is not purely random, but there is much noise.