************************************************
ABSTRACT: In a previous post “The Ceiling Effect #2” [Hake (2011a)] I noted that Sheila Tobias (2011), in a recent APS News editorial “Teachers in the Crosshairs. . .”, called attention to the “Value-Added Inequity” (VAI) experienced by a Houston high-school physics teacher who reported that her students entered her course with high test scores. As a consequence her students achieved relatively small pre-to-posttest gains (a consequence of the ceiling effect) and she, in turn, received a relatively small value-added bonus.
EDDRA2’s Michael Martin responded that the Houston teacher’s Value-Added Inequity" (VAI) is “best appreciated as a grossly unsophisticated process being employed by fundamentally incompetent administrators.” But EdResMeth's Tony Milanowski wrote that, as he understood it, the model used in Houston was based on the report “SAS EVAAS Statistical Models” [Wright et al., 2010 at http://bit.ly/hPrO7s] which converts Item-Response-Theory-based scale scores to Normal-Curve-Equivalents - but such sophistication evidently did not prevent the VAI experienced by the Houston high-school physics teacher.
Other VAI's were described by PhysLrnR's Boris Korsunsky and Math-Teach's Haim. Boris wrote (paraphrasing): “My own bonus was zero. Massachusetts used a value added method that took into account a town's average parental income. Since my town is by far the wealthiest in the state, we are often ranked ‘below expectations’ in various state-produced rankings - even though Boston magazine has repeatedly ranked us No.1 in the state. Similarly, Haim wrote (paraphrasing): “The value-added formula for evaluating schools in NYC resulted in Stuyvesant High School (one of the top academic institutions in the U.S.) earning a ‘B’.”
Consistent with the above VAI’s, reports by recognized experts critical of the use of value added measures to grade teachers are: (a) “Letter Report to the U.S. Department of Education on the Race to the Top Fund” [NRC (2009)], (b) “Getting Value Out of Value-Added: Report of a Workshop”[NRC (2010)], (c) “Problems With The Use of Student Test Scores to Evaluate Teachers” [EPI (2010)], (d) “Grading teachers on value-added measures falls short” [UCLA Today (2010)], (e) “Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains” [Schochet & Chiang (2010)], (f) “Hurdles Emerge in Rising Effort to Rate Teachers” [Otterman (2010)].
Despite the above criticism, the Department of Education has designed its “Race to the Top” scoring system to reward states that use value-added calculations in teacher evaluations.
************************************************
To access the complete 41 kB post please click on http://bit.ly/fN1HmD .
Richard Hake, Emeritus Professor of Physics, Indiana University
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands
President, PEdants for Definitive Academic References which
Recognize the Invention of the Internet (PEDARRII)
rrhake@earthlink.net
http://www.physics.indiana.edu/~hake
http://www.physics.indiana.edu/~sdi
http://HakesEdStuff.blogspot.com
http://iub.academia.edu/RichardHake
“The [Race to the Top] initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches (which evaluate teachers based on gains in their students' performance) to reward or punish teachers.”
“Letter Report to the U.S. Dept. of Education on the Race to the Top Fund”
[NRC (2009)]
REFERENCES [URL's shortened by http://bit.ly/ and accessed on 18 Jan 2011.]
Hake, R.R. 2011a. “The Ceiling Effect #2” online on the OPEN! AERA-L archives at http://bit.ly/hUnHZe . Post of 12 Jan 2011 16:19:49-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists are also online on this blog “Hake'sEdStuff” at with a provision for comments.
Hake, R.R. 2011b. “ Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” online on the OPEN! AERA-L archives at http://bit.ly/fN1HmD . Post of 18 Jan 2011 15:34:47-0800to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.
NRC. 2009. National Research Council, Board on Testing and Assessment (chaired by E.H. Haertel), Letter Report to the U.S. Department of Education on the Race to the Top Fund, online at http://bit.ly/dOg8v6 .
http://www.physics.indiana.edu/~hake
http://www.physics.indiana.edu/~sdi
http://HakesEdStuff.blogspot.com
http://iub.academia.edu/RichardHake
“The [Race to the Top] initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches (which evaluate teachers based on gains in their students' performance) to reward or punish teachers.”
“Letter Report to the U.S. Dept. of Education on the Race to the Top Fund”
[NRC (2009)]
REFERENCES [URL's shortened by http://bit.ly/ and accessed on 18 Jan 2011.]
Hake, R.R. 2011a. “The Ceiling Effect #2” online on the OPEN! AERA-L archives at
Hake, R.R. 2011b. “ Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” online on the OPEN! AERA-L archives at http://bit.ly/fN1HmD . Post of 18 Jan 2011 15:34:47-0800to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.
NRC. 2009. National Research Council, Board on Testing and Assessment (chaired by E.H. Haertel), Letter Report to the U.S. Department of Education on the Race to the Top Fund, online at http://bit.ly/dOg8v6 .
