Some blog followers might be interested in a discussion list post “Re: Scientific American Article on Educational Research and Evaluation” [Hake (2014)]. The abstract reads:
********************************************
ABSTRACT: EvalTalk’s David Colton wrote at http://bit.ly/1vuEiDu (my inserts at “. . . . . . .[[insert]] . . . . . . . . . ”:
“A decade ago, the ‘American Evaluation Association’ . . . . . [[http://bit.ly/1s1m5Mb]]. . . . issued a position paper . . . . . .[[(AEA, 2003) at http://bit.ly/1tgrYsI, highly critical of ]]. . . . . . . . the U.S. Dept. of Education’s (USDE’s) decision to award research grants based on methodology, with experimental and quasi experimental designs given funding prior over other approaches . . . . . . .[[“experimental” is RCT enthusiasts’ code for methodology utilizing “Randomized Control Trails” (RCTs)]]. . . . So I was very interested in an article in this month’s Scientific American which describes the results of this process ten years out: ‘The Science of Learning’ . . . . . . .[[Kantrowitz (2014), re-titled in the online version “Scientists Bring New Rigor to Education Research” and online at http://bit.ly/1v23502]]. . . . . . . . .”
In the present post I excerpt and annotate 8 noteworthy passages from Kantrowitz’s article dealing with e.g., the RCT debate http://bit.ly/1vV222A; the “What Works Clearing House” http://ies.ed.gov/ncee/wwc/; the “Institute of Education Sciences” (IES) http://ies.ed.gov/; Paulo Blikstein's “FabLab” at Stanford http://bit.ly/1yIcekw; teacher evaluation http://bit.ly/1xm6R6b, class size http://bit.ly/1naRX27; student engagement http://bit.ly/9484DG, discovery learning http://bit.ly/1snHAK3, Grover Whitehurst http://bit.ly/RIcEz4, “Finnish Lessons” http://bit.ly/JpU9fD; and “Finnishing Touches” http://bit.ly/Ixkqa7.
**********************************
To access the complete 131 kB post please click on http://bit.ly/1smsIKA.
Richard Hake, Emeritus Professor of Physics, Indiana University; LINKS TO: Academia http://bit.ly/a8ixxm; Articles http://bit.ly/a6M5y0; Blog http://bit.ly/9yGsXh; Facebook http://on.fb.me/XI7EKm; GooglePlus http://bit.ly/KwZ6mE; Google Scholar http://bit.ly/Wz2FP3; Linked In http://linkd.in/14uycpW; Research Gate http://bit.ly/1fJiSwB; Socratic Dialogue Inducing (SDI) Labs http://bit.ly/9nGd3M; Twitter http://bit.ly/juvd52.
REFERENCES [URL shortened by http://bit.ly/ and accessed on 10 Oct 2014.]
Hake, R.R. 2014, “Re: Scientific American Article on Educational Research and Evaluation," online on the OPEN! AERA-L archives at http://bit.ly/1smsIKA. The abstract and link to the complete post are being transmitted to several discussion lists.
Showing posts with label Teacher Evaluation. Show all posts
Showing posts with label Teacher Evaluation. Show all posts
Friday, October 10, 2014
Tuesday, January 18, 2011
Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?
Some blog followers might be interested in a recent post “Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” [Hake (2011b)]. The abstract reads:
************************************************
ABSTRACT: In a previous post “The Ceiling Effect #2” [Hake (2011a)] I noted that Sheila Tobias (2011), in a recent APS News editorial “Teachers in the Crosshairs. . .”, called attention to the “Value-Added Inequity” (VAI) experienced by a Houston high-school physics teacher who reported that her students entered her course with high test scores. As a consequence her students achieved relatively small pre-to-posttest gains (a consequence of the ceiling effect) and she, in turn, received a relatively small value-added bonus.
EDDRA2’s Michael Martin responded that the Houston teacher’s Value-Added Inequity" (VAI) is “best appreciated as a grossly unsophisticated process being employed by fundamentally incompetent administrators.” But EdResMeth's Tony Milanowski wrote that, as he understood it, the model used in Houston was based on the report “SAS EVAAS Statistical Models” [Wright et al., 2010 at http://bit.ly/hPrO7s] which converts Item-Response-Theory-based scale scores to Normal-Curve-Equivalents - but such sophistication evidently did not prevent the VAI experienced by the Houston high-school physics teacher.
Other VAI's were described by PhysLrnR's Boris Korsunsky and Math-Teach's Haim. Boris wrote (paraphrasing): “My own bonus was zero. Massachusetts used a value added method that took into account a town's average parental income. Since my town is by far the wealthiest in the state, we are often ranked ‘below expectations’ in various state-produced rankings - even though Boston magazine has repeatedly ranked us No.1 in the state. Similarly, Haim wrote (paraphrasing): “The value-added formula for evaluating schools in NYC resulted in Stuyvesant High School (one of the top academic institutions in the U.S.) earning a ‘B’.”
Consistent with the above VAI’s, reports by recognized experts critical of the use of value added measures to grade teachers are: (a) “Letter Report to the U.S. Department of Education on the Race to the Top Fund” [NRC (2009)], (b) “Getting Value Out of Value-Added: Report of a Workshop”[NRC (2010)], (c) “Problems With The Use of Student Test Scores to Evaluate Teachers” [EPI (2010)], (d) “Grading teachers on value-added measures falls short” [UCLA Today (2010)], (e) “Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains” [Schochet & Chiang (2010)], (f) “Hurdles Emerge in Rising Effort to Rate Teachers” [Otterman (2010)].
Despite the above criticism, the Department of Education has designed its “Race to the Top” scoring system to reward states that use value-added calculations in teacher evaluations.
************************************************
To access the complete 41 kB post please click on http://bit.ly/fN1HmD .
Richard Hake, Emeritus Professor of Physics, Indiana University
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands
President, PEdants for Definitive Academic References which
************************************************
ABSTRACT: In a previous post “The Ceiling Effect #2” [Hake (2011a)] I noted that Sheila Tobias (2011), in a recent APS News editorial “Teachers in the Crosshairs. . .”, called attention to the “Value-Added Inequity” (VAI) experienced by a Houston high-school physics teacher who reported that her students entered her course with high test scores. As a consequence her students achieved relatively small pre-to-posttest gains (a consequence of the ceiling effect) and she, in turn, received a relatively small value-added bonus.
EDDRA2’s Michael Martin responded that the Houston teacher’s Value-Added Inequity" (VAI) is “best appreciated as a grossly unsophisticated process being employed by fundamentally incompetent administrators.” But EdResMeth's Tony Milanowski wrote that, as he understood it, the model used in Houston was based on the report “SAS EVAAS Statistical Models” [Wright et al., 2010 at http://bit.ly/hPrO7s] which converts Item-Response-Theory-based scale scores to Normal-Curve-Equivalents - but such sophistication evidently did not prevent the VAI experienced by the Houston high-school physics teacher.
Other VAI's were described by PhysLrnR's Boris Korsunsky and Math-Teach's Haim. Boris wrote (paraphrasing): “My own bonus was zero. Massachusetts used a value added method that took into account a town's average parental income. Since my town is by far the wealthiest in the state, we are often ranked ‘below expectations’ in various state-produced rankings - even though Boston magazine has repeatedly ranked us No.1 in the state. Similarly, Haim wrote (paraphrasing): “The value-added formula for evaluating schools in NYC resulted in Stuyvesant High School (one of the top academic institutions in the U.S.) earning a ‘B’.”
Consistent with the above VAI’s, reports by recognized experts critical of the use of value added measures to grade teachers are: (a) “Letter Report to the U.S. Department of Education on the Race to the Top Fund” [NRC (2009)], (b) “Getting Value Out of Value-Added: Report of a Workshop”[NRC (2010)], (c) “Problems With The Use of Student Test Scores to Evaluate Teachers” [EPI (2010)], (d) “Grading teachers on value-added measures falls short” [UCLA Today (2010)], (e) “Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains” [Schochet & Chiang (2010)], (f) “Hurdles Emerge in Rising Effort to Rate Teachers” [Otterman (2010)].
Despite the above criticism, the Department of Education has designed its “Race to the Top” scoring system to reward states that use value-added calculations in teacher evaluations.
************************************************
To access the complete 41 kB post please click on http://bit.ly/fN1HmD .
Richard Hake, Emeritus Professor of Physics, Indiana University
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands
President, PEdants for Definitive Academic References which
Recognize the Invention of the Internet (PEDARRII)
rrhake@earthlink.net
http://www.physics.indiana.edu/~hake
http://www.physics.indiana.edu/~sdi
http://HakesEdStuff.blogspot.com
http://iub.academia.edu/RichardHake
“The [Race to the Top] initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches (which evaluate teachers based on gains in their students' performance) to reward or punish teachers.”
“Letter Report to the U.S. Dept. of Education on the Race to the Top Fund”
[NRC (2009)]
REFERENCES [URL's shortened by http://bit.ly/ and accessed on 18 Jan 2011.]
Hake, R.R. 2011a. “The Ceiling Effect #2” online on the OPEN! AERA-L archives at http://bit.ly/hUnHZe . Post of 12 Jan 2011 16:19:49-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists are also online on this blog “Hake'sEdStuff” at with a provision for comments.
Hake, R.R. 2011b. “ Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” online on the OPEN! AERA-L archives at http://bit.ly/fN1HmD . Post of 18 Jan 2011 15:34:47-0800to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.
NRC. 2009. National Research Council, Board on Testing and Assessment (chaired by E.H. Haertel), Letter Report to the U.S. Department of Education on the Race to the Top Fund, online at http://bit.ly/dOg8v6 .
http://www.physics.indiana.edu/~hake
http://www.physics.indiana.edu/~sdi
http://HakesEdStuff.blogspot.com
http://iub.academia.edu/RichardHake
“The [Race to the Top] initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches (which evaluate teachers based on gains in their students' performance) to reward or punish teachers.”
“Letter Report to the U.S. Dept. of Education on the Race to the Top Fund”
[NRC (2009)]
REFERENCES [URL's shortened by http://bit.ly/ and accessed on 18 Jan 2011.]
Hake, R.R. 2011a. “The Ceiling Effect #2” online on the OPEN! AERA-L archives at
Hake, R.R. 2011b. “ Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” online on the OPEN! AERA-L archives at http://bit.ly/fN1HmD . Post of 18 Jan 2011 15:34:47-0800to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.
NRC. 2009. National Research Council, Board on Testing and Assessment (chaired by E.H. Haertel), Letter Report to the U.S. Department of Education on the Race to the Top Fund, online at http://bit.ly/dOg8v6 .
Subscribe to:
Posts (Atom)
