Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?

Some blog followers might be interested in a recent post “Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” [Hake (2011b)]. The abstract reads:

************************************************
ABSTRACT: In a previous post “The Ceiling Effect #2” [Hake (2011a)] I noted that Sheila Tobias (2011), in a recent APS News editorial “Teachers in the Crosshairs. . .”, called attention to the “Value-Added Inequity” (VAI) experienced by a Houston high-school physics teacher who reported that her students entered her course with high test scores. As a consequence her students achieved relatively small pre-to-posttest gains (a consequence of the ceiling effect) and she, in turn, received a relatively small value-added bonus.

EDDRA2’s Michael Martin responded that the Houston teacher’s Value-Added Inequity" (VAI) is “best appreciated as a grossly unsophisticated process being employed by fundamentally incompetent administrators.” But EdResMeth's Tony Milanowski wrote that, as he understood it, the model used in Houston was based on the report “SAS EVAAS Statistical Models” [Wright et al., 2010 at http://bit.ly/hPrO7s] which converts Item-Response-Theory-based scale scores to Normal-Curve-Equivalents - but such sophistication evidently did not prevent the VAI experienced by the Houston high-school physics teacher.

Other VAI's were described by PhysLrnR's Boris Korsunsky and Math-Teach's Haim. Boris wrote (paraphrasing): “My own bonus was zero. Massachusetts used a value added method that took into account a town's average parental income. Since my town is by far the wealthiest in the state, we are often ranked ‘below expectations’ in various state-produced rankings - even though Boston magazine has repeatedly ranked us No.1 in the state. Similarly, Haim wrote (paraphrasing): “The value-added formula for evaluating schools in NYC resulted in Stuyvesant High School (one of the top academic institutions in the U.S.) earning a ‘B’.”

Consistent with the above VAI’s, reports by recognized experts critical of the use of value added measures to grade teachers are: (a) “Letter Report to the U.S. Department of Education on the Race to the Top Fund” [NRC (2009)], (b) “Getting Value Out of Value-Added: Report of a Workshop”[NRC (2010)], (c) “Problems With The Use of Student Test Scores to Evaluate Teachers” [EPI (2010)], (d) “Grading teachers on value-added measures falls short” [UCLA Today (2010)], (e) “Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains” [Schochet & Chiang (2010)], (f) “Hurdles Emerge in Rising Effort to Rate Teachers” [Otterman (2010)].

Despite the above criticism, the Department of Education has designed its “Race to the Top” scoring system to reward states that use value-added calculations in teacher evaluations.
************************************************

To access the complete 41 kB post please click on http://bit.ly/fN1HmD .


Richard Hake, Emeritus Professor of Physics, Indiana University
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands
President, PEdants for Definitive Academic References which

Recognize the Invention of the Internet (PEDARRII)

rrhake@earthlink.net
http://www.physics.indiana.edu/~hake
http://www.physics.indiana.edu/~sdi
http://HakesEdStuff.blogspot.com
http://iub.academia.edu/RichardHake

“The [Race to the Top] initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches (which evaluate teachers based on gains in their students' performance) to reward or punish teachers.”
“Letter Report to the U.S. Dept. of Education on the Race to the Top Fund”
[NRC (2009)]


REFERENCES [URL's shortened by http://bit.ly/ and accessed on 18 Jan 2011.]
Hake, R.R. 2011a. “The Ceiling Effect #2” online on the OPEN! AERA-L archives at http://bit.ly/hUnHZe . Post of 12 Jan 2011 16:19:49-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists are also online on this blog “Hake'sEdStuff” at with a provision for comments.

Hake, R.R. 2011b. “ Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” online on the OPEN! AERA-L archives at http://bit.ly/fN1HmD . Post of 18 Jan 2011 15:34:47-0800to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.

NRC. 2009. National Research Council, Board on Testing and Assessment (chaired by E.H. Haertel), Letter Report to the U.S. Department of Education on the Race to the Top Fund, online at http://bit.ly/dOg8v6 .

The Ceiling Effect #2

Some blog followers might be interested in a recent post “The Ceiling Effect #2” [Hake (2011)]. The abstract reads:

******************************************

ABSTRACT: Sheila Tobias (2010) in a recent APS News “Back Page” editorial “Teachers in the Crosshairs. . .”: wrote:

“In Houston, where a new value-added formula is being used to grade teachers’ skills, ‘N.Y. Times’ education writer Sam Dillon found a high school physics teacher out of the running for high bonuses because, as she reported, ‘My kids come in at a very high level of competence, scoring well before the semester begins.’ After she teaches them for a year, they continue to score well on a state science test but show less measurable ‘gain’ than other classes, so her bonus is small compared with those of other teachers. The teacher has invented a term of her own to characterize this distortion. She calls it the ‘ceiling effect.’ We might call it a saturation effect.”

It's unfortunate that the teacher’s bonus assigners were not aware that it’s the “average normalized gain” [g] = ([%post] – [%pre]) / (100% - [%pre]) [the square brackets [. . .] indicate class averages], often used in physics education research, that can (in some cases) be used as a gauge of relative course effectiveness, NOT the absolute average gain [G] = ([%post] – [%pre]).

******************************************

To access the complete 11 kB post please click on http://bit.ly/hUnHZe .

Richard Hake, Emeritus Professor of Physics, Indiana University

Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands

President, PEdants for Definitive Academic References which Recognize the Invention of the Internet (PEDARRII)

rrhake@earthlink.net>

http://www.physics.indiana.edu/~hake

http://www.physics.indiana.edu/~sdi

http://HakesEdStuff.blogspot.com

http://iub.academia.edu/RichardHake

REFERENCES [URL’s shortened by http://bit.ly/ and accessed on 12 Jan 2010.]

Hake, R.R. 2011. “The Ceiling Effect #2” online on the OPEN! AERA-L archives at http://bit.ly/hUnHZe . Post of 12 Jan 2011 16:19:49-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.

Tobias, S. 2010. “Teachers in the Crosshairs: The Impact of ‘School Choice,’ ‘Reform,’ and ‘Accountability’,” APS News 20(1), January; online at http://bit.ly/fc6jEj .

Sustainable Energy Without the Hot Air, FREE Online Books

Some blog followers might be interested in the post “Sustainable Energy Without the Hot Air, FREE Online Books)” [Hake (2010b)]. The abstract reads:

*******************************************

ABSTRACT: Some subscribers may not be aware of the FREE online book “Sustainable Energy - Without the Hot Air” [MacKay (2009)] dedicated to “those who will not have the benefit of two billion years' accumulated energy reserves,” and favorably reviewed by John Roeder (2009) and David Hafemeister (2010). This post:

(a) contains brief excerpts from those reviews;

(b) lists three other free online books that may be of interest: "Teaching Physics With the Physics Suite" (Redish, 2003)], "Science Teaching as a Profession: Why It Isn't How It Could Be" (Tobias & Baffert, 2009), and “Educational Psychology” (Seifert & Sutton, 2009);

(c) quotes from Steven Pearlstein (2010) on trends in publishing and James Koch (2006) on escalating college textbook prices; and

(d) references “Over Two-Hundred Education & Science Blogs” (Hake, 2009b) for references on OPEN ACCESS, internet usage, the Academic Discussion List Sphere (ADLsphere) and the Blogosphere.

*******************************************

To access the complete 13 kB post please click on http://groups.yahoo.com/group/Net-Gold/message/31840.

REFERENCES [Tiny URL's courtesy http://tinyurl.com/create.php .]

Hake, R.R. 2010a. “Sustainable Energy - Without the Hot Air, FREE Online Books,” online on the Physoc archives at http://tinyurl.com/yb6h8ew. Post of 8 Feb 2010 21:43:39-0600 to AP-Physics, Phys-L, Physhare, and PhysLrnR (submitted to the moderator). To access the archives of PHYSOC one needs to subscribe, but that takes only a few minutes by clicking on http://listserv.uark.edu/archives/physoc.html and then clicking on "Join or leave the list (or change settings)." If you're busy, then subscribe using the "NOMAIL" option under "Miscellaneous." Then, as a subscriber, you may access the archives and/or post messages at any time, while receiving NO MAIL from the list!

Hake, R.R. 2010b. “Sustainable Energy Without the Hot Air, FREE Online Books,” online on the OPEN! Net-Gold archives at http://groups.yahoo.com/group/Net-Gold/message/31840. Post of 9 Feb 2010 09:05:16-0800 to AERA-L and Net-Gold. For a preliminary and less refined version of this post see Hake (2010a).

MacKay, D.J.C. 2009. “Sustainable Energy - Without the Hot Air,” online at http://www.withouthotair.com. To download a high-resolution copy of the entire book click on http://www.inference.phy.cam.ac.uk/sustainable/book/tex/sewtha.pdf (49.2 MB) - Version 3.5.2. November 3, 2008. This (as well as the low resolution version) displays color and has the advantage of accommodating MacKay's expectation “to further update some of the numbers in this book as I continue to learn about sustainable energy.”

High-School Science Teaching as a Profession

Some blog followers may be interested in a recent discussion-list post of the above title [Hake (2009)]. The abstract reads:

ABSTRACT: Jane Jackson, in a Physoc post of 1 July 2009, called attention to Tobias & Baffert's (2009) FREE online book Science Teaching as a Profession: Why It Isn't, How It Could Be.

I join Jane in urging teachers to (a) read the Tobias/Baffert book, and (b) work collectively to promote increased professionalism in high-school science teaching, one of the country's most vital occupations.

Biologist James Gentile, in a Huffington Post article at http://tinyurl.com/nh2uo2 provides an excellent review of Tobias & Baffert's (2009) book in the context of “Improving Science Teaching in America's Schools."

To access the complete 9 kB post, please click on https://carnot.physics.buffalo.edu/archives/2009/12_2009/msg00131.html .

REFERENCES

Hake, R.R. 2009. “High-School Science Teaching as a Profession,” online on the OPEN! Phys-L archives at https://carnot.physics.buffalo.edu/archives/2009/12_2009/msg00131.html . Post of 18-19 Dec 2009 to AERA-K, AERA-L, AP-Bio, AP-Chem, AP-Physics, Biopi-L, Chemed-L, Net-Gold, Physhare, Phys-L, PhysLrnR, & Physoc.

Re: Persistent Myths in Feminist Scholarship

Some blog followers may be interested in a recent post of the above title. The abstract reads:

********************************************

ABSTRACT: Stephen Black, in a TIPS (Teaching in the Psychological Sciences) post of 3 July 2009 called attention to Christina Hoff Sommers' (2009) provocative Chronicle article "Persistent Myths in Feminist Scholarship" at http://chronicle.com/free/v55/i40/40sommers.htm .

In the ensuing discussion: (a) Paul Brandon pointed out that, although he respects her work, Hoff Sommers is “slightly right-wing” and paid by the American Enterprise Institute (AEI); (b) Allen Esterson then responded that Hoff Sommers' work ”should be treated on its merits, regardless of whether her socio-political views are right or left of centre!”

I agree with Esterson and point out that, as indicated in Gender Issues in Science/Math Education (GISME) [Hake & Mallow (2008)] and by Esterson, the persistent errors in of some of the feminist literature have been repeatedly pointed out by many non-AEI-sponsored authors from all sectors of the socio-political spectrum, e.g.: Almeder et al. (2003), Esterson (2006), Holton (1993), Koertge (1998), Newton (1997), Patai & Koertge (2003), & Tobias et al. (2002).

********************************************

To access the complete 18 kB post, please click on http://tinyurl.com/qfc6t6 .

REFERENCES

Hake, R.R. & J.V. Mallow. 2008. "Gender Issues in Science/Math Education (GISME)," over 700 Annotated References & 1000 URL's: Part 1 - All references in alphabetical order; Part 2 - Some references in subject order; both online at ref. 55 at http://www.physics.indiana.edu/~hake/, and on this blog at http://hakesedstuff.blogspot.com/2008/11/gender-issues-in-sciencemath-education.html .

Hake, R.R. 2009. “Re: Persistent Myths in Feminist Scholarship,” online on the OPEN! AERA-L archives at http://tinyurl.com/qfc6t6 . Post of 5 Jul 2009 to AERA-L. The abstract only was transmitted to various other discussion lists.

Sommers, C. H. 2009. "Persistent Myths in Feminist Scholarship," Chronicle of Higher Education, 29 June; online at http://chronicle.com/free/v55/i40/40sommers.htm , with five comments - the last by Hake - at http://chronicle.com/forums/index.php/topic,61347.0.html as of 9 July 2009 10:54:00-0700.