How Much Value is Added at Elite Institutions - Response to Haim #2

Some blog followers might be interested in a recent post “How Much Value is Added at Elite Institutions - Response to Haim #2” [Hake (2011d)].

********************************************

ABSTRACT: In my post “How Much Value is Added at Elite Institutions?” [Hake (2011c)], I wrote:

“. . . . demonstrations that the less-than-stellar value-added assessments of Korsunsky's high school and Stuyvesant High School are inequitable would require meaningful value-added measures such as normalized average pre-to-posttest gains on valid and consistently reliable tests of higher-order learning. . .”

Math-Teach's “Haim” responded: “The problem is that you are not sure what you are measuring."

NONSENSE! In the case of Harvard, the higher-order learning consisted of conceptual understanding of Newtonian mechanics.

Haim continued: “First, parents and students seem to know something about Stuyvesant that educator assessments clearly fail to discern. . . . . Second . . . very many of Stuyvesant's students graduate at a very high level (certainly by comparison to most other high school graduates) of academic achievement. . . . the real problem is transparent. It is the ceiling effect. . . .City-wide and state-wide assessments are simply not designed for academic institutions.”

The above has nothing whatsoever to do with the theme of my post: “It is conceivable that if there were ‘Eric Mazurs’ or ‘John Belchers’ at Korsunsky's high-school and the Stuyvesant High School, scenarios similar to that at Harvard and MIT might occur. . . . . [[i.e., realization that students were not learning much from traditional passive-student lecture methods followed by a switch to interactive-engagement pedagogy.]]. . . . , even though all those institutions are regarded as ‘elite.’ ”

********************************************

To access the complete 18 kB post please click on http://bit.ly/gxUOAb .

Richard Hake, Emeritus Professor of Physics, Indiana University

Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands

President, PEdants for Definitive Academic References which Recognize the Invention of the Internet (PEDARRII)

rrhake@earthlink.net

http://www.physics.indiana.edu/~hake

http://www.physics.indiana.edu/~sdi

http://HakesEdStuff.blogspot.com

http://iub.academia.edu/RichardHake

“Above all things we must be aware of what I will call ‘inert ideas’

- that is to say, ideas that are merely received into the mind

without being utilized, or tested, or thrown into fresh combinations.”

Alfred North Whitehead (1929, 1965) in The Aims of Education

REFERENCES [URL's shortened by http://bit.ly/ and accessed on 26 Jan 2011.]

Hake, R.R. 2011a. “The Ceiling Effect #2” online on the OPEN! AERA-L archives at http://bit.ly/hUnHZe. Post of 12 Jan 2011 16:19:49-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists are also online on my blog “Hake'sEdStuff” at http://bit.ly/gLWr7W with a provision for comments.

Hake, R.R. 2011b. “Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” online on the OPEN! AERA-L archives at http://bit.ly/fN1HmD. Post of 18 Jan 2011 15:34:47-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists are also online on my blog “Hake'sEdStuff” at http://bit.ly/h23shQ with a provision for comments.

Hake, R.R. 2011c. “How Much Value is Added at Elite Institutions?” online on the OPEN! AERA-L archives at http://bit.ly/g25OHd. Post of 22 Jan 2011 14:50:14-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists are also online on my blog “Hake'sEdStuff” at http://bit.ly/gnLPjH with a provision for comments.

Hake, R.R. 2011d. “How Much Value is Added at Elite Institutions – Response to Haim #2” online on the OPEN! AERA-L archives at http://bit.ly/gxUOAb. Post of 26 Jan 2011 16:14:36-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists. See also the relevant previous posts Hake (2011a,b,c)].

Whitehead, A.N. 1967. Aims of Education and other essays. Free Press. Amazon.com information at http://amzn.to/fIUbXB. First published in 1929. Note the “Look Inside” feature.

Research on Physics First

Some blog followers might be interested in a recent post “Research on Physics First” [Hake (2011b)].

***************************************

ABSTRACT: In response to my earlier post “Re: Physics First empirical studies” [Hake (2011a)]:

(1) Zahra Sana Hazari wrote: “Check out Sadler & Tai's ‘Science’ report ‘The Two High-School Pillars Supporting College Science’ ” http://bit.ly/elkueZ ;

(2) Boris Korsunsky wrote (paraphrasing): “I've written two articles on Physics First students’ attitudes and expectations for ‘The Physics Teacher’: ‘Physics First? Survey First!’ at http://bit.ly/f0Is4V and another that will appear this Spring.”

Regarding Hazari's reference, I think Sadler & Tai's claim that their research casts doubt on the advisability of the physics/chemistry/biology sequence of the Physics First regime is problematic because their reliance on course grades to measure college student learning has been shown to be invalid by research on physics education and on collegiate education generally as described in Academically Adrift http://bit.ly/hOOK09.

Among recent reports that offer evidence in favor of the Physics First sequence are “Squaring the Circle: A Mathematically Rigorous Physics First” [Goodman & Etkina (2008)] and “Effectiveness of Ninth-Grade Physics in Maine: Conceptual Understanding” [O’Brien & Thompson (2009)].

***************************************

To access the complete 15 kB post please click on http://bit.ly/gJIu4i .

Richard Hake, Emeritus Professor of Physics, Indiana University

Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands

President, PEdants for Definitive Academic References which Recognize theInvention of the Internet (PEDARRII)

rrhake@earthlink.net

http://www.physics.indiana.edu/~hake

http://www.physics.indiana.edu/~sdi

http://HakesEdStuff.blogspot.com

http://iub.academia.edu/RichardHake

“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.”

Wood & Gentile (2003)

REFERENCES [URL's shortened by http://bit.ly/ and accessed on 25 Jan 2011.]

Hake, R.R. 2011a. “Re: Physics First empirical studies,” online on the PhysLrnR archives at http://bit.ly/iefCMr . To access the archives of PhysLnR one needs to subscribe, but that takes only a few minutes by clicking on http://bit.ly/beuikb and then clicking on “Join or leave the list (or change settings).” If you're busy, then subscribe using the “NOMAIL” option under “Miscellaneous.” Then, as a subscriber, you may access the archives and/or post messages at any time, while receiving NO MAIL from the list!

Hake, R.R. 2011b. “Research on Physics First,” online on the OPEN! AERA-L archives at http://bit.ly/gJIu4i. Post of 25 Jan 2011 15:41:08-0800 to AERA-L and Net-Gold. The abstract and link to the complete post were transmitted to various discussion lists.

Wood, W.B., & J.M. Gentile. 2003. “Teaching in a research context,” Science 302: 1510; 28 November; online to subscribers at http://bit.ly/9izfFz . A summary is online to all at http://bit.ly/9qGR6m .

Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?

Some blog followers might be interested in a recent post “Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” [Hake (2011b)]. The abstract reads:

************************************************
ABSTRACT: In a previous post “The Ceiling Effect #2” [Hake (2011a)] I noted that Sheila Tobias (2011), in a recent APS News editorial “Teachers in the Crosshairs. . .”, called attention to the “Value-Added Inequity” (VAI) experienced by a Houston high-school physics teacher who reported that her students entered her course with high test scores. As a consequence her students achieved relatively small pre-to-posttest gains (a consequence of the ceiling effect) and she, in turn, received a relatively small value-added bonus.

EDDRA2’s Michael Martin responded that the Houston teacher’s Value-Added Inequity" (VAI) is “best appreciated as a grossly unsophisticated process being employed by fundamentally incompetent administrators.” But EdResMeth's Tony Milanowski wrote that, as he understood it, the model used in Houston was based on the report “SAS EVAAS Statistical Models” [Wright et al., 2010 at http://bit.ly/hPrO7s] which converts Item-Response-Theory-based scale scores to Normal-Curve-Equivalents - but such sophistication evidently did not prevent the VAI experienced by the Houston high-school physics teacher.

Other VAI's were described by PhysLrnR's Boris Korsunsky and Math-Teach's Haim. Boris wrote (paraphrasing): “My own bonus was zero. Massachusetts used a value added method that took into account a town's average parental income. Since my town is by far the wealthiest in the state, we are often ranked ‘below expectations’ in various state-produced rankings - even though Boston magazine has repeatedly ranked us No.1 in the state. Similarly, Haim wrote (paraphrasing): “The value-added formula for evaluating schools in NYC resulted in Stuyvesant High School (one of the top academic institutions in the U.S.) earning a ‘B’.”

Consistent with the above VAI’s, reports by recognized experts critical of the use of value added measures to grade teachers are: (a) “Letter Report to the U.S. Department of Education on the Race to the Top Fund” [NRC (2009)], (b) “Getting Value Out of Value-Added: Report of a Workshop”[NRC (2010)], (c) “Problems With The Use of Student Test Scores to Evaluate Teachers” [EPI (2010)], (d) “Grading teachers on value-added measures falls short” [UCLA Today (2010)], (e) “Error Rates in Measuring Teacher and School Performance Based on Student Test Score Gains” [Schochet & Chiang (2010)], (f) “Hurdles Emerge in Rising Effort to Rate Teachers” [Otterman (2010)].

Despite the above criticism, the Department of Education has designed its “Race to the Top” scoring system to reward states that use value-added calculations in teacher evaluations.
************************************************

To access the complete 41 kB post please click on http://bit.ly/fN1HmD .


Richard Hake, Emeritus Professor of Physics, Indiana University
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands
President, PEdants for Definitive Academic References which

Recognize the Invention of the Internet (PEDARRII)

rrhake@earthlink.net
http://www.physics.indiana.edu/~hake
http://www.physics.indiana.edu/~sdi
http://HakesEdStuff.blogspot.com
http://iub.academia.edu/RichardHake

“The [Race to the Top] initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches (which evaluate teachers based on gains in their students' performance) to reward or punish teachers.”
“Letter Report to the U.S. Dept. of Education on the Race to the Top Fund”
[NRC (2009)]


REFERENCES [URL's shortened by http://bit.ly/ and accessed on 18 Jan 2011.]
Hake, R.R. 2011a. “The Ceiling Effect #2” online on the OPEN! AERA-L archives at http://bit.ly/hUnHZe . Post of 12 Jan 2011 16:19:49-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists are also online on this blog “Hake'sEdStuff” at with a provision for comments.

Hake, R.R. 2011b. “ Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” online on the OPEN! AERA-L archives at http://bit.ly/fN1HmD . Post of 18 Jan 2011 15:34:47-0800to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.

NRC. 2009. National Research Council, Board on Testing and Assessment (chaired by E.H. Haertel), Letter Report to the U.S. Department of Education on the Race to the Top Fund, online at http://bit.ly/dOg8v6 .