Re: SETs under attack again. . .

Some blog followers might be interested in a recent post “Re: SETs under attack again. . .” [Hake (2013)]. The abstract reads:

**************************************
ABSTRACT: POD’s Nira Hativa in her post “SETs under attack again. . .” at http://bit.ly/11GDlpH wrote (paraphrasing): “How would you respond to the recent article in Psychology Today: “Do the Best Professors Get the Worst Ratings? Do students give low ratings to teachers who instill deep learning?” by psychologist Nate Kornell (2013)] at http://bit.ly/190OXdi.

On the basis of: (a) “Appearances Can Be Deceiving: Instructor Fluency Increases Perceptions of Learning Without Increasing Actual Learning” [Carpenter et al. (2013)] at http://bit.ly/18Ruqvl, and (b) “Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors” [Carrell & West (2010)] at http://bit.ly/KtOnHp, Kornell concluded that “Student Evaluations Are of Questionable Value.”

Kornell's conclusion is consistent with Linda Nilson’s (2013a) post at http://bit.ly/1asQQRL, stating that “Carrell and West’s article is just one of several recent studies showing that student ratings are no longer positively related to student learning [as found by Cohen, 1981 at http://bit.ly/11TYGjV].”

NO LONGER? In “Re: Problems with Student Evaluations: Is Assessment the Remedy?” [Hake (2002a)] at http://bit.ly/hjt9ll I wrote (paraphrasing): “Neither Cohen nor any other SET champion has countered the fatal objection of McKeachie (1987) at http://bit.ly/18Ou9t6 that the evidence for the validity of SETs as gauges of the cognitive impact of courses rests for the most part on measures of students' lower-level thinking as exhibited in course grades or exams.”

Nevertheless, the tired debate on SETs continues unabated. A search of the archives of AERA-D, ASSESS, EvalTalk, Phys-L, PhysLrnR, POD, & STLHE-L for “student evaluations” yielded hit totals of 762 on 5 April 2002 and 1987 on 17 June 2013. Has the tenor of the debate changed over those 11 years? POD’s Ed Nuhfer (2013) at http://bit.ly/15lk3IC thinks so, pointing out that his 1990 opinion “SETs Are Direct Measures of Student Satisfaction”, once regarded as heretical, now appears to be mainstream.

Hativa's post stimulated (at last count) 112 POD posts, none of them mentioning pre/post testing with Concept Inventories http://bit.ly/dARkDY (developed through arduous qualitative and quantitative research by disciplinary experts), as a way to gauge students’ higher-order learning, despite the fact that such has been occurring in physics education for almost 3 decades. In this post I reference 19 failed attempts to inform academia that “Sets Are Not Valid Measures of Students' Higher-Order Learning.”
**************************************

To access the complete 69 kB post please click on http://yhoo.it/19WKjA5 .

Richard Hake, Emeritus Professor of Physics, Indiana University
Links to Articles: http://bit.ly/a6M5y0
Links to Socratic Dialogue Inducing (SDI) Labs: http://bit.ly/9nGd3M
Academia: http://bit.ly/a8ixxm
Blog: http://bit.ly/9yGsXh
GooglePlus: http://bit.ly/KwZ6mE
Google Scholar: http://bit.ly/Wz2FP3
Twitter: http://bit.ly/juvd52
Facebook: http://on.fb.me/XI7EKm
LinkedIn: http://linkd.in/14uycpW

“The defenders of the SET process are generally found in the colleges of education, in the national teachers' unions, and among those who consult in the area. Their positive attitude toward SETs is compatible with a holistic environment that consists of positive research findings, currently accepted educational philosophy, and a communication system largely centered within their own academic disciplines. They are confident enough in their positive conclusions to dismiss negative findings as ‘myths’ (Aleamoni, 1999; Marsh & Roche, 2000) and to wonder why negative comments continue to be found in the literature (Theall & Franklin, 2001). Because instructional-related research is the province of their occupation, it would be expected that the majority of the research on SETs would come from those in educational disciplines.”
- Dennis Clayson (2009)

“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.”
- William Wood & James Gentile (2003)

REFERENCES [URL shortened by http://bit.ly/ and accessed on 21 June 2013.]
Hake, R.R. 2013. “Re: SETs under attack again. . .,” online on the OPEN! Net-Gold archives at http://yhoo.it/19WKjA5. Post of 20 Jun 2013 15:41:08-0700 to AERA-L and Net-Gold. The abstract and link to the complete post are being distributed to various discussion lists.