Some blog followers might be interested in a discussion list post “The Defiant Parents: Testing’s Discontents – Response to Hunt” [Hake (2014)]. The abstract reads:
***************************************************
ABSTRACT: In a post “Re: The Defiant Parents: Testing's Discontents” [Hake (2014)] at http://bit.ly/1mYwWoa, I pointed to the “vigorous leadership, voluminous messaging, and pro-public-/anti-private-education positions of (a) Diane Ravitch and (b) FairTest http://www.fairtest.org/. Then I commented that neither appeared to be informed regarding the virtues of rigorous measurement of students' higher-order learning by means of zero-stakes formative evaluation “designed and used to improve an intervention, especially when it is still being developed” [JCSEE, copied onto p. 132 by Frechtling et al. (2010) at http://bit.ly/1aYcgYn.
In response, Russ Hunt (2014) at http://bit.ly/1hSsq6L wrote: “The virtues of rigorous testing aren't really the point: it's how the tests are administered and what uses they’re put to that Ravitch and Fair Test (and I) are concerned with.”
However, it IS to the point for many of those who wish to enhance students’ higher-level learning, as I have emphasized in, e.g.:
1. “Lessons from the Physics Education Reform Effort” [Hake (2002)] at http://bit.ly/aL87VT;
2. “The Physics Education Reform Effort: A Possible Model for Higher Education” [Hake (2005)] at http://bit.ly/9aicfh;
3. “Should We Measure Change? Yes!” [Hake (2007a)] at http://bit.ly/d6WVKO (2.5 MB);
4. “Re: pre-to-post tests as measures of learning/teaching” [Hake (2008a)] at http://bit.ly/MmPxwp;
5. “Design-Based Research in Physics Education Research: A Review” [Hake (2008b)] at http://bit.ly/9kORMZ (1.1 MB);
6. “The Impact of Concept Inventories On Physics Education and Its Relevance For Engineering Education” [Hake (2011a)] at http://bit.ly/nmPY8F (8.7 MB);
7. “SET's Are Not Valid Gauges of Students' Higher-Level Learning #2” [Hake (2011b)] at http://bit.ly/jLZaz5;
8. “The NRC Finally Comes to Its Senses on Improving STEM Education” [Hake (2013a)] at http://bit.ly/154M5yf; and
9. “Can the Cognitive Impact of Calculus Courses be Enhanced?” Hake (2013b)] at http://bit.ly/1loHgC4 (2.7 MB).
***************************************************
To access the complete 66 kB post please click on http://bit.ly/1lqiR4u.
Richard Hake, Emeritus Professor of Physics, Indiana University; Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands; President, PEdants for Definitive Academic References which Recognize the Invention of the Internet (PEDARRII); LINKS TO: Academia http://bit.ly/a8ixxm; Articles http://bit.ly/a6M5y0; Blog http://bit.ly/9yGsXh; Facebook http://on.fb.me/XI7EKm; GooglePlus http://bit.ly/KwZ6mE; Google Scholar http://bit.ly/Wz2FP3; Linked In http://linkd.in/14uycpW; Research Gate http://bit.ly/1fJiSwB; Socratic Dialogue Inducing (SDI) Labs http://bit.ly/9nGd3M; Twitter http://bit.ly/juvd52.
“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.” - Wood & Gentile (2003).
REFERENCES [URLs shortened by http://bit.ly/ and accessed on 31 Jan 2014.]
Hake, R.R. 2014. “The Defiant Parents: Testing's Discontents – Response to Hunt,” online on the OPEN! AERA-L archives at http://bit.ly/1lqiR4u. The abstract and link to the complete post are being transmitted to several discussion lists.
Wood, W.B., & J.M. Gentile. 2003. “Teaching in a research context,” Science 302: 1510; 28 November; online as a 213 kB pdf http://bit.ly/SyhOvL, thanks to Ecoplexity http://bit.ly/152aFQ9.
Showing posts with label SETs. Show all posts
Showing posts with label SETs. Show all posts
Friday, January 31, 2014
Thursday, October 31, 2013
Why Do Colleges Tie Academic Careers To Winning the Approval of Teenagers?
Some blog followers might be interested in a recent post “Why Do Colleges Tie Academic Careers To Winning the Approval of Teenagers?” [Hake (2013)]. The abstract reads:
**********************************************
ABSTRACT: An insightful critique of the misuse of Student Evaluations of Teaching (SETs) for the evaluation of faculty appeared in the WSJ of 27 Oct 2013 as a piece “When Students Rate Teachers, Standards Drop: Why do colleges tie academic careers to winning the approval of teenagers? Something is seriously amiss” [Asher (2013)] at http://on.wsj.com/17te3oN and copied into the APPENDIX of this post in accord with the Fair Use provision of U.S. Copyright Law.
Coincidentally, on 29 Oct 2013, I received an email from an assistant professor “X” who fears that he will be denied tenure because the Chair of his department, winner of numerous teaching awards [probably based solely on superior student teaching evaluations (SETs)] thinks X’s SETs are inferior and therefore that X is an inferior teacher. I surmise that since the Chair’s superior SETs, taken at face value, indicate that he, himself, is a superior teacher, a judgment with which he doubtless concurs, the Chair is prone to regard SETs as definitive evidence of teaching effectiveness. But the references in this post indicate that (a) teaching awards do not necessarily equate with teaching effectiveness; (b) SETs are not valid gauges of the cognitive (as opposed to the affective) impact of courses; and (c) SETs should not be used to evaluate faculty.
**********************************************
To access the complete 78 kB post please click on http://bit.ly/1bHiHwp.
Richard Hake, Emeritus Professor of Physics, Indiana University. LINKS TO: Academia http://bit.ly/a8ixxm; Articles http://bit.ly/a6M5y0; Blog http://bit.ly/9yGsXh; Facebook http://on.fb.me/XI7EKm; GooglePlus http://bit.ly/KwZ6mE; Google Scholar http://bit.ly/Wz2FP3; Linked In http://linkd.in/14uycpW; Research Gate http://bit.ly/1fJiSwB; Socratic Dialogue Inducing (SDI) Labs http://bit.ly/9nGd3M; Twitter http://bit.ly/juvd52.
“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.” - Wood & Gentile (2003) at http://bit.ly/SyhOvL .
REFERENCES [URL shortened by http://bit.ly/ and accessed on 31 Oct. 2013.]
Hake, R.R. 2013. “Why Do Colleges Tie Academic Careers To Winning the Approval of Teenagers?” online on the OPEN! AERA-L archives http://bit.ly/1bHiHwp. Post 31 Oct 2013 18:56:04-0400 to AERA-L and NetGold. The abstract and link to the complete post are being transmitted to various discussion lists.
**********************************************
ABSTRACT: An insightful critique of the misuse of Student Evaluations of Teaching (SETs) for the evaluation of faculty appeared in the WSJ of 27 Oct 2013 as a piece “When Students Rate Teachers, Standards Drop: Why do colleges tie academic careers to winning the approval of teenagers? Something is seriously amiss” [Asher (2013)] at http://on.wsj.com/17te3oN and copied into the APPENDIX of this post in accord with the Fair Use provision of U.S. Copyright Law.
Coincidentally, on 29 Oct 2013, I received an email from an assistant professor “X” who fears that he will be denied tenure because the Chair of his department, winner of numerous teaching awards [probably based solely on superior student teaching evaluations (SETs)] thinks X’s SETs are inferior and therefore that X is an inferior teacher. I surmise that since the Chair’s superior SETs, taken at face value, indicate that he, himself, is a superior teacher, a judgment with which he doubtless concurs, the Chair is prone to regard SETs as definitive evidence of teaching effectiveness. But the references in this post indicate that (a) teaching awards do not necessarily equate with teaching effectiveness; (b) SETs are not valid gauges of the cognitive (as opposed to the affective) impact of courses; and (c) SETs should not be used to evaluate faculty.
**********************************************
To access the complete 78 kB post please click on http://bit.ly/1bHiHwp.
Richard Hake, Emeritus Professor of Physics, Indiana University. LINKS TO: Academia http://bit.ly/a8ixxm; Articles http://bit.ly/a6M5y0; Blog http://bit.ly/9yGsXh; Facebook http://on.fb.me/XI7EKm; GooglePlus http://bit.ly/KwZ6mE; Google Scholar http://bit.ly/Wz2FP3; Linked In http://linkd.in/14uycpW; Research Gate http://bit.ly/1fJiSwB; Socratic Dialogue Inducing (SDI) Labs http://bit.ly/9nGd3M; Twitter http://bit.ly/juvd52.
“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.” - Wood & Gentile (2003) at http://bit.ly/SyhOvL .
REFERENCES [URL shortened by http://bit.ly/ and accessed on 31 Oct. 2013.]
Hake, R.R. 2013. “Why Do Colleges Tie Academic Careers To Winning the Approval of Teenagers?” online on the OPEN! AERA-L archives http://bit.ly/1bHiHwp. Post 31 Oct 2013 18:56:04-0400 to AERA-L and NetGold. The abstract and link to the complete post are being transmitted to various discussion lists.
Friday, June 21, 2013
Re: SETs under attack again. . .
Some blog followers might be interested in a recent post “Re: SETs under attack again. . .” [Hake (2013)]. The abstract reads:
**************************************
ABSTRACT: POD’s Nira Hativa in her post “SETs under attack again. . .” at http://bit.ly/11GDlpH wrote (paraphrasing): “How would you respond to the recent article in Psychology Today: “Do the Best Professors Get the Worst Ratings? Do students give low ratings to teachers who instill deep learning?” by psychologist Nate Kornell (2013)] at http://bit.ly/190OXdi.
On the basis of: (a) “Appearances Can Be Deceiving: Instructor Fluency Increases Perceptions of Learning Without Increasing Actual Learning” [Carpenter et al. (2013)] at http://bit.ly/18Ruqvl, and (b) “Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors” [Carrell & West (2010)] at http://bit.ly/KtOnHp, Kornell concluded that “Student Evaluations Are of Questionable Value.”
Kornell's conclusion is consistent with Linda Nilson’s (2013a) post at http://bit.ly/1asQQRL, stating that “Carrell and West’s article is just one of several recent studies showing that student ratings are no longer positively related to student learning [as found by Cohen, 1981 at http://bit.ly/11TYGjV].”
NO LONGER? In “Re: Problems with Student Evaluations: Is Assessment the Remedy?” [Hake (2002a)] at http://bit.ly/hjt9ll I wrote (paraphrasing): “Neither Cohen nor any other SET champion has countered the fatal objection of McKeachie (1987) at http://bit.ly/18Ou9t6 that the evidence for the validity of SETs as gauges of the cognitive impact of courses rests for the most part on measures of students' lower-level thinking as exhibited in course grades or exams.”
Nevertheless, the tired debate on SETs continues unabated. A search of the archives of AERA-D, ASSESS, EvalTalk, Phys-L, PhysLrnR, POD, & STLHE-L for “student evaluations” yielded hit totals of 762 on 5 April 2002 and 1987 on 17 June 2013. Has the tenor of the debate changed over those 11 years? POD’s Ed Nuhfer (2013) at http://bit.ly/15lk3IC thinks so, pointing out that his 1990 opinion “SETs Are Direct Measures of Student Satisfaction”, once regarded as heretical, now appears to be mainstream.
Hativa's post stimulated (at last count) 112 POD posts, none of them mentioning pre/post testing with Concept Inventories http://bit.ly/dARkDY (developed through arduous qualitative and quantitative research by disciplinary experts), as a way to gauge students’ higher-order learning, despite the fact that such has been occurring in physics education for almost 3 decades. In this post I reference 19 failed attempts to inform academia that “Sets Are Not Valid Measures of Students' Higher-Order Learning.”
**************************************
To access the complete 69 kB post please click on http://yhoo.it/19WKjA5 .
Richard Hake, Emeritus Professor of Physics, Indiana University
Links to Articles: http://bit.ly/a6M5y0
Links to Socratic Dialogue Inducing (SDI) Labs: http://bit.ly/9nGd3M
Academia: http://bit.ly/a8ixxm
Blog: http://bit.ly/9yGsXh
GooglePlus: http://bit.ly/KwZ6mE
Google Scholar: http://bit.ly/Wz2FP3
Twitter: http://bit.ly/juvd52
Facebook: http://on.fb.me/XI7EKm
LinkedIn: http://linkd.in/14uycpW
“The defenders of the SET process are generally found in the colleges of education, in the national teachers' unions, and among those who consult in the area. Their positive attitude toward SETs is compatible with a holistic environment that consists of positive research findings, currently accepted educational philosophy, and a communication system largely centered within their own academic disciplines. They are confident enough in their positive conclusions to dismiss negative findings as ‘myths’ (Aleamoni, 1999; Marsh & Roche, 2000) and to wonder why negative comments continue to be found in the literature (Theall & Franklin, 2001). Because instructional-related research is the province of their occupation, it would be expected that the majority of the research on SETs would come from those in educational disciplines.”
- Dennis Clayson (2009)
“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.”
- William Wood & James Gentile (2003)
REFERENCES [URL shortened by http://bit.ly/ and accessed on 21 June 2013.]
Hake, R.R. 2013. “Re: SETs under attack again. . .,” online on the OPEN! Net-Gold archives at http://yhoo.it/19WKjA5. Post of 20 Jun 2013 15:41:08-0700 to AERA-L and Net-Gold. The abstract and link to the complete post are being distributed to various discussion lists.
**************************************
ABSTRACT: POD’s Nira Hativa in her post “SETs under attack again. . .” at http://bit.ly/11GDlpH wrote (paraphrasing): “How would you respond to the recent article in Psychology Today: “Do the Best Professors Get the Worst Ratings? Do students give low ratings to teachers who instill deep learning?” by psychologist Nate Kornell (2013)] at http://bit.ly/190OXdi.
On the basis of: (a) “Appearances Can Be Deceiving: Instructor Fluency Increases Perceptions of Learning Without Increasing Actual Learning” [Carpenter et al. (2013)] at http://bit.ly/18Ruqvl, and (b) “Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors” [Carrell & West (2010)] at http://bit.ly/KtOnHp, Kornell concluded that “Student Evaluations Are of Questionable Value.”
Kornell's conclusion is consistent with Linda Nilson’s (2013a) post at http://bit.ly/1asQQRL, stating that “Carrell and West’s article is just one of several recent studies showing that student ratings are no longer positively related to student learning [as found by Cohen, 1981 at http://bit.ly/11TYGjV].”
NO LONGER? In “Re: Problems with Student Evaluations: Is Assessment the Remedy?” [Hake (2002a)] at http://bit.ly/hjt9ll I wrote (paraphrasing): “Neither Cohen nor any other SET champion has countered the fatal objection of McKeachie (1987) at http://bit.ly/18Ou9t6 that the evidence for the validity of SETs as gauges of the cognitive impact of courses rests for the most part on measures of students' lower-level thinking as exhibited in course grades or exams.”
Nevertheless, the tired debate on SETs continues unabated. A search of the archives of AERA-D, ASSESS, EvalTalk, Phys-L, PhysLrnR, POD, & STLHE-L for “student evaluations” yielded hit totals of 762 on 5 April 2002 and 1987 on 17 June 2013. Has the tenor of the debate changed over those 11 years? POD’s Ed Nuhfer (2013) at http://bit.ly/15lk3IC thinks so, pointing out that his 1990 opinion “SETs Are Direct Measures of Student Satisfaction”, once regarded as heretical, now appears to be mainstream.
Hativa's post stimulated (at last count) 112 POD posts, none of them mentioning pre/post testing with Concept Inventories http://bit.ly/dARkDY (developed through arduous qualitative and quantitative research by disciplinary experts), as a way to gauge students’ higher-order learning, despite the fact that such has been occurring in physics education for almost 3 decades. In this post I reference 19 failed attempts to inform academia that “Sets Are Not Valid Measures of Students' Higher-Order Learning.”
**************************************
To access the complete 69 kB post please click on http://yhoo.it/19WKjA5 .
Richard Hake, Emeritus Professor of Physics, Indiana University
Links to Articles: http://bit.ly/a6M5y0
Links to Socratic Dialogue Inducing (SDI) Labs: http://bit.ly/9nGd3M
Academia: http://bit.ly/a8ixxm
Blog: http://bit.ly/9yGsXh
GooglePlus: http://bit.ly/KwZ6mE
Google Scholar: http://bit.ly/Wz2FP3
Twitter: http://bit.ly/juvd52
Facebook: http://on.fb.me/XI7EKm
LinkedIn: http://linkd.in/14uycpW
“The defenders of the SET process are generally found in the colleges of education, in the national teachers' unions, and among those who consult in the area. Their positive attitude toward SETs is compatible with a holistic environment that consists of positive research findings, currently accepted educational philosophy, and a communication system largely centered within their own academic disciplines. They are confident enough in their positive conclusions to dismiss negative findings as ‘myths’ (Aleamoni, 1999; Marsh & Roche, 2000) and to wonder why negative comments continue to be found in the literature (Theall & Franklin, 2001). Because instructional-related research is the province of their occupation, it would be expected that the majority of the research on SETs would come from those in educational disciplines.”
- Dennis Clayson (2009)
“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.”
- William Wood & James Gentile (2003)
REFERENCES [URL shortened by http://bit.ly/ and accessed on 21 June 2013.]
Hake, R.R. 2013. “Re: SETs under attack again. . .,” online on the OPEN! Net-Gold archives at http://yhoo.it/19WKjA5. Post of 20 Jun 2013 15:41:08-0700 to AERA-L and Net-Gold. The abstract and link to the complete post are being distributed to various discussion lists.
Subscribe to:
Posts (Atom)
