Is the ‘Teacher Effect’ the Dominant Factor in Students’ Academic Gain?

Some blog followers might be interested in discussion-list post “Is the ‘Teacher Effect’ the Dominant Factor in Students’ Academic Gain?” [Hake (2011)].

The abstract reads:

**************************************

ABSTRACT: In his PhysLrnR post Rob Spencer pointed out that Sanders & Rivers (1996) in “Cumulative and Residual Effects of Teachers on Future Student Academic Achievement” concluded that “Within grade levels, the single most dominant factor affecting student academic gain is teacher effect.” This post considers four items relevant to Spencer's post:

I . VALUE-ADDED ASSESSMENT (VAA) criticized earlier in (a) “First, Let's Fire All the Teachers!” [Hake (2010a,b)], and (b) “Value-Added Inequities: Should Value-Added Measures Be Used to Evaluate Teachers?” [Hake (2011a,b)]; and in this post by reference to (c) “Using Student Progress To Evaluate Teachers: A Primer on Value-Added Models” [Braun (2005) http://bit.ly/gZiJdH], (d) “Challenges of Value-Added Assessment” [Doran & Fleischman (2005) http://bit.ly/dF6CuB], (e) “Problems with the use of student test scores to evaluate teachers” [Baker, Barton, Darling-Hammond, Haertel, Ladd, Linn, Ravitch, Rothstein, Shavelson, & Shepard (2010) http://bit.ly/h8k5Fw], (f) The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education [Ravitch (2010) http://amzn.to/faJ9JZ].

II. SOCIOECONOMIC STATUS (SES) stressed by Bernard Ricca, who pointed to Toni Feder's 2009 Physics Today report on Marder & Bansal (2009). Their statistical-mechanics study of “Flow and diffusion of high-stakes test scores” http://bit.ly/hYbbLe indicated that: Poverty is a more powerful influence on test scores than value added by teachers and schools. The case that poverty is an overriding influence on students' classroom achievement has been forcefully argued by David Berliner in “Our Impoverished View of Educational Reform” [Berliner (2009) <http://bit.ly/ff8BVj>], and “Poverty and Potential: Out-of-School Factors and School Success” [Berliner (2010) <http://bit.ly/fqiCUA>].

III. LESSONS FROM THE PHYSICS EDUCATION REFORM EFFORT suggest that: (a) high school, college, and university courses employing “Interactive Engagement" (IE) methods result in average normalized gains g(ave) in conceptual understanding that are about two standard deviations greater the g(ave) achieved by traditional (T) passive-student lecture courses [Hake (1998a,b; 2008); (b) “Teachers who possess both content knowledge and ‘pedagogical content knowledge’ are better equipped to deliver IE instruction” [Hake (2002)]; and (c) paraphrasing Wells et al. (1995): “teacher expertise is the critical factor in improving introductory physics instruction.” I suspect that the growing body of educational research in disciplines other than physics will eventually show that these same lessons apply to most other academic subjects.

IV. IS THE 'TEACHER EFFECT' THE DOMINANT FACTOR IN STUDENTS' ACADEMIC GAIN?" Judging from the all above:

(a) if “academic gain” means “gain in higher-level learning for U.S. K-12 students,” then the answer is: “NO! POVERTY is the dominant factor” - see ‘I’ and ‘II’ above, and the next most important factor is the degree to which ‘Interactive Engagement’ is provided by courses and teachers - see III above”;

(b) if “academic gain” means “gain in higher-level learning for students in higher education” then the answer is: “The dominant factor in promoting academic gain is the degree to which ‘Interactive Engagement’ in provided by courses and teachers - see III above.”

**************************************

To access the complete 69 kB post please click on http://bit.ly/g6UWUZ.

Richard Hake, Emeritus Professor of Physics, Indiana University

Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands

President, PEdants for Definitive Academic References which Recognize the

Invention of the Internet (PEDARRII)

rrhake@earthlink.net

http://www.physics.indiana.edu/~hake

http://www.physics.indiana.edu/~sdi

http://HakesEdStuff.blogspot.com

http://iub.academia.edu/RichardHake

“The [Race to the Top] initiative should support research based on data that links student test scores with their teachers, but should not prematurely promote the use of value-added approaches (which evaluate teachers based on gains in their students' performance) to reward or punish teachers.”

“Letter Report to the U.S. Dept. of Education on the Race to the Top Fund” [NRC (2009)]

REFERENCES [All URL’s accessed on 08 April 2011 and shortened by http://bit.ly/.]

Hake, R.R. 2011. “Is the ‘Teacher Effect’ the Dominant Factor in Students’ Academic Gain?” online on the OPEN! AERA-L archives at http://bit.ly/g6UWUZ. Post of 7 Apr 2011 17:51:59-0700 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.

NRC. 2009. National Research Council, Board on Testing and Assessment (chaired by E.H. Haertel), “Letter Report to the U.S. Department of Education on the Race to the Top Fund,” online at http://bit.ly/dOg8v6.

Learning Outcomes: Face-to-Face vs Online #3

Some blog followers might be interested in a recent post “Learning Outcomes: Face-to-Face vs Online #3” [Hake (2011)]. The abstract reads:

*************************************

ABSTRACT: In an earlier post, “Learning Outcomes: Face-to-Face vs Online,” I responded to a question posed by STLHE-L’s Martin Rosenzweig: “Does anyone know of any published studies comparing online to face-to-face instruction with regards to learning outcomes?” I wrote: “As far as I know the answer is ‘NO.’ The reason is [as pointed out in ‘Can Distance and Classroom Learning Be Increased?’ (Hake, 2008a)] ‘scholars of teaching and learning continue to rely on low-resolution gauges of students’ learning.’ ”

In response, several discussion-list subscribers called attention to “Evaluation of Evidence-Based Practices in Online Learning: A Meta-Analysis and Review of Online Learning Studies” [USDE (2009)] at http://bit.ly/e1VXvA, of which I had been unaware.

In my opinion, the USDE (2009) study is yet another example of reliance on low-resolution gauges of students’ learning. On pages 11-12 it is stated that examples of learning outcome measures included: (a) scores on standardized tests, (b) scores on researcher-created assessments, (c) grades/scores on teacher-created assessments (e.g., assignments, midterm/final exams), and (d) grades or grade point averages.

But among lessons of the physics education research effort [Hake (2002)] are that: (1) “c” and “d” are *invalid* measures of students' *higher-order* learning; and (2) analyses of “a” and “b” are best carried out in terms of the course average *normalized* gain [g], ignored in USDE (2009).

Furthermore, on page 18 of http://bit.ly/e1VXvA the USDE report states: “The mean effect size for all 50 contrasts of online vs face-to-face instruction was +0.20.”

Contrast the above with the effect size d = +2.43 for the superiority of [[g]] (the average of the course average [g]) for 48 face-to-face “interactive engagement” physics courses vs 14 face-to-face “traditional”introductory physics courses [Hake (1998a,b; 2002; 2008b)].

I suspect that similar large effect sizes would be found for the superiority of online “interactive engagement” courses vs online “traditional” courses.

In my opinion it makes little sense to meta-analyze online vs face-to-face instruction without taking into account the relatively large effects on higher-order learning of “interactive-engagement” vs “traditional” instruction.

*************************************

To access the complete 20 kB post please click on http://bit.ly/egC8I3.

Richard Hake, Emeritus Professor of Physics, Indiana University

Honorary Member, Curmudgeon Lodge of Deventer, The

Netherlands

President, PEdants for Definitive Academic References which Recognize the

Invention of the Internet (PEDARRII)

rrhake@earthlink.net>

http://www.physics.indiana.edu/~hake

http://www.physics.indiana.edu/~sdi

http://HakesEdStuff.blogspot.com

http://iub.academia.edu/RichardHake

“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively incourses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.”

Wood & Gentile (2003)

REFERENCES [URL’s shortened by http://bit.ly/ and accessed on 05 Jan 2010.

Hake, R.R. 2002. “Lessons from the physics education reform effort,” Ecology and Society 5(2): 28; online at http://bit.ly/aL87VT.

Hake, R.R. 2008a. “Can Distance and Classroom Learning Be Increased?” IJ-SoTL 2(1): January; online at http://bit.ly/98dL0Y.

Hake, R.R. 2011. “Learning Outcomes: Face-to-Face vs Online #3,” online on the OPEN! AERA-L archives at http://bit.ly/egC8I3. Post of 5 Jan 2011 13:54:14 -0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.

Wood, W.B. & J.M. Gentile. 2003. “Teaching in a research context,” Science 302: 1510; 28 November; online to subscribers at http://bit.ly/9izfFz . A summary is online for all at http://bit.ly/9qGR6m.

Seventeen Statements by Gold-Standard Skeptics #2

Some blog followers might be interested in a post of the above title. The abstract reads:


***************************************
ABSTRACT: Andy Rudd in an EdResMeth post 6 Apr 2010 titled "Cause and Effect" wrote: "Today I dealt with a doctoral student who was adamantly opposed to the idea that causal relationships can be studied using non experimental designs. . . . . . I am curious what others think about the use of non experimental designs to study causal relationships if it is not possible to use an experimental or quasi-experimental design."


This initiated an 18-post thread of diverse comments on the student's opinion, accessible at http://tinyurl.com/y4um3g3 . Rudd's student may have been influenced by the fact that the "Randomized Control Trial" has been enthroned by the U.S. Dept. of Education (USDE, 2008) and Mosteller and Boruch (2002) as the "gold standard" for demonstrating causality in education research.


For consideration by Rudd's student and others, herewith are SEVENTEEN STATEMENTS BY GOLD-STANDARD SKEPTICS:

(1) American Education Research Association;

(2) American Evaluation Association;

(3) Hugh Burkhardt and Alan Schoenfeld;

(4) Tom Cook and Monique Payne;

(5) Margaret Eisenhart and Lisa Towne;

(6) European Evaluation Society;

(7) Richard Hake;

(8) Burke Johnson;

(9) Annette Lareau and Pamela Barnhouse;

(10) Joseph Maxwell;

(11) National Education Association;

(12) Dennis Phillips;

(13) Barbara Schneider, Martin Carnoy, Jeremy Kilpatrick, William Schmidt, and Richard Shavelson;

(14) Michael Scriven;

(15) Mack Shelley, Larry Yore, and Brian Hand;

(16) Deborah Stipek; and

(17) Carol Weiss.
***************************************



To access the complete 85 kB post please click on http://tinyurl.com/3az7tvu .



Richard Hake, Emeritus Professor of Physics, Indiana University

Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands

President, PEdants for Definitive Academic References which Recognize the Invention of the Internet (PEDARRII)

rrhake@earthlink.net
http://www.physics.indiana.edu/~hake
http://www.physics.indiana.edu/~sdi
http://HakesEdStuff.blogspot.com
http://iub.academia.edu/RichardHake



REFERENCES [Tiny URL's courtesy http://tinyurl.com/create.php .]


Hake, R.R. 2010. "Seventeen Statements by Gold-Standard Skeptics," online on the OPEN! AERA-L archives at http://tinyurl.com/3az7tvu . Post of 27 Apr 2010 10:03:58-0700 to AERA-L and Net-Gold. The abstract and link to the complete post are also being transmitted to various discussion lists.