Two Different Meanings of “Formative Evaluation” #2

Some blog followers might be interested in a discussion list post “Two Different Meanings of ‘Formative Evaluation’ #2” [Hake (2014)]. The abstract reads:

*******************************************************
ABSTRACT: In response to my post “FairTest Appears To Be Uninformed on ‘Formative Evaluation’ In The JCSEE Sense” at http://bit.ly/1dL3c5K, Michael Paul Goldenberg and FairTest’s Monty Neil posted what appeared to be non sequitures on EDDRA2, symptomatic of the general failure of educators to recognize the existence of two very different meanings of “Formative Evaluation”:

(1) “Evaluation designed and used to improve
an intervention, especially when it is still being
developed” – JCSEE (1994) as copied on p. 132
of Frechtling et al. (2010) at http://bit.ly/1aYcgYn. . .(FE-JCSEE)

(2) “All those activities undertaken to provide
information to be used as feedback so as to adapt
the teaching to meet student needs” – paraphrased
from Black & Wiliam (1998, p. 2)
at http://bit.ly/1jTqiwK.. . . . . . . . . . . . . . . . . . . . (FE-B&W)

An example of FE-JCSEE is zero-stakes pre/post testing utilizing Concept Inventories http://bit.ly/dARkDY which are constructed by disciplinary experts through arduous qualitative and quantitative research– see e.g.: (a) “The Impact of Concept Inventories on Physics Education and Its Relevance for Engineering Education” [Hake (2011)]at http://bit.ly/nmPY8F (8.7 MB), and (b) “Can the Cognitive Impact of Calculus Courses be Enhanced?” [Hake (2013)] at http://bit.ly/1loHgC4 (2.7 MB).

An example of FE-B&W is its use in the “interactive engagement” (IE) methods that have been shown –Hake (1998a) at http://bit.ly/9484DG and many others to achieve average normalized gains g(ave) on Concept Inventories that are about two standard deviations above those of traditional passive-student lecture courses. Here IE methods are defined [Hake 1998a)] as: “methods designed to promote conceptual understanding through the active engagement of students in heads-on (always) and hands-on (usually) activities that yield immediate feedback through discussion with peers and/or instructors.”
******************************************************


To access the complete 90 kB post please click on http://bit.ly/1e8Zhpr.

Richard Hake, Emeritus Professor of Physics, Indiana University; Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands; President, PEdants for Definitive Academic References which Recognize the Invention of the Internet (PEDARRII); LINKS TO: Academia http://bit.ly/a8ixxm; Articles http://bit.ly/a6M5y0; Blog http://bit.ly/9yGsXh; Facebook http://on.fb.me/XI7EKm; GooglePlus http://bit.ly/KwZ6mE; Google Scholar http://bit.ly/Wz2FP3; Linked In http://linkd.in/14uycpW; Research Gate http://bit.ly/1fJiSwB; Socratic Dialogue Inducing (SDI) Labs http://bit.ly/9nGd3M; Twitter http://bit.ly/juvd52.

“There is substantial evidence that scientific teaching in the sciences, i.e., teaching that employs instructional strategies that encourage undergraduates to become actively engaged in their own learning, can produce levels of understanding, retention and transfer of knowledge that are greater than those resulting from traditional lecture/lab classes. But widespread acceptance by university faculty of new pedagogies and curricular materials still lies in the future.” - Robert DeHaan (2005)


REFERENCES [URLs shortened by http://bit.ly/ and accessed on 29 Jan 2014.]
Hake, R.R. 2014. “Two Different Meanings of 'Formative Evaluation' #2” online on the OPEN! AERA-L archives at http://bit.ly/1e8Zhpr. The abstract and link to the complete post are being transmitted to several discussion lists.

DeHaan, R.L. 2005. “The Impending Revolution in Undergraduate Science Education,” Journal of Science Education and Technology 14(2): 253-269; online as a 152 kB pdf at http://bit.ly/ncAuQa.

ETS Pushes For FORMATIVE Assessment?

Some blog followers might be interested in discussion-list post “ETS Pushes For FORMATIVE Assessment?” [Hake (2011)].

The abstract reads:

*******************************************

ABSTRACT: ARN-L’s Monty Neil has pointed to Valerie Strauss’ “Parents spell out detailed school reform blueprint” at http://wapo.st/edky9k. Among the recommendations is: “Less standardized testing overall. Utilizing tests as DIAGNOSTIC tools, not as a way to punish students, teachers, and schools.”

Then ARN-L’s Bob Schaeffer indicated a related ETS news release “ETS and schoolBrains (TM) Join Forces to Provide School Districts with Items for Benchmark Assessments” at http://bit.ly/hpRH1X, wherein it’s stated:

“Benchmark assessments are a must in today's educational environment with districts needing the ability to demonstrate student proficiency . . . . in order to meet state and federal guidelines for success. . . . The ETS formative assessment Item Bank contains more than 64,000 high-quality items that measure English Language Arts (reading and writing) and mathematics proficiency for kindergarten through grade 12 and science proficiency for grades 3 through high school.”

However, formative evaluation is defined by the “Joint Committee on Standards for Educational Evaluation” http://bit.ly/efXea6 as: “evaluation designed and used to improve an object, especially when it is still being developed.” But state and federal guidelines require SUMMATIVE evaluation. So it's not clear to me why ETS and schoolBrains are touting the new tests as “FORMATIVE assessment.”

I have discussed the unrealized potential that formative assessment might have for enhancing learning in U.S. Higher Education in “The Physics Education Reform Effort: A Possible Model for Higher Education” at http://bit.ly/9aicfh.

BTW, regarding the distinction sometimes made between assessment and evaluation, the late Bob Leamnson opined http://bit.ly/g2i7Oe: “To the several who suggested that ‘we all get together’ (somehow) and agree on how ‘evaluation’ and ‘assessment’ are to be defined and thereafter used by one and all, I suggest that you're trying to swim up Niagara Falls.”

*******************************************

To access the complete 14 kB post please click on http://bit.ly/hQHID2.

Richard Hake, Emeritus Professor of Physics, Indiana University

Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands

President, PEdants for Definitive Academic References which Recognize the Invention of the Internet (PEDARRII)

rrhake@earthlink.net

http://www.physics.indiana.edu/~hake

http://www.physics.indiana.edu/~sdi

http://HakesEdStuff.blogspot.com

http://iub.academia.edu/RichardHake

“What we assess is what we value. We get what we assess, and if we don’t assess it, we won’t get it.”

Lauren Resnick [quoted by Grant Wiggins (1990)]

REFERENCES [All URL's shortened by http://bit.ly/ and accessed on 24 April 2011.]

Hake, R.R. 2011. “ETS Pushes For FORMATIVE Assessment?” online on the OPEN! AERA-L archives at http://bit.ly/hQHID2. Post of 24 Apr 2011 14:04:36 -0700 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to various discussion lists.

Neil, M. 2011. “Parents offer reform blueprint to counter NCLB & Dept Blueprint - greatly parallels FEA proposals,” ARN-L post of 23 Apr 2011 08:22:34-0400; online on the OPEN! ARN-L archives at http://bit.ly/i3JpdN.

Schaeffer, B. 2011. “Test-Makers Pursue Market for Common Standards ‘Formative Assessments’,” ARN-L post of 23 Apr 2011 13:30:16-0400; online on the OPEN! ARN-L archives at http://bit.ly/hYg989.

Wiggins, G. 1990. “The Truth May Make You Free, But the Test May Keep You Imprisoned: Toward Assessment Worthy of the Liberal Arts,” AAHE Assessment Forum: 17-31; online at http://bit.ly/a7g09T.