In Defense of Pre/Post Testing With Concept Inventories

Some blog followers might be interested in a recent discussion-list post “In Defense of Pre/Post Testing With Concept Inventories” [Hake (2012)]. The abstract reads:

********************************************
Bruce Sherwood and Paul Camp, in recent posts on the “Physics Education Research Topical Group” (PERTG) discussion list, denigrate pre/post testing with concept inventories, typifying a trend in some physics education research circles to discount such practice as naive and useless.

I think such sentiment betrays an ignorance of the history of PER. In “Lessons from the Physics Education Reform Effort” [Hake (2002)] at http://bit.ly/aL87VT I wrote:

“For more than three decades, physics education researchers have repeatedly shown that traditional introductory physics courses with passive student lectures, recipe labs, and algorithmic problem exams are of limited value in enhancing students’ conceptual understanding of the subject. Unfortunately, this work was largely ignored by the physics and education communities until Halloun and Hestenes (1985a,b) devised the Mechanics Diagnostic test of conceptual understanding of Newtonian mechanics.”

For a review of pre/post testing with concept inventories in physics and engineering see “The Impact of Concept Inventories On Physics Education and It’s Relevance For Engineering Education” [Hake (2011)] at http://bit.ly/nmPY8F (8.5 MB).
********************************************

To access the complete 12 kB post please click on http://bit.ly/LaPNbm.

Richard Hake, Emeritus Professor of Physics, Indiana University
Links to Articles: http://bit.ly/a6M5y0
Links to SDI Labs: http://bit.ly/9nGd3M
Blog: http://bit.ly/9yGsXh
Twitter http://bit.ly/juvd52
GooglePlus: http://bit.ly/KwZ6mE

"He who knows only his own generation
Remains always a child."
- Cicero (in Orator)


REFERENCES [URL shortened by http://bit.ly/ and accessed on 25 June 2012.
Hake, R.R. 2012. “In Defense of Pre/Post Testing With Concept Inventories,” online on the OPEN! AERA-L archives at http://bit.ly/LaPNbm. Post of 25 Jun 2012 12:31:15-0700 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to several discussion lists.

No Standard Outcome Measures For Science Education? #2

Some blog followers might be interested in “No Standard Outcome Measures For Science Education? #2” [Hake (2011)].

The abstract reads:

*************************************************
ABSTRACT: Robin Millar and Jonathan Osborne in Chapter 3, “Research and Practice: A Complex Relationship” of Shelley et al. (2009). claimed that: (a) NO STANDARD OR COMMONLY AGREED OUTCOME MEASURES EXIST FOR ANY MAJOR TOPIC IN SCIENCE EDUCATION. . . . [[my CAPS]]. . . ; (b) the Force Concept Inventory (FCI) reflects a choice of values that is arguable; and (c) the FCI has not been subjected to the same rigorous scrutiny of factorial structure and content validity as have standard measures in psychology.

That no standard outcome measures exist for any major topic in science education is negated by the existence of Concept Inventories for astronomy, biology, chemistry, economics, engineering, geoscience, and math.

That the FCI reflects values that are arguable is correct only if the arguers think that there’s little value in students’ learning the basic concepts of Newtonian mechanics.

That the FCI has not been subjected to rigorous scrutiny of factorial structure ignores the 1995 factor analyses of Huffman & Heller and Heller & Huffman; and responses to those analyses by Hestenes & Halloun and Halloun & Hestenes.

That the FCI has not been as not been subjected to rigorous scrutiny of content validity ignores section IIB. “Validity and reliability of the mechanics test” (Mechanics Diagnostic) in Halloun & Hestenes (1985a) - that verification of validity applies also to the FCI since it's almost the same as the Mechanics Diagnostic.
*************************************************

To access the complete 24 kB article please click on http://bit.ly/rfyamc.

Richard Hake, Emeritus Professor of Physics, Indiana University
Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands
President, PEdants for Definitive Academic References
which Recognize the Invention of the Internet (PEDARRII)

rrhake@earthlink.net
http://www.physics.indiana.edu/~hake
http://www.physics.indiana.edu/~sdi
http://HakesEdStuff.blogspot.com
http://iub.academia.edu/RichardHake

“50 years of research, curriculum development, and implementation have not presented consistent and compelling patterns of outcomes.”
Shelley et al. (2009, p. 4), summarizing a claim by Osborne (2007)

“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.”
Wood & Gentile (2003) "Teaching in a research context"


REFERENCES [All URL’s shortened by http://bit.ly/ and accessed on 16 Oct 2011.]

Hake, R.R. 2011. “No Standard Outcome Measures For Science Education? #2” online on the OPEN! AERA-L archives at http://bit.ly/rfyamc . Post of 16 Oct 2011 11:04:41-0700 to AERA-L and Net-Gold. The abstract and link to the complete post were transmitted to various discussion lists.

Osborne, J. 2007. “In praise of armchair science education,” contained within E-NARST News 50(2), online as a 3.2 MB pdf at http://bit.ly/qsRwaK . The talk itself is online as a 112 kB pdf at http://bit.ly/r4Khl7.

Shelley, M.C., L.D. Yore, & B. Hand, eds. 2009. Quality Research in Literacy and Science Education: International Perspectives and Gold Standards. Springer, publisher's information at http://bit.ly/b58vbP. Amazon.com information at http://amzn.to/97OVJx, note the searchable “Look Inside” feature. Barnes & Noble information at http://bit.ly/p40bKu. An expurgated (teaser) version is online as a Google “book preview” at http://bit.ly/qK8T9P.

Wood, W.B., & J.M. Gentile. 2003. “Teaching in a research context,” Science 302: 1510; 28 November; online as a 209 kB pdf at http://bit.ly/oK46p7.

The Impact of Concept Inventories On Physics Education and It’s Relevance For Engineering Education

Some blog followers might be interested in “The Impact of Concept Inventories On Physics Education and It’s Relevance For Engineering Education” [Hake (2011)].

The abstract reads:

*********************************************

I review the:
(a) Before Concept Inventory (BCI) dark ages of post-secondary introductory physics education;

(b) 1985 advent of the first physics CI, the Mechanics Diagnostic (MD) by Halloun & Hestenes (HH);

(c) 1987-90 early research use of the (MD) by HH, Hake, and Crouch & Mazur;

(d) 1992 Force Concept Inventory (FCI), successor to the MD, and the Factor Analysis Debate (1995);

(e) 1995 revision of the FCI by Halloun, Hake, Mosca, and Hestenes;

(f) 1998 meta-analysis of FCI/MD results on 62 introductory physics courses (N = 6542) showing about a two-standard-deviation superiority in average normalized gains for “interactive engagement” over traditional passive-student lecture courses by Hake and subsequent confirmation by about 25 other physics education research studies.

I then indicate:

(a) fourteen hard lessons from the physics education reform effort;

(b) suggestions for the administration and reporting of CI’s;

(c) listings of CI’s, including those for physics and engineering; and comment that:

(d) for physics education the road to reform has been all uphill;

(e) the glacial inertia of the educational system, though not well understood, appears to be typical of the slow Diffusion of Innovations [Rogers (2003)] in human society;

(f) there are at least “Eleven Barriers to Change in Higher Education”;

(g) but, even so, for physics education, Rogers’ “early adopters” of reform have now appeared at Harvard, North Carolina State University, MIT, the Univ. of Colorado, California Polytechnic at San Luis Obispo, and the Univ. of British Columbia, possibly presaging a Rogers “take off” for physics education reform, about two decades ACI (After Concept Inventory).

I conclude that:

(a) CI’s can stimulate reform, but judging from the results in physics it may take about two decades before even early adopters become evident;

(b) there are at least seven reasons why the rate of adoption of reforms may be greater in engineering education than in physics education.

In an Appendix I respond to criticisms of the FCI and the average normalized gain g(ave).

*********************************************


To access the complete 8.7 MB article please click on http://bit.ly/nmPY8F.

Richard Hake, Emeritus Professor of Physics, Indiana University 24245 Hatteras Street, Woodland Hills, CA 91367

Honorary Member, Curmudgeon Lodge of Deventer, The Netherlands.

rrhake@earthlink.net http://www.physics.indiana.edu/~hake/ http://www.physics.indiana.edu/~sdi/ http://HakesEdStuff.blogspot.com/ http://iub.academia.edu/RichardHake

REFERENCES

Hake, R.R. 2011. “The Impact of Concept Inventories On Physics Education and It's Relevance For Engineering Education,” invited talk, 8 August, second annual NSF-sponsored “National Meeting on STEM Concept Inventories,” Washington, D.C., online as an 8.7 MB pdf at http://bit.ly/nmPY8F or as ref. 64 at http://bit.ly/a6M5y0.