Re: How Reliable Are the Social Sciences?

Some blog followers might be interested in a recent discussion-list post “Re: How Reliable Are the Social Sciences?” The abstract reads:

****************************************************
ABSTRACT: Rick Froman of the TIPS discussion list has pointed to a New York Times Opinion Piece “How Reliable Are the Social Sciences?” by Gary Gutting at http://nyti.ms/K0xVQL. Gutting wrote that Obama, in his State of the Union address http://wapo.st/JnuBCO cited “The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood” (Chetty et al., 2011) at http://bit.ly/KkanoU to support his emphasis on evaluating teachers by their students' test scores. That study purportedly shows that students with teachers who raise their standardized test scores are “more likely to attend college, earn higher salaries, live in better neighborhoods, and save more for retirement.”

After comparing the reliability of social-science research unfavorably with that of physical-science research, Getting wrote [my italics): “is there any work on the effectiveness of teaching that is solidly enough established to support major policy decisions? the case for a negative answer lies in the [superior] predictive power of the core natural sciences compared with even the most highly developed social sciences.”

Most education experts would probably agree with Getting's negative answer. Even economist Eric Hanushek http://en.wikipedia.org/wiki/Eric_Hanushek, as reported by Lowery http://nyti.ms/KnRvDh, states: “Very few people suggest that you should use value-added scores alone to make personnel decisions.”

But then Getting goes on to write (slightly edited): “While the physical sciences produce many detailed and precise predictions, the social sciences do not. The reason is that such predictions almost always require randomized controlled trials (RCT’s) which are seldom possible when people are involved. . . . . . Jim Manzi . . . . . . . . . . .[[according to Wikipedia http://bit.ly/KqMf1M, a senior fellow at the conservative Manhattan Institute http://bit.ly/JvwKG1 ]]. . . . in his recent book Uncontrolled http://amzn.to/JFalMD offers a careful and informed survey of the problems of research in the social sciences and concludes that non-RCT social science is not capable of making useful, reliable, and nonobvious predictions for the effects of most proposed policy interventions.” BUT:

(1) Randomized controlled trails may be the “gold standard” for medical research, but they are not such for the social science of educational research - see e.g., “Seventeen Statements by Gold-Standard Skeptics #2” (Hake, 2010) at http://bit.ly/oRGnBp .

(2) Unknown to most of academia, and probably to Getting and Manzi, ever since the pioneering work of Halloun & Hestenes (1985a) at http://bit.ly/fDdJHm, physicists have been engaged in the social science of Physics Education Research that is “capable of making useful, reliable, and nonobvious predictions,” e.g., that “interactive engagement” courses can achieve average normalized pre-to-posttest gains which are about two-standard deviations above comparison courses subjected to “traditional” passive-student lecture courses. This work employs pre/post testing with Concept Inventories http://en.wikipedia.org/wiki/Concept_inventory - see e.g., (a) “The Impact of Concept Inventories on Physics Education and It’s Relevance For Engineering Education” (Hake, 2011) at http://bit.ly/nmPY8F, and (b) “Why Not Try a Scientific Approach to Science Education?” (Wieman, 2007) at http://bit.ly/anTMfF.
****************************************************

To access the complete 26 kB post please click on http://bit.ly/K432fC.

Richard Hake, Emeritus Professor of Physics, Indiana University
rrhake@earthlink.net
Links to Articles: http://bit.ly/a6M5y0
Links to SDI Labs: http://bit.ly/9nGd3M
Blog: http://bit.ly/9yGsXh
Academia: http://iub.academia.edu/RichardHake
Twitter https://twitter.com/#!/rrhake

“In some quarters, particularly medical ones, the randomized experiment is considered the causal ‘gold standard.’ It is clearly not that in educational contexts, given the difficulties with implementing and maintaining randomly created groups, with the sometimes incomplete implementation of treatment particulars, with the borrowing of some treatment particulars by control group units, and with the limitations to external validity that often follow from how the random assignment is achieved.”
- Tom Cook & Monique Payne (2002, p. 174)

“. . .the important distinction. . .[between, e.g., education and physics]. . . is really not between the hard and the soft sciences. Rather, it is between the hard and the easy sciences.”
- David Berliner (2002)

“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.”
- Wood & Gentile (2003)

REFERENCES [All URL’s shortened by http://bit.ly/ and accessed on 21 May 2012.]
Berliner, D. 2002. “Educational research: The hardest science of all,” Educational Researcher 31(8): 18-20; online as a 49 kB pdf at http://bit.ly/GAitqc.

Cook, T.D. & M.R. Payne. 2002. “Objecting to the Objections to Using Random Assignment in Educational Research” in Mosteller & Boruch (2002).

Hake, R.R. 2012. “Re: How Reliable Are the Social Sciences?” online on the OPEN! AERA-L archives at http://bit.ly/K432fC. Post of 20 May 2012 20:08:07-0700 to AERA-L and Net-Gold. The abstract and link to the complete post are also being transmitted to several discussion lists.

Mosteller, F. & R. Boruch, eds. 2002. Evidence Matters: Randomized Trials in Education Research. Brookings Institution. Amazon.com information at http://amzn.to/n6T0Uo. A searchable expurgated Google Book Preview is online at http://bit.ly/mTcPIE.

Wood, W.B. & J.M. Gentile. 2003. “Teaching in a research context,” Science 302: 1510; 28 November; online to subscribers at http://bit.ly/9izfFz. A summary is online to all at http://bit.ly/9qGR6m.