Some blog followers might be interested in a discussion list post “Re: Scientific American Article on Educational Research and Evaluation” [Hake (2014)]. The abstract reads:
********************************************
ABSTRACT: EvalTalk’s David Colton wrote at http://bit.ly/1vuEiDu (my inserts at “. . . . . . .[[insert]] . . . . . . . . . ”:
“A decade ago, the ‘American Evaluation Association’ . . . . . [[http://bit.ly/1s1m5Mb]]. . . . issued a position paper . . . . . .[[(AEA, 2003) at http://bit.ly/1tgrYsI, highly critical of ]]. . . . . . . . the U.S. Dept. of Education’s (USDE’s) decision to award research grants based on methodology, with experimental and quasi experimental designs given funding prior over other approaches . . . . . . .[[“experimental” is RCT enthusiasts’ code for methodology utilizing “Randomized Control Trails” (RCTs)]]. . . . So I was very interested in an article in this month’s Scientific American which describes the results of this process ten years out: ‘The Science of Learning’ . . . . . . .[[Kantrowitz (2014), re-titled in the online version “Scientists Bring New Rigor to Education Research” and online at http://bit.ly/1v23502]]. . . . . . . . .”
In the present post I excerpt and annotate 8 noteworthy passages from Kantrowitz’s article dealing with e.g., the RCT debate http://bit.ly/1vV222A; the “What Works Clearing House” http://ies.ed.gov/ncee/wwc/; the “Institute of Education Sciences” (IES) http://ies.ed.gov/; Paulo Blikstein's “FabLab” at Stanford http://bit.ly/1yIcekw; teacher evaluation http://bit.ly/1xm6R6b, class size http://bit.ly/1naRX27; student engagement http://bit.ly/9484DG, discovery learning http://bit.ly/1snHAK3, Grover Whitehurst http://bit.ly/RIcEz4, “Finnish Lessons” http://bit.ly/JpU9fD; and “Finnishing Touches” http://bit.ly/Ixkqa7.
**********************************
To access the complete 131 kB post please click on http://bit.ly/1smsIKA.
Richard Hake, Emeritus Professor of Physics, Indiana University; LINKS TO: Academia http://bit.ly/a8ixxm; Articles http://bit.ly/a6M5y0; Blog http://bit.ly/9yGsXh; Facebook http://on.fb.me/XI7EKm; GooglePlus http://bit.ly/KwZ6mE; Google Scholar http://bit.ly/Wz2FP3; Linked In http://linkd.in/14uycpW; Research Gate http://bit.ly/1fJiSwB; Socratic Dialogue Inducing (SDI) Labs http://bit.ly/9nGd3M; Twitter http://bit.ly/juvd52.
REFERENCES [URL shortened by http://bit.ly/ and accessed on 10 Oct 2014.]
Hake, R.R. 2014, “Re: Scientific American Article on Educational Research and Evaluation," online on the OPEN! AERA-L archives at http://bit.ly/1smsIKA. The abstract and link to the complete post are being transmitted to several discussion lists.
Showing posts with label class size. Show all posts
Showing posts with label class size. Show all posts
Friday, October 10, 2014
Sunday, December 9, 2012
Grover Whitehurst Testifies Against Class Size Reduction
Some blog followers might be interested in a recent post “Grover Whitehurst Testifies Against Class Size Reduction” [Hake (2012)]. The abstract reads:
*********************************************
ABSTRACT: Diane Ravitch in her blog entry “When Grover Whitehurst Testified Against Class Size Reduction” at http://bit.ly/QRniWu pointed to Leonie Haimson’s http://huff.to/12fHNyy “Grover Whitehurst's big pay day, testifying class size doesn't matter” at http://bit.ly/Vsp2T2 and asked: “DOES CLASS SIZE MATTER? READ HAIMSON'S ACCOUNT AND REACH YOUR OWN JUDGMENT.”
Haimson pointed out that:
a. According to report by Will Weissert (2012a) at http://yhoo.it/VxrBCQ, economist Diane Whitmore Schanzenbach http://bit.ly/TVN6hv testified that students in smaller classes “tend to do better on standardized tests and even eventually become better citizens, more likely to own their own homes and save for retirement” and that “study after study shows that smaller classes often mean greater success for students.” Schanzenbach also coathored: (1) “Experimental Evidence on the Effect of Childhood Investments on Postsecondary Attainment and Degree Completion” [Dynarski et al. (2011)] at http://bit.ly/YRF0h7 showing that smaller classes increased the rate of college attendance, especially among poor students, and improved the probability of earning a college degree, especially in high-earning fields such as science, technology, engineering, and mathematics; and (2) “How Does Your Kindergarten Classroom Affect Your Earnings? Evidence From Project Star” [Chetty et al. (2011)] at http://bit.ly/U7oJNn showing that these students were also more likely to own their own home and a 401K more than twenty years later.
b. Whitehurst & Chingos (2011) wrote “Class Size: What Research Says and What it Means for State Policy” [Chingos & Whitehurst (2011)] at http://bit.ly/VXroeA which argued that LOWERING CLASS SIZE WAS A WASTE OF MONEY despite admitting in the report that “very large class-size reductions, on the order of magnitude of 7-10 fewer students per class, can have significant long-term effects on student achievement and other meaningful outcomes.”
c. When Whitehurst was at the US Department of Education from 2002-2008, he headed the Institute of Education Sciences http://ies.ed.gov/, which in a report Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide [USDE (2003)] at http://bit.ly/ds1sRS cited CLASS SIZE REDUCTION AS ONLY ONE OF FOUR EXAMPLES OF EDUCATION REFORMS "FOUND TO BE EFFECTIVE in randomized controlled trials – research’s 'gold standard."[Yet] as the lead off witness for the state on Friday Whitehurst argued that contrary to the claims of the plaintiffs, “Texas is doing pretty good” and that these huge budget cuts were immaterial because CLASS SIZE DOESN'T MATTER.
d. Terrence Stutz (2012) at http://bit.ly/VXwA2l reported in the Dallas Morning News: “State attorneys also have been arguing that larger class sizes in Texas - the result of a $5.4 billion funding cut by the Legislature last year - have not hurt students because CLASS SIZES DON'T AFFECT ACHIEVEMENT. Whitehurst testified in support of that position. But again, under cross examination by Dallas lawyer John Turner, Whitehurst had to acknowledge that he wrote an article praising a well-publicized study of lower class sizes in Tennessee that found significant improvement in student achievement. Whitehurst explained that he had changed his mind since writing the article and now has DOUBTS THAT CLASS SIZE HAS MUCH IMPACT ON LEARNING. In later testimony, he said he was being paid $340 an hour by the state to testify in the case, and had already racked up 220 billable hours - for just under $75,000 - before he took the witness stand."
e. Whitehurst racked up 220 billable hours? That means Whitehurst must have worked nearly thirty 8-hour days on it. Wonder what took him so long?
*********************************************
To access the complete 23 kB post please click on http://bit.ly/Ufro6y.
Richard Hake, Emeritus Professor of Physics, Indiana University
Links to Articles: http://bit.ly/a6M5y0
Links to Socratic Dialogue Inducing (SDI) Labs: http://bit.ly/9nGd3M
Academia: http://bit.ly/a8ixxm
Blog: http://bit.ly/9yGsXh
GooglePlus: http://bit.ly/KwZ6mE
Twitter: http://bit.ly/juvd52
“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.”
- Wood & Gentile (2003)
“In science education, there is almost nothing of proven efficacy.”
- Grover Whitehurst, as quoted by Sharon Begley (2004)
“Well-designed and implemented randomized controlled trials are considered the ‘gold standard’ for evaluaing an intervention's effectiveness, in fields such as medicine, welfare and employment policy, and psychology.”
- USDE (2003)
“Scientifically rigorous studies - particularly, the ‘gold standard’ of Randomized Controlled Trials (RCT’s) - are a mainstay of medicine, providing conclusive evidence of effectiveness for most major medical advances in recent history. In social spending, by contrast, such studies have only a toehold. Where they have been used, however, they have demonstrated the same ability to produce important, credible evidence about what works - and illuminated a path to major progress.”
- Jon Barron (2012)
“In some quarters, particularly medical ones, the randomized experiment is considered the causal 'gold standard.' It is clearly not that in educational contexts, given the difficulties with implementing and maintaining randomly created groups, with the sometimes incomplete implementation of treatment particulars, with the borrowing of some treatment particulars by control group units, and with the limitations to external validity that often follow from how the random assignment is achieved.”
- Thomas Cook and Monique Payne in Evidence Matters [Mosteller &
Boruch (2002)]
“According to the California Class Size Reduction Research Consortium [CCSRRC (2002)], California's attempt to duplicate the vaunted Tennessee RCT study of reduced class size benefits results yielded no conclusive evidence of increased student achievement. One reason appears to be that there were simply not enough teachers in California to support any substantive class size reduction without deterioration of teaching effectiveness.”
- R.R. Hake (2009)
REFERENCES [URL’s shortened by http://bit.ly/ and accessed on 10 Dec 2012.]
Baron, J. 2012. “Applying Evidence to Social Programs.” New York Times, 29 Nov; online at http://nyti.ms/Um9vVI.
Begley, S. 2004. “To Improve Education, We Need Clinical Trials To Show What Works,” Wall Street Journal, 17 December, page B1; online as a 41 kB pdf at http://bit.ly/SSmaym, thanks to David Klahr.
CCSRRC. 2002. “What We Have Learned About Class Size Reduction in California,” California Class Size Reduction Research Consortium [American Institutes for Research (AIR), RAND, Policy Analysis for California Education (PACE), WestEd, and EdSource]; full report online as a 9.5 MB pdf at http://bit.ly/YRD5ZS. A press release is online at http://bit.ly/V923Ms.
Hake, R.R. 2009. “A Response to ‘It's Not All About Class Size’,” AERA-L post of 6 Feb 2009 09:42:04-0800; online on the OPEN! AERA-L archives at http://bit.ly/KBzuXV. Post of 6 Feb 2009 09:42:04-0800 to AERA-L and Net-Gold. The abstract and link to the complete post were also distributed to various discussion lists.
Hake, R.R. 2012. “Grover Whitehurst Testifies Against Class Size Reduction” online on the OPEN! online on the OPEN! AERA-L archives at http://bit.ly/VYtD1l. Post of 9 Dec 2012 18:34:56-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to several discussion lists.
Mosteller, F. & R. Boruch, eds. 2002. Evidence Matters: Randomized Trials in Education Research. Brookings Institution, publisher's information at http://bit.ly/UoX3sA. Amazon.com information at http://amzn.to/n6T0Uo. An expurgated Google book preview is online at http://bit.ly/RX1k3u.
USDE. 2003. U.S. Department of Education, Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide. Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, online as a 140 kB pdf at http://bit.ly/ds1sRS.
Wood, W.B., & J.M. Gentile. 2003. “Teaching in a Research Context,” Science 302: 1510; 28 November; online as a 213 kB pdf at http://bit.ly/SyhOvL thanks to Portland State's “Ecoplexity” site.
*********************************************
ABSTRACT: Diane Ravitch in her blog entry “When Grover Whitehurst Testified Against Class Size Reduction” at http://bit.ly/QRniWu pointed to Leonie Haimson’s http://huff.to/12fHNyy “Grover Whitehurst's big pay day, testifying class size doesn't matter” at http://bit.ly/Vsp2T2 and asked: “DOES CLASS SIZE MATTER? READ HAIMSON'S ACCOUNT AND REACH YOUR OWN JUDGMENT.”
Haimson pointed out that:
a. According to report by Will Weissert (2012a) at http://yhoo.it/VxrBCQ, economist Diane Whitmore Schanzenbach http://bit.ly/TVN6hv testified that students in smaller classes “tend to do better on standardized tests and even eventually become better citizens, more likely to own their own homes and save for retirement” and that “study after study shows that smaller classes often mean greater success for students.” Schanzenbach also coathored: (1) “Experimental Evidence on the Effect of Childhood Investments on Postsecondary Attainment and Degree Completion” [Dynarski et al. (2011)] at http://bit.ly/YRF0h7 showing that smaller classes increased the rate of college attendance, especially among poor students, and improved the probability of earning a college degree, especially in high-earning fields such as science, technology, engineering, and mathematics; and (2) “How Does Your Kindergarten Classroom Affect Your Earnings? Evidence From Project Star” [Chetty et al. (2011)] at http://bit.ly/U7oJNn showing that these students were also more likely to own their own home and a 401K more than twenty years later.
b. Whitehurst & Chingos (2011) wrote “Class Size: What Research Says and What it Means for State Policy” [Chingos & Whitehurst (2011)] at http://bit.ly/VXroeA which argued that LOWERING CLASS SIZE WAS A WASTE OF MONEY despite admitting in the report that “very large class-size reductions, on the order of magnitude of 7-10 fewer students per class, can have significant long-term effects on student achievement and other meaningful outcomes.”
c. When Whitehurst was at the US Department of Education from 2002-2008, he headed the Institute of Education Sciences http://ies.ed.gov/, which in a report Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide [USDE (2003)] at http://bit.ly/ds1sRS cited CLASS SIZE REDUCTION AS ONLY ONE OF FOUR EXAMPLES OF EDUCATION REFORMS "FOUND TO BE EFFECTIVE in randomized controlled trials – research’s 'gold standard."[Yet] as the lead off witness for the state on Friday Whitehurst argued that contrary to the claims of the plaintiffs, “Texas is doing pretty good” and that these huge budget cuts were immaterial because CLASS SIZE DOESN'T MATTER.
d. Terrence Stutz (2012) at http://bit.ly/VXwA2l reported in the Dallas Morning News: “State attorneys also have been arguing that larger class sizes in Texas - the result of a $5.4 billion funding cut by the Legislature last year - have not hurt students because CLASS SIZES DON'T AFFECT ACHIEVEMENT. Whitehurst testified in support of that position. But again, under cross examination by Dallas lawyer John Turner, Whitehurst had to acknowledge that he wrote an article praising a well-publicized study of lower class sizes in Tennessee that found significant improvement in student achievement. Whitehurst explained that he had changed his mind since writing the article and now has DOUBTS THAT CLASS SIZE HAS MUCH IMPACT ON LEARNING. In later testimony, he said he was being paid $340 an hour by the state to testify in the case, and had already racked up 220 billable hours - for just under $75,000 - before he took the witness stand."
e. Whitehurst racked up 220 billable hours? That means Whitehurst must have worked nearly thirty 8-hour days on it. Wonder what took him so long?
*********************************************
To access the complete 23 kB post please click on http://bit.ly/Ufro6y.
Richard Hake, Emeritus Professor of Physics, Indiana University
Links to Articles: http://bit.ly/a6M5y0
Links to Socratic Dialogue Inducing (SDI) Labs: http://bit.ly/9nGd3M
Academia: http://bit.ly/a8ixxm
Blog: http://bit.ly/9yGsXh
GooglePlus: http://bit.ly/KwZ6mE
Twitter: http://bit.ly/juvd52
“Physics educators have led the way in developing and using objective tests to compare student learning gains in different types of courses, and chemists, biologists, and others are now developing similar instruments. These tests provide convincing evidence that students assimilate new knowledge more effectively in courses including active, inquiry-based, and collaborative learning, assisted by information technology, than in traditional courses.”
- Wood & Gentile (2003)
“In science education, there is almost nothing of proven efficacy.”
- Grover Whitehurst, as quoted by Sharon Begley (2004)
“Well-designed and implemented randomized controlled trials are considered the ‘gold standard’ for evaluaing an intervention's effectiveness, in fields such as medicine, welfare and employment policy, and psychology.”
- USDE (2003)
“Scientifically rigorous studies - particularly, the ‘gold standard’ of Randomized Controlled Trials (RCT’s) - are a mainstay of medicine, providing conclusive evidence of effectiveness for most major medical advances in recent history. In social spending, by contrast, such studies have only a toehold. Where they have been used, however, they have demonstrated the same ability to produce important, credible evidence about what works - and illuminated a path to major progress.”
- Jon Barron (2012)
“In some quarters, particularly medical ones, the randomized experiment is considered the causal 'gold standard.' It is clearly not that in educational contexts, given the difficulties with implementing and maintaining randomly created groups, with the sometimes incomplete implementation of treatment particulars, with the borrowing of some treatment particulars by control group units, and with the limitations to external validity that often follow from how the random assignment is achieved.”
- Thomas Cook and Monique Payne in Evidence Matters [Mosteller &
Boruch (2002)]
“According to the California Class Size Reduction Research Consortium [CCSRRC (2002)], California's attempt to duplicate the vaunted Tennessee RCT study of reduced class size benefits results yielded no conclusive evidence of increased student achievement. One reason appears to be that there were simply not enough teachers in California to support any substantive class size reduction without deterioration of teaching effectiveness.”
- R.R. Hake (2009)
REFERENCES [URL’s shortened by http://bit.ly/ and accessed on 10 Dec 2012.]
Baron, J. 2012. “Applying Evidence to Social Programs.” New York Times, 29 Nov; online at http://nyti.ms/Um9vVI.
Begley, S. 2004. “To Improve Education, We Need Clinical Trials To Show What Works,” Wall Street Journal, 17 December, page B1; online as a 41 kB pdf at http://bit.ly/SSmaym, thanks to David Klahr.
CCSRRC. 2002. “What We Have Learned About Class Size Reduction in California,” California Class Size Reduction Research Consortium [American Institutes for Research (AIR), RAND, Policy Analysis for California Education (PACE), WestEd, and EdSource]; full report online as a 9.5 MB pdf at http://bit.ly/YRD5ZS. A press release is online at http://bit.ly/V923Ms.
Hake, R.R. 2009. “A Response to ‘It's Not All About Class Size’,” AERA-L post of 6 Feb 2009 09:42:04-0800; online on the OPEN! AERA-L archives at http://bit.ly/KBzuXV. Post of 6 Feb 2009 09:42:04-0800 to AERA-L and Net-Gold. The abstract and link to the complete post were also distributed to various discussion lists.
Hake, R.R. 2012. “Grover Whitehurst Testifies Against Class Size Reduction” online on the OPEN! online on the OPEN! AERA-L archives at http://bit.ly/VYtD1l. Post of 9 Dec 2012 18:34:56-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to several discussion lists.
Mosteller, F. & R. Boruch, eds. 2002. Evidence Matters: Randomized Trials in Education Research. Brookings Institution, publisher's information at http://bit.ly/UoX3sA. Amazon.com information at http://amzn.to/n6T0Uo. An expurgated Google book preview is online at http://bit.ly/RX1k3u.
USDE. 2003. U.S. Department of Education, Identifying and Implementing Educational Practices Supported by Rigorous Evidence: A User Friendly Guide. Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, online as a 140 kB pdf at http://bit.ly/ds1sRS.
Wood, W.B., & J.M. Gentile. 2003. “Teaching in a Research Context,” Science 302: 1510; 28 November; online as a 213 kB pdf at http://bit.ly/SyhOvL thanks to Portland State's “Ecoplexity” site.
Tuesday, December 4, 2012
Randomized Control Trials - The Tarnished Gold Standard
Some blog followers might be interested in a recent post “Randomized Control Trials - The Tarnished Gold Standard” [Hake (2012a)]. The abstract reads:
************************************************
ABSTRACT: In response to “The Randomistas' War On Global Poverty - Erratum & Addendum” at http://bit.ly/YfMESg, Art Burke of the EvalTalk list pointed to an NYT piece “Applying Evidence to Social Programs” by Jon Baron at http://nyti.ms/Um9vVI.
Baron wrote (slightly edited): “Scientifically rigorous studies - particularly, the 'gold standard' of Randomized Controlled Trials (RCT’s) - are a mainstay of medicine, providing conclusive evidence of effectiveness for most major medical advances in recent history. In social spending, by contrast, such studies have only a toehold. Where they have been used, however, they have demonstrated the same ability to produce important, credible evidence about what works - and illuminated a path to major progress.”
In this post I cite arguments that the “gold standard” RCT studies may not be as lustrous as claimed by Baron:
(1) Ever since the pioneering work of Halloun & Hestenes (1985a) at http://bit.ly/fDdJHm, physicists have been engaged in social science of Physics Education Research (PER) that has made useful, reliable, and nonobvious predictions without resort to RCT’s - e.g. “Why Not Try a Scientific Approach to Science Education?” [Wieman (2007)] at http://bit.ly/anTMfF.
(2) In “A Response to ‘It's Not All About Class Size’ ” [Hake (2009)], I pointed out that according to the California Class Size Reduction Research Consortium [CCSRRC (2002)] at http://bit.ly/V923Ms, California's attempt to duplicate the vaunted Tennessee RCT study of reduced class size benefits results yielded no conclusive evidence of increased student achievement.
(3) In “A Summative Evaluation of RCT Methodology: & An Alternative Approach to Causal Research” [Scriven (2008] at http://bit.ly/93VcWD wrote: “In standard scientific usage, experiments are just carefully constrained explorations, and the RCT is simply a special case of these. To call the RCT the only ‘true experiment’ is part of an attempt at redefinition that distorts the original and continuing usage, and excludes experiments designed to test many simple hypotheses about - or simple efforts to find out - what happens if we do this.”
(4) In “Seventeen Statements by Gold-Standard Skeptics #2” [Hake (2010)] at http://bit.ly/TNpTR9 I cite, among others, the comments of the American Education Research Association; the American Evaluation Association; the National Education Association; the European Evaluation Society; Thomas Cook and Monique Payne, Hugh Burkhardt & Alan Schoenfeld; Margaret Eisenhart & Lisa Towne; Burke Johnson; Annette Lareau & Pamela Barnhouse; Joseph Maxwell; Dennis Phillips; Barbara Schneider, Martin Carnoy, Jeremy Kilpatrick, William Schmidt, & Richard Shavelson; Mack Shelley, Larry Yore, & Brian Hand; Deborah Stipek; and Carol Weiss.
(5) In "Why Most Published Research Findings Are False," John Ioannidis' (2005 at http://1.usa.gov/YxUxkL states ". . . . there is strong evidence that selective outcome reporting, with manipulation of the outcomes and analyses reported, is a common problem even for randomized trails [Chan et al. (2004)] at http://1.usa.gov/X8SB1T.
(6) the present signature quote of Thomas Cook and Monique Payne.
************************************************
To access the complete 18 kB post please click on http://bit.ly/VzVc0K.
Richard Hake, Emeritus Professor of Physics, Indiana University
Links to Articles: http://bit.ly/a6M5y0
Links to Socratic Dialogue Inducing (SDI) Labs: http://bit.ly/9nGd3M
Academia: http://bit.ly/a8ixxm
Blog: http://bit.ly/9yGsXh
GooglePlus: http://bit.ly/KwZ6mE
Twitter: http://bit.ly/juvd52
"In some quarters, particularly medical ones, the randomized experiment is considered the causal 'gold standard.' It is clearly not that in educational contexts, given the difficulties with implementing and maintaining randomly created groups, with the sometimes incomplete implementation of treatment particulars, with the borrowing of some treatment particulars by control group units, and with the limitations to external validity that often follow from how the random assignment is achieved."
- Thomas Cook and Monique Payne in "Evidence Matters" [Mosteller & Boruch (2002)]
REFERENCES [URL shortened by http://bit.ly/ and accessed on 04 Dec 2012.]
Hake, R.R. 2012a. "Randomized Control Trials - The Tarnished Gold Standard" online on the OPEN! AERA-L archives at http://bit.ly/VzVc0K. Post of 4 Dec 2012 19:26:48-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to several discussion lists.
Hake, R.R. 2012b. “The Randomistas' War On Global Poverty - ERRATUM & ADDENDUM,” online on the OPEN! AERA-L archives at http://bit.ly/YfMESg. Post of 30 Nov 2012 12:15:33-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to several discussion lists, and are also on my blog “Hake'sEdStuff” at http://bit.ly/11c5w3e with a provision for comments.
Mosteller, F. & R. Boruch, eds. 2002. Evidence Matters: Randomized Trials in Education Research. Brookings Institution, publisher's information at http://bit.ly/UoX3sA. Amazon.com information at http://amzn.to/n6T0Uo. An expurgated Google book preview is online at http://bit.ly/RX1k3u.
************************************************
ABSTRACT: In response to “The Randomistas' War On Global Poverty - Erratum & Addendum” at http://bit.ly/YfMESg, Art Burke of the EvalTalk list pointed to an NYT piece “Applying Evidence to Social Programs” by Jon Baron at http://nyti.ms/Um9vVI.
Baron wrote (slightly edited): “Scientifically rigorous studies - particularly, the 'gold standard' of Randomized Controlled Trials (RCT’s) - are a mainstay of medicine, providing conclusive evidence of effectiveness for most major medical advances in recent history. In social spending, by contrast, such studies have only a toehold. Where they have been used, however, they have demonstrated the same ability to produce important, credible evidence about what works - and illuminated a path to major progress.”
In this post I cite arguments that the “gold standard” RCT studies may not be as lustrous as claimed by Baron:
(1) Ever since the pioneering work of Halloun & Hestenes (1985a) at http://bit.ly/fDdJHm, physicists have been engaged in social science of Physics Education Research (PER) that has made useful, reliable, and nonobvious predictions without resort to RCT’s - e.g. “Why Not Try a Scientific Approach to Science Education?” [Wieman (2007)] at http://bit.ly/anTMfF.
(2) In “A Response to ‘It's Not All About Class Size’ ” [Hake (2009)], I pointed out that according to the California Class Size Reduction Research Consortium [CCSRRC (2002)] at http://bit.ly/V923Ms, California's attempt to duplicate the vaunted Tennessee RCT study of reduced class size benefits results yielded no conclusive evidence of increased student achievement.
(3) In “A Summative Evaluation of RCT Methodology: & An Alternative Approach to Causal Research” [Scriven (2008] at http://bit.ly/93VcWD wrote: “In standard scientific usage, experiments are just carefully constrained explorations, and the RCT is simply a special case of these. To call the RCT the only ‘true experiment’ is part of an attempt at redefinition that distorts the original and continuing usage, and excludes experiments designed to test many simple hypotheses about - or simple efforts to find out - what happens if we do this.”
(4) In “Seventeen Statements by Gold-Standard Skeptics #2” [Hake (2010)] at http://bit.ly/TNpTR9 I cite, among others, the comments of the American Education Research Association; the American Evaluation Association; the National Education Association; the European Evaluation Society; Thomas Cook and Monique Payne, Hugh Burkhardt & Alan Schoenfeld; Margaret Eisenhart & Lisa Towne; Burke Johnson; Annette Lareau & Pamela Barnhouse; Joseph Maxwell; Dennis Phillips; Barbara Schneider, Martin Carnoy, Jeremy Kilpatrick, William Schmidt, & Richard Shavelson; Mack Shelley, Larry Yore, & Brian Hand; Deborah Stipek; and Carol Weiss.
(5) In "Why Most Published Research Findings Are False," John Ioannidis' (2005 at http://1.usa.gov/YxUxkL states ". . . . there is strong evidence that selective outcome reporting, with manipulation of the outcomes and analyses reported, is a common problem even for randomized trails [Chan et al. (2004)] at http://1.usa.gov/X8SB1T.
(6) the present signature quote of Thomas Cook and Monique Payne.
************************************************
To access the complete 18 kB post please click on http://bit.ly/VzVc0K.
Richard Hake, Emeritus Professor of Physics, Indiana University
Links to Articles: http://bit.ly/a6M5y0
Links to Socratic Dialogue Inducing (SDI) Labs: http://bit.ly/9nGd3M
Academia: http://bit.ly/a8ixxm
Blog: http://bit.ly/9yGsXh
GooglePlus: http://bit.ly/KwZ6mE
Twitter: http://bit.ly/juvd52
"In some quarters, particularly medical ones, the randomized experiment is considered the causal 'gold standard.' It is clearly not that in educational contexts, given the difficulties with implementing and maintaining randomly created groups, with the sometimes incomplete implementation of treatment particulars, with the borrowing of some treatment particulars by control group units, and with the limitations to external validity that often follow from how the random assignment is achieved."
- Thomas Cook and Monique Payne in "Evidence Matters" [Mosteller & Boruch (2002)]
REFERENCES [URL shortened by http://bit.ly/ and accessed on 04 Dec 2012.]
Hake, R.R. 2012a. "Randomized Control Trials - The Tarnished Gold Standard" online on the OPEN! AERA-L archives at http://bit.ly/VzVc0K. Post of 4 Dec 2012 19:26:48-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to several discussion lists.
Hake, R.R. 2012b. “The Randomistas' War On Global Poverty - ERRATUM & ADDENDUM,” online on the OPEN! AERA-L archives at http://bit.ly/YfMESg. Post of 30 Nov 2012 12:15:33-0800 to AERA-L and Net-Gold. The abstract and link to the complete post are being transmitted to several discussion lists, and are also on my blog “Hake'sEdStuff” at http://bit.ly/11c5w3e with a provision for comments.
Mosteller, F. & R. Boruch, eds. 2002. Evidence Matters: Randomized Trials in Education Research. Brookings Institution, publisher's information at http://bit.ly/UoX3sA. Amazon.com information at http://amzn.to/n6T0Uo. An expurgated Google book preview is online at http://bit.ly/RX1k3u.
Friday, February 6, 2009
A Response to "It's Not All About Class Size"
Some blog readers may be interested in a recent post [Hake (2009] with the above title. The abstract reads:
****************************************************
Stephen Ceci and Spryos Konstantopoulos in a "Chronicle of Higher Education" commentary "It's Not All About Class Size" point out that the new administration needs to be aware that attempts to fulfill its pledge to narrow the *international* achievement gap between America and it international competitors by reforms such a decreasing class sizes might thereby widen the *domestic* achievement gap that separates black and white, rich and poor. But, in my opinion, the new administration also needs to be aware that a major cause of the domestic achievement gap is POVERTY, as forcefully argued by David Berliner. I submit that: (a) reduction of poverty in the U.S. might reduce *both* the domestic and international achievement gaps, since then the learning of *all* students, not just the advantaged, would be markedly increased by reforms such a reduction in class size, more effective teachers, and more effective pedagogy; (b) a more important reason for increasing student learning than narrowing the international achievement gap is the need to alleviate the science/math illiteracy of the general population and thereby lessen the threat to life on planet Earth.
*******************************************************
To access the entire post please click on http://tinyurl.com/daf85y .
REFERENCES
Hake, R.R. 2009. "A Response to 'It's Not All About Class Size', " AERA-L post of 6 February 2009 09:42:04-0800; online at http://tinyurl.com/daf85y .
****************************************************
Stephen Ceci and Spryos Konstantopoulos in a "Chronicle of Higher Education" commentary "It's Not All About Class Size" point out that the new administration needs to be aware that attempts to fulfill its pledge to narrow the *international* achievement gap between America and it international competitors by reforms such a decreasing class sizes might thereby widen the *domestic* achievement gap that separates black and white, rich and poor. But, in my opinion, the new administration also needs to be aware that a major cause of the domestic achievement gap is POVERTY, as forcefully argued by David Berliner. I submit that: (a) reduction of poverty in the U.S. might reduce *both* the domestic and international achievement gaps, since then the learning of *all* students, not just the advantaged, would be markedly increased by reforms such a reduction in class size, more effective teachers, and more effective pedagogy; (b) a more important reason for increasing student learning than narrowing the international achievement gap is the need to alleviate the science/math illiteracy of the general population and thereby lessen the threat to life on planet Earth.
*******************************************************
To access the entire post please click on http://tinyurl.com/daf85y .
REFERENCES
Hake, R.R. 2009. "A Response to 'It's Not All About Class Size', " AERA-L post of 6 February 2009 09:42:04-0800; online at http://tinyurl.com/daf85y .
Subscribe to:
Posts (Atom)
