{"id":192,"date":"2018-06-20T23:09:04","date_gmt":"2018-06-20T22:09:04","guid":{"rendered":"https:\/\/skepticink.com\/onusloom\/?p=192"},"modified":"2018-06-20T23:09:04","modified_gmt":"2018-06-20T22:09:04","slug":"fraudulent-psychology-stanford-prison-experiment-and-others","status":"publish","type":"post","link":"https:\/\/skepticink.com\/onusloom\/2018\/06\/20\/fraudulent-psychology-stanford-prison-experiment-and-others\/","title":{"rendered":"Fraudulent Psychology? Stanford Prison Experiment and Others&#8230;"},"content":{"rendered":"<p style=\"text-align: justify;\">[This is from A Tippling Philosopher, posted here due to reference to an Onus title].<\/p>\n<p style=\"text-align: justify;\">Experimental psychology has come in for a rough time in the last few years. Initially, back in 2015, a meta-study cam out that supposedly showed a replication crisis in psychology and that many studies over the years <a href=\"https:\/\/www.nytimes.com\/2015\/08\/28\/science\/many-social-science-findings-not-as-strong-as-claimed-study-says.html?_r=0\">were not as strong as claimed<\/a>. This caused a right stir. And then came <a href=\"http:\/\/www.slate.com\/blogs\/the_slatest\/2016\/03\/03\/psychology_study_that_induced_the_reproducibility_crisis_was_wrong.html?via=gdpr-consent\">a study of that study that showed<\/a> that the meta-analysis was in fact very poorly done and that the initial studies probably were alright:<\/p>\n<section class=\"content \">\n<div class=\"newbody body parsys\">\n<div class=\"text text-1 parbase section\">\n<blockquote>\n<p style=\"text-align: justify;\">Remember\u00a0<a href=\"http:\/\/www.nytimes.com\/2015\/08\/28\/science\/many-social-science-findings-not-as-strong-as-claimed-study-says.html?_r=0\" target=\"_blank\" rel=\"noopener\">that study that found<\/a>\u00a0that most psychology studies were wrong? Yeah, that study was wrong. That\u2019s the conclusion of four researchers who recently interrogated the methods of that study, which itself interrogated the methods of 100 psychology studies to find that very few could be replicated. (Whoa.) Their damning\u00a0<a href=\"http:\/\/science.sciencemag.org\/content\/351\/6277\/1037.2\" target=\"_blank\" rel=\"noopener\">commentary<\/a>\u00a0will be published Friday in the journal\u00a0<a href=\"http:\/\/www.sciencemag.org\/journals\" target=\"_blank\" rel=\"noopener\"><em>Science<\/em><\/a>. (The scientific body that publishes the journal sent\u00a0<strong><em>Slate\u00a0<\/em><\/strong>an early copy.)<\/p>\n<\/blockquote>\n<\/div>\n<div class=\"text-2 text parbase section\">\n<blockquote>\n<p style=\"text-align: justify;\">In case you missed the hullabaloo: A key feature of the scientific method is that scientific results should be reproducible\u2014that is, if you run an experiment again, you should get the same results. If you don\u2019t, you\u2019ve got a problem. And a problem is exactly what 270 scientists found last August, when they decided to\u00a0<a href=\"http:\/\/science.sciencemag.org\/content\/349\/6251\/aac4716\" target=\"_blank\" rel=\"noopener\">try to reproduce 100 peer-reviewed journal studies<\/a>\u00a0in the field of social psychology. Only around 39 percent of the reproduced studies, they found, came up with similar results to the originals.<\/p>\n<div class=\"text-3 text parbase section\">\n<p style=\"text-align: justify;\">That meta-analysis, published in\u00a0<em>Science<\/em>\u00a0by a group called the\u00a0<a href=\"http:\/\/centerforopenscience.github.io\/osc\/pages\/about.html\" target=\"_blank\" rel=\"noopener\">Open Science Collaboration<\/a>, led to\u00a0<a href=\"http:\/\/www.nature.com\/news\/over-half-of-psychology-studies-fail-reproducibility-test-1.18248\" target=\"_blank\" rel=\"noopener\">mass hand-wringing<\/a>\u00a0over the \u201creplicability crisis\u201d in psychology. (It wasn\u2019t the first time that the field has faced such criticism, as Michelle N. Meyer and Christopher Chabris have\u00a0<a href=\"http:\/\/www.slate.com\/articles\/health_and_science\/science\/2014\/07\/replication_controversy_in_psychology_bullying_file_drawer_effect_blog_posts.html\">reported in\u00a0<em><strong>Slate<\/strong><\/em><\/a>, but this particular study was a doozy.)<\/p>\n<\/div>\n<div class=\"text parbase text-4 section\">\n<p style=\"text-align: justify;\">Now this new commentary, from Harvard\u2019s\u00a0<a href=\"http:\/\/gking.harvard.edu\/\" target=\"_blank\" rel=\"noopener\">Gary King<\/a>\u00a0and Daniel Gilbert and the University of Virginia\u2019s Timothy Wilson, finds that the OSC study was bogus\u2014for a dazzling array of reasons.<\/p>\n<\/div>\n<\/blockquote>\n<p style=\"text-align: justify;\">Well, it appears that the crisis is not yet over. Studies such as the hugely influential Stanford Prison Experiment and the Marshmallow Experiment have supposedly, again, been called into question. I, for one, find the latter to be intriguing because I am under the impression that the marshmallow experiment has been replicated a large number of times, and measuring for different things (from SAT scores\u00a0to Body Mass Index and so on).<\/p>\n<p>In the case of the Stanford Prison Experiment, <a href=\"https:\/\/www.vox.com\/2018\/6\/13\/17449118\/stanford-prison-experiment-fraud-psychology-replication\">it was apparently based, to some degree, on deceit<\/a>:<\/p>\n<blockquote>\n<p id=\"m7VLu5\" style=\"text-align: justify;\">But its findings were wrong. Very wrong. And not just due to its questionable ethics\u00a0<a href=\"https:\/\/twitter.com\/david_m_amodio\/status\/1006958007983460361\">or lack of concrete data<\/a>\u00a0\u2014 but because of deceit.<\/p>\n<p id=\"cmyBh2\" style=\"text-align: justify;\">A new\u00a0<a href=\"https:\/\/medium.com\/s\/trustissues\/the-lifespan-of-a-lie-d869212b1f62\">expos\u00e9<\/a>\u00a0published by Medium based on previously unpublished\u00a0<a href=\"https:\/\/www.vox.com\/science-and-health\/2018\/6\/14\/17464516\/stanford-prison-experiment-audio\">recordings<\/a>\u00a0of Philip Zimbardo, the Stanford psychologist who ran the study, and interviews with his participants,\u00a0<a href=\"https:\/\/medium.com\/s\/trustissues\/the-lifespan-of-a-lie-d869212b1f62\">offers<\/a>\u00a0convincing evidence that the guards in the experiment were coached to be cruel. It also shows that the experiment\u2019s most memorable moment \u2014 of a prisoner descending into a screaming fit, proclaiming, \u201cI\u2019m burning up inside!\u201d \u2014 was the result of the prisoner acting. \u201cI took it as a kind of an improv exercise,\u201d one of the guards told reporter\u00a0<a href=\"https:\/\/twitter.com\/benzblum?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor\">Ben Blum<\/a>. \u201cI believed that I was doing what the researchers wanted me to do.\u201d<\/p>\n<p id=\"QaLj0J\" style=\"text-align: justify;\">The findings have long been subject to scrutiny \u2014 many think of them\u00a0<a href=\"https:\/\/twitter.com\/david_m_amodio\/status\/1006958007983460361\">as more of a dramatic demonstration<\/a>, a sort-of academic reality show, than a serious bit of science. But these new revelations incited an immediate response. \u201cWe must stop celebrating this work,\u201d personality psychologist Simine Vazire\u00a0<a href=\"https:\/\/twitter.com\/siminevazire\/status\/1006070517772607488\">tweeted<\/a>, in response to the\u00a0<a href=\"https:\/\/medium.com\/s\/trustissues\/the-lifespan-of-a-lie-d869212b1f62\">article<\/a>. \u201cIt\u2019s anti-scientific. Get it out of textbooks.\u201d Many other psychologists have expressed similar sentiments.<\/p>\n<p id=\"c8SIec\" style=\"text-align: justify;\">Many of the classic show-stopping experiments in psychology have lately turned out to be wrong, fraudulent, or outdated. And in recent years, social scientists have begun to reckon with the truth that their old work needs a redo, the \u201c<a href=\"https:\/\/www.vox.com\/2016\/3\/14\/11219446\/psychology-replication-crisis\">replication crisis<\/a>.\u201d But there\u2019s been a lag \u2014 in the popular consciousness and in how psychology is taught by teachers and textbooks. It\u2019s time to catch up.<\/p>\n<\/blockquote>\n<p style=\"text-align: justify;\">The Guardian <a href=\"https:\/\/www.theguardian.com\/science\/2018\/apr\/16\/a-real-life-lord-of-the-flies-the-troubling-legacy-of-the-robbers-cave-experiment\">recently ran an interesting piece<\/a> on how the Robbers Cave experiment from the 1950s was a &#8220;do-over from a failed previous version of an experiment, which the scientists never mentioned in an academic paper&#8221;. Next one on the list is surely the Milgram Experiment&#8230;:<\/p>\n<blockquote>\n<p id=\"TYA1u7\" style=\"text-align: justify;\"><a href=\"http:\/\/journals.sagepub.com\/doi\/10.1177\/1948550617693060\">And<\/a>\u00a0it seems like Milgram\u2019s conclusions may hold up: In a recent study, many people found demands from\u00a0<a href=\"http:\/\/journals.sagepub.com\/doi\/10.1177\/1948550617693060\">an authority figure to be a compelling<\/a>\u00a0reason to shock another. However, it\u2019s possible, due to something known as the file-drawer effect, that failed replications of the Milgram experiment have not been published. Replication attempts at the Stanford prison study, on the other hand,\u00a0<a href=\"http:\/\/www.bbcprisonstudy.org\/pdfs\/bjsp(2006)tyrannny.pdf\">have been a mess<\/a>.<\/p>\n<p id=\"dTlTu4\" style=\"text-align: justify;\">In science, too often, the first demonstration of an idea becomes the lasting one \u2014 in both pop culture and academia. But this isn\u2019t how science is supposed to work at all!<\/p>\n<\/blockquote>\n<p>Where there is found to be replication, often the results, so it is claimed, are less robust, are weaker.<\/p>\n<p>For example, let&#8217;s look at the Marshmallow Test, one that I often refer to in my talks on free will. As Vox <a href=\"https:\/\/www.vox.com\/science-and-health\/2018\/6\/6\/17413000\/marshmallow-test-replication-mischel-psychology\">reports<\/a>:<\/p>\n<blockquote>\n<p id=\"WzjWxp\" style=\"text-align: justify;\">While successes at the marshmallow test at age 4 did predict achievement at age 15, the size of the correlation was half that of the original paper. And the correlation almost vanished when Watts and his colleagues controlled for factors like family background and intelligence.<\/p>\n<p id=\"CRzq22\" style=\"text-align: justify;\">That means \u201cif you have two kids who have the same background environment, they get the same kind of parenting, they are the same ethnicity, same gender, they have a similar home environment, they have similar early cognitive ability,\u201d Watts says. \u201cThen if one of them is able to delay gratification, and the other one isn\u2019t, does that matter? Our study says, \u2018Eh, probably not.\u2019\u201d<\/p>\n<p id=\"lxaBJv\" style=\"text-align: justify;\">In other words: Delay of gratification is not a unique lever to pull to positively influence other aspects of a person\u2019s life. It\u2019s a consequence of bigger-picture, harder-to-change components of a person, like their intelligence and environment they live in.<\/p>\n<p id=\"QyCIxN\" style=\"text-align: justify;\">The results imply that if you can teach a kid to delay gratification, it won\u2019t necessarily lead to benefits later on. Their background characteristics have already put them on that path.<\/p>\n<p id=\"0L8dKx\" style=\"text-align: justify;\">What\u2019s more, the study found no correlation \u2014 even without controls \u2014 between delaying gratification and behavioral outcomes later in life. \u201cIn that sense, that\u2019s the one piece of the paper that\u2019s really a failure to replicate,\u201d Watts says.<\/p>\n<p id=\"m9G7U8\" style=\"text-align: justify;\">His paper also found something that they still can\u2019t make sense of. Most of the predictive power of the marshmallow test can be accounted for kids just making it 20 seconds before they decide to eat the treat. \u201cSo being able to wait for two minutes, five minutes, or seven minutes, the max, it didn\u2019t really have any additional benefits over being able to wait for 20 seconds.\u201d<\/p>\n<p id=\"nVuOtC\" style=\"text-align: justify;\">That makes it hard to imagine the kids are engaging in some sort of complex cognitive trick to stay patient, and that the test is revealing something deep and lasting about their potential in life. And perhaps it\u2019s an indication that the marshmallow experiment is not a great test of delay of gratification or some other underlying measure of self-control.<\/p>\n<p id=\"IsRpDr\" style=\"text-align: justify;\">Their study doesn\u2019t completely reverse the finding of the original marshmallow paper. But it reduces the findings to a point where it\u2019s right to wonder if they have any practical meaning.<\/p>\n<\/blockquote>\n<p>Being a skeptic is tough work. You can never rest on your laurels. I&#8217;m going to have to look in, more depth at these and other psychological experiments that I have for so long taken for granted.<\/p>\n<p>Bad ideas do persist in psychology, that is for sure. It&#8217;s why I edited Caleb Lack&#8217;s fascinating\u00a0<em><a href=\"https:\/\/www.amazon.com\/Psychology-Gone-Astray-Literature-Psychological\/dp\/0992600030\/ref=as_sl_pc_ss_til?tag=atipplingphil-20&amp;linkCode=w00&amp;linkId=7SZJ6HCDE2XYLITF&amp;creativeASIN=0992600030\">Psychology Gone Astray:\u00a0<span id=\"productTitle\" class=\"a-size-large\">A Selection of Racist &amp; Sexist Literature from Early Psychological Research<\/span><\/a>\u00a0<\/em>that looked at the biases in early psychology and the destructive power of preconceived ideals.<\/p>\n<p><a href=\"https:\/\/www.amazon.com\/Psychology-Gone-Astray-Literature-Psychological\/dp\/0992600030\/ref=as_sl_pc_ss_til?tag=atipplingphil-20&amp;linkCode=w00&amp;linkId=7SZJ6HCDE2XYLITF&amp;creativeASIN=0992600030\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-8544\" src=\"http:\/\/wp.production.patheos.com\/blogs\/tippling\/files\/2016\/07\/PsychologyGoneAstray.jpg\" alt=\"\" width=\"333\" height=\"499\" \/><\/a><\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n<\/div>\n<\/div>\n<\/section>\n","protected":false},"excerpt":{"rendered":"<p>[This is from A Tippling Philosopher, posted here due to reference to an Onus title]. Experimental psychology has come in&hellip;<\/p>\n","protected":false},"author":80,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5,13,24],"tags":[22,14],"class_list":["post-192","post","type-post","status-publish","format-standard","hentry","category-dr-caleb-lack","category-dr-charles-abramson","category-psychology","tag-psychology","tag-psychology-gone-astray"],"_links":{"self":[{"href":"https:\/\/skepticink.com\/onusloom\/wp-json\/wp\/v2\/posts\/192","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/skepticink.com\/onusloom\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/skepticink.com\/onusloom\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/skepticink.com\/onusloom\/wp-json\/wp\/v2\/users\/80"}],"replies":[{"embeddable":true,"href":"https:\/\/skepticink.com\/onusloom\/wp-json\/wp\/v2\/comments?post=192"}],"version-history":[{"count":0,"href":"https:\/\/skepticink.com\/onusloom\/wp-json\/wp\/v2\/posts\/192\/revisions"}],"wp:attachment":[{"href":"https:\/\/skepticink.com\/onusloom\/wp-json\/wp\/v2\/media?parent=192"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/skepticink.com\/onusloom\/wp-json\/wp\/v2\/categories?post=192"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/skepticink.com\/onusloom\/wp-json\/wp\/v2\/tags?post=192"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}