Placebo Effects: Psychology’s Fundamental Flaw? Why active controls are not enough
A new paper published in Perspectives in Psychological Science (open access) suggests there is “a fundamental design flaw that potentially undermines any causal inference” in much psychology research. The paper is very accessible and well worth a read. The flowchart below describes the hypothesis:
Way back in the 1950’s a phenomenon called the Hawthorne effect (AKA the observer effect) was observed, an experiment to discover whether workers in the Hawthorne Works plant were more productive under increased lighting, found that productivity increased whenever the workers were being observed, regardless of the variable in question. The effect is a common cause of the placebo effect, which in most aspects of medicine is effectively handled by the gold standard of double blind randomised controlled trials (RCT’s). Unfortunately this is not always possible in psychology, not least because it is often impossible to blind an individual to the intervention, let alone the experimenter.
The problem is currently sometimes handled in psychology by using an active control, in which the control group are led to believe they are being experimented on too. The researchers in the new paper complain that this is not good enough, because all too often a difference remains in the participant’s expectations – an expectation effect. I’ve described studies where active controls might not be well chosen before, they are everywhere, in such cases expectation effects may be the cause of the results found, which would completely alter the conclusion of a study.
To demonstrate the severity of the problem, the researchers address video-gaming and brain-training research, not because the field is an example of poor research but because it is a rare example of psychology research that actually uses active placebos. The original research in question demonstrates a benefit in performance on vision and attention tasks following playing an action video game (Unreal Tournament) rather than Tetris or The Sims. The researchers in the new study asked people (admittedly only using Amazon’s Mechanical Turk) what improvements they would expect to see, following playing Unreal Tournament or following playing Tetris or The Sims. The result was that people expected the very same benefits as were found in the original research. More visual processing benefits were expected by participants from playing Unreal Tournament than from playing Tetris or The Sims, furthermore, more such benefits were expected by the participants from Tetris than The Sims whilst participants thought more benefits would be achieved in story-telling from The Sims and participants thought more benefits to visual rotation would be obtained from Tetris than from Unreal Tournament.
Could it be that in reality there is no real effect that can be generalised outside of the laboratory? Could it be that while they were being observed the gamers in the original experiment unconsciously let their behaviour be moulded by their own prior expectations?
If this is the case, then this is only the tip of the iceberg. Vast amounts of psychology experiments do not use active controls, let alone active controls that sufficiently avoid expectation effects. This will be no easy problem to fix, the researchers have some suggestions, many of which will likely be seen as impractical by researchers, but some of which may be taken up – the paper itself is valuable reading if you are a researcher yourself. The problem requires a major sea change in the way things are done, even with the best efforts of researchers, the problem will never go away entirely. You’d therefore be well advised to always keep the possibility of expectation effects in the back of your mind when you are reading a piece of psychology research. The question you want to ask yourself is this – these results might look very impressive on paper, but what are they compared to and will they still apply to me in my living room rather than to someone who knows they are being experimented on, and might think they know why, in a far away laboratory under the watchful gaze of a scientist in a metaphorical white coat?
The paper has also been well discussed in a blog post by Daniel Simons one of the paper’s authors, as well as a post by Ed Yong, a post by Micah Allen and an article in the Washington Post by Daniel Willingham. On the project’s own Open Science Framework page you can even do the experiment as if you were a participant yourself!
Reference:
Boot W.R., Simons D.J., Stothart C. & Stutts C. (2013). The Pervasive Problem With Placebos in Psychology: Why Active Control Groups Are Not Sufficient to Rule Out Placebo Effects, Perspectives on Psychological Science, 8 (4) 445-454. DOI: 10.1177/1745691613491271 (PDF)
To keep up to date with this blog you can follow Neurobonkers on Twitter, Facebook, RSS or join the mailing list.
Image Credit: Shutterstock/Lawrence Wee
