What Intelligence Tests Miss
WHAT INTELLIGENCE TESTS MISS
What Intelligence Tests Miss
The Psychology of Rational Thought
KEITH E. STANOVICH
Published with assistance from the Mary cady Tew Memorial Fund.
Copyright © 2009 by Keith Stanovich.
All rights reserved.
This book may not be reproduced, in whole or in part, including illustrations, in any form (beyond that copying permitted by Sections 107 and 108 of the U.S. Copyright Law and except by reviewers for the public press), without written permission from the publishers.
Set in Electra type by Tseng Information Systems, Inc..
Printed in the United States of America.
Library of Congress Cataloging-in-Publication
Data Stanovich, Keith E., 1950–
What intelligence tests miss : the psychology of rational thought / Keith E. Stanovich.
p. cm.
Includes bibliographical references and index.
ISBN 978-0-300-12385-2 (hardcover : alk. paper) 1. Intelligence tests.
2. Thought and thinking. I. Title.
BF431.s687 2009
153.9—dc22 2008037325
A Catalogue record for this book is available from the British library.
This paper meets the requirements of ANSI/NISO Z39.48-1992 (Permanence of Paper).
It contains 30 percent postconsumer waste (PCW) and is certified
by the Forest Stewardship council (FSC).
10 9 8 7 6 5 4 3 2 1
For Paula,
who has never measured a person’s worth in IQ points
CONTENTS
Preface
Acknowledgments
ONE
Inside George W. Bush’s Mind: Hints at What IQ Tests Miss
TWO
Dysrationalia: Separating Rationality and Intelligence
THREE
The Reflective Mind, the Algorithmic Mind, and the Autonomous Mind
FOUR
Cutting Intelligence Down to Size
FIVE
Why Intelligent People Doing Foolish Things Is No Surprise
SIX
The Cognitive Miser: Ways to Avoid Thinking
SEVEN
Framing and the Cognitive Miser
EIGHT
Myside Processing: Heads I Win—Tails I Win Too!
NINE
A Different Pitfall of the Cognitive Miser: Thinking a Lot, but Losing
TEN
Mindware Gaps
ELEVEN
Contaminated Mindware
TWELVE
How Many Ways Can Thinking Go Wrong? A Taxonomy of Irrational Thinking Tendencies and Their Relation to Intelligence
THIRTEEN
The Social Benefits of Increasing Human Rationality—and Meliorating Irrationality
Notes
Bibliography
Index
PREFACE
In 2002, cognitive scientist Daniel Kahneman of Princeton University won the Nobel Prize in Economics for work done with his longtime collaborator Amos Tversky (who died in 1996). The press release for the award from the Royal Swedish Academy of Sciences drew attention to the roots of the award-winning work in “the analysis of human judgment and decision-making by cognitive psychologists.” Kahneman was cited for discovering “how human judgment may take heuristic shortcuts that systematically depart from basic principles of probability. His work has inspired a new generation of researchers in economics and finance to enrich economic theory using insights from cognitive psychology into intrinsic human motivation.”
In short, Kahneman and Tversky’s work was about how humans make choices and assess probabilities, and they uncovered some very basic errors that are typical in decision making. Their work includes some of the most influential and highly cited studies in all of psychology, and it deserved to be honored with the Nobel Prize. One reason that this work was so influential was that it addressed deep issues concerning human rationality. As the Nobel announcement noted, “Kahneman and Tversky discovered how judgment under uncertainty systematically departs from the kind of rationality postulated in traditional economic theory.” The thinking errors uncovered by Kahneman and Tversky are thus not trivial errors in a parlor game. Being rational means acting to achieve one’s own life goals using the best means possible. To violate the thinking rules examined by Kahneman and Tversky thus has the practical consequence that we are less satisfied with our lives than we might be.
The work of Kahneman and Tversky, along with that of many other investigators, has shown how the basic architecture of human cognition makes all of us prone to these errors of judgment and decision making. But being prone to these errors does not mean that we always make them. Every person, on some occasions, overrides the tendency to make these reasoning errors and instead makes the rational response. It is not that we always make errors all the time. Even more important, it has been shown that there are systematic differences among individuals in the tendency to make errors of judgment and decision making. My own research group has tried to find out what predicts these individual differences.
The fact that there are systematic individual differences in the judgment and decision-making situations studied by Kahneman and Tversky means that there are variations in important attributes of human cognition related to rationality—how efficient we are in achieving our goals. It is a curious fact that none of these critical attributes of human thinking are assessed on IQ tests (or their proxies such as the SAT test). This fact is curious for two related reasons. First, most laypeople are prone to think that IQ tests are tests of, to put it colloquially, good thinking. Scientists and laypeople alike would tend to agree that “good thinking” encompasses good judgment and decision making—the type of thinking that helps us achieve our goals. In fact, the type of “good thinking” that Kahneman and Tversky studied was deemed so important that research on it was awarded the Nobel Prize. Yet assessments of such good thinking are nowhere to be found on IQ tests.
A second, and related, point is that when people use the term intelligence (again, laypersons and psychologists alike), they often talk as if the concept of intelligence encompassed rationality. For example, many conceptions of intelligence define it as involving adaptive decision making. Adaptive decision making is the quintessence of rationality, but the items used to assess intelligence on widely accepted tests bear no resemblance to measures of rational decision making. This creates some curious phenomena that we do in fact tend to notice. We do tend to notice, and to find mildly perplexing, “smart people doing dumb things.” But the way that we have historically measured intelligence makes this phenomenon not perplexing at all. If by smart we mean IQ-test smart and by dumb we mean poor decision making, then the source of the phenomenon is clear. IQ tests do not measure adaptive decision making. So if we are surprised at a high-IQ person acting foolishly, it can only mean that we think that all good mental attributes must co-occur with high intelligence—in this case, that rational thinking must go with high intelligence. However, research is increasingly bringing this assumption into question. Rational thinking skills of the type studied by Kahneman and Tversky show only small-to-medium correlations with intelligence test performance—not surprisingly, because tests of the latter make no direct assessment of the former.
In the present book, I explore the issue of whether they should. Judgment and decision-making skills—the skills of rational thought—are at least as important as the attributes that are assessed on IQ tests. Like intelligence, rational thinking skills relate to goal achievement in the real world. Yet we fail to teach them in schools or to focus our attention on them as a society. Instead, we keep using intelligence prox
ies as selection devices in educational institutions from exclusive preschools to graduate schools. corporations and the military are likewise excessively focused on IQ measures. The lavish attention devoted to intelligence (raising it, praising it, worrying when it is low, etc.) seems wasteful when we virtually ignore another set of mental skills with just as much social consequence.
The thinking skills studied by Kahneman and Tversky cash out in terms of real-world behaviors that affect people’s happiness and well-being. They are just as important as the cognitive skills assessed on IQ tests. Intelligence tests are thus radically incomplete as measures of cognitive functioning. Because of their vast influence, IQ tests have both explicitly and implicitly defined, for the layperson and psychologist alike, what cognitive attributes to value. These are important abilities, to be sure, but the tests leave out huge domains of cognitive functioning. We do not need to stretch to noncognitive domains—to notions such as emotional intelligence or social intelligence—to see important lacunae in the tests. That would be implicitly conceding too much. It would seem to concede that the tests cover the cognitive domain quite well and that we need to go outside the cognitive domain, or at least straddle it (into things like emotion, creativity, aesthetic sensibility, interpersonal skills) in order to find things that IQ tests miss. I believe we need not look so far afield. The skills of judgment and decision making are cognitive skills that are the foundation of rational thought and action, and they are missing from IQ tests.
This book, then, is an extended meditation on the scientific and social consequences of a historical irony of the behavioral sciences: The Nobel Prize was awarded for studies of cognitive characteristics that are entirely missing from the most well-known mental assessment device in the behavioral sciences—the intelligence test.
ACKNOWLEDGMENTS
My intellectual debts in writing this book are immense and are represented in the wide literature that I cite. Nonetheless, I would single out several foundational influences. Decades ago, the work of Daniel Kahneman and Amos Tversky inspired my interest in rational thinking tasks that were new to psychology at the time. More recently, the work of Jonathan Evans and David Over provoked me to make my own contributions to dual-process theory. I have long admired Jonathan Baron’s use of the heuristics and biases literature to illuminate public policy issues. I thank David Perkins for coining the term mindware, which I have used liberally in this book. From the standpoint of my interest in individual differences in cognitive functioning, the work of Robert Sternberg has been influential. The key points in several chapters of this book were inspired by his theoretical and empirical contributions. I say this in full awareness that he will dislike several of my arguments. Nevertheless, I thank him for his relentless probing of the intelligence construct and his willingness to mix it up with me over the dysrationalia concept a decade ago.
My literary agent, Susan Arellano, is thanked for her patience in helping me work out what would be the central themes of this book. She aided me greatly in seeing what should be central and what should be peripheral. My editor at Yale University Press, Keith Condon, is thanked for his enthusiasm for the project and for making important structural suggestions for the book. Susan Laity was very helpful with manuscript editing, and Katherine Scheuer did wonderful work with the copyediting.
This book was crafted in many places with great views: in my ninth-floor office overlooking downtown Toronto and Lake Ontario; in St. Ives, Cornwall, overlooking the Atlantic; and on the Oregon coast overlooking the Pacific. Two people have been with me in all of these settings. Richard West, my colleague of thirty years, has been a continuous sounding board for these ideas. Many nights on my balcony in Toronto and down in the clearing have done wonders for my morale. Paula Stanovich has been the shining light behind all of the work that I do. She has helped me build the life that made this work possible.
David Over of the University of Durham, Maggie Toplak of York University, and an anonymous reviewer read the entire manuscript and provided many discerning comments. Three conferences were seminal in allowing me to discuss these ideas at length: the Fourth International Thinking Conference in Durham, England; the Conference on Dual-Process Theories of Reasoning and Rationality in Cambridge, England, organized by Jonathan Evans and Keith Frankish; and a workshop on dual-process theory at the University of Virginia organized by Tim Wilson and Jonathan Evans.
The Chairs of my Department while this book was being written, Janet Astington and Esther Geva, and my Deans, Michael Fullan and Jane Gaskell, provided great administrative support for this work. Mary Macri, our Department’s business officer, took care of my technical and logistical needs with extraordinary dedication, as did my secretaries Diana Robinson and Marisa Freire. My empirical research on some of the issues discussed in this volume was made possible by support received from the Social Sciences and Humanities Research Council of Canada and by the Canada Research Chairs program. Marilyn Kertoy and Anne Cunningham are always part of my personal and intellectual support team.
Most members of the Stanovich/West lab (a joint lab linking the University of Toronto and James Madison University) in the past decade have contributed in some way to the research of our own that is cited in this volume. Leaders in the lab now well into their post-Ph.D. Careers are represented by Caroline Ho, Robyn Macpherson, Walter sá, and Maggie Toplak. Other lab members thanked for their participation are Maria Grunewald, Carol kelley, Judi kokis, eleanor Liu, Russ Meserve, laura Page, George Potworowski, Jason Riis, Rachel Ryerson, Robin Sidhu, Ron Stringer, Rebecca Wells-Jopling, and Joan Wolforth.
ONE
Inside George W. Bush’s Mind: Hints at What IQ Tests Miss
I’m also not very analytical. You know I don’t spend a lot of time thinking about myself, about why I do things.
—President George W. Bush, aboard Air Force One, June 4, 2003
For years, there have been debates about George W. Bush’s intelligence. His many opponents never seem to tire of pointing out his mental shortcomings. The president’s strangled syntax, goofy phrasing (“Too many good docs are getting out of the business. Too many OB-GYNs aren’t able to practice their love with women all across this country.”—Sept. 6, 2004), and lack of familiarity with many issues have been used as evidence by his opponents to argue that this is a man of truly inferior intelligence. Even Bush’s supporters often implicitly concede the point by arguing that although he lacks “school smarts” he makes up for it with “street smarts.” Therefore, it came as something of a surprise when scores on various college placement exams and Armed Forces tests that the president had taken over the years were converted into an estimated IQ score. The president’s score was approximately 120—roughly the same as that of Bush’s opponent in the 2004 presidential election, John Kerry, when Kerry’s exam results from young adulthood were converted into IQ scores using the same formulas.1
These results surprised many critics of the president (as well as many of his supporters), but I, as a scientist who studies individual differences in cognitive skills, was not surprised. Virtually all commentators on the president’s cognition, including sympathetic commentators such as his onetime speechwriter David Frum, admit that there is something suboptimal about the president’s thinking. The mistake they make is assuming that all intellectual deficiencies are reflected in a lower IQ score.
In a generally positive portrait of the president, Frum nonetheless notes that “he is impatient and quick to anger; sometimes glib, even dogmatic; often uncurious and as a result ill-informed” (2003, p. 272). Conservative commentator George Will agrees, when he states that in making Supreme Court appointments, the president “has neither the inclination nor the ability to make sophisticated judgments about competing approaches to construing the Constitution” (2005, p. 23).
In short, there is considerable agreement that President Bush’s thinking has several problematic aspects: lack of intellectual engagement, cognitive inflexibility, need for closure, belief persev
erance, confirmation bias, over-confidence, and insensitivity to inconsistency. These are all cognitive characteristics that have been studied by psychologists and that can be measured with at least some precision. However, they are all examples of thinking styles that are not tapped by IQ tests. Thus, it is not surprising that someone could suffer from many of these cognitive deficiencies and still have a moderately high IQ.
Bush’s cognitive deficiencies do not impair performance on intelligence tests, but they do impair rational decision making. His cognitive deficiencies instead are the causes of “dysrationalia” (an analogue of the word “dyslexia”), which is a term that I coined in the mid-1990s in order to draw attention to what is missing in IQ tests. I define dysrationalia as the inability to think and behave rationally despite having adequate intelligence. The president is, in fact, not unintelligent, but he may well be dysrationalic.
And he is not alone. Many people display the systematic inability to think or behave rationally despite the fact that they have more than adequate IQs. One of the reasons that many of us are dysrationalic to some extent is that, for a variety of reasons, we have come to overvalue the kinds of thinking skills that IQ tests measure and undervalue other critically important cognitive skills, such as the ability to think rationally.
Although most people would say that the ability to think rationally is a clear sign of a superior intellect, standard IQ tests devote no section to rational thinking as cognitive scientists would define the term. To think rationally means adopting appropriate goals, taking the appropriate action given one’s goals and beliefs, and holding beliefs that are commensurate with available evidence. Although IQ tests do assess the ability to focus on an immediate goal in the face of distraction, they do not assess at all whether a person has the tendency to develop goals that are rational in the first place. Likewise, IQ tests are good measures of how well a person can hold beliefs in short-term memory and manipulate those beliefs, but they do not assess at all whether a person has the tendency to form beliefs rationally when presented with evidence. And again, similarly, IQ tests are good measures of how efficiently a person processes information that has been provided, but they do not at all assess whether the person is a critical assessor of information as it is gathered in the natural environment.