What Intelligence Tests Miss Read online

  For example, in a series of studies, developmental psychologist Paul Klaczynski has shown that when evaluating evidence, if subjects are not given explicit instructions to decontextualize—that is, to set aside their prior opinion—there is little correlation between intelligence and the tendency to reason in an unbiased manner.21 My research group has produced evidence consistent with this finding. In one study, Maggie Toplak and I had subjects generate arguments relevant to a controversial issue (should people be allowed to sell their internal organs?). We also assessed where individuals stood on the issues in question. We found a substantial myside bias on the task (people tended to give more arguments in favor of their position than against), but the degree of myside bias was not correlated with cognitive ability.

  In short, our research converges with that of other researchers in indicating that in informal reasoning situations where people are not told to put aside their prior beliefs, intelligence is unrelated to the tendency to reason in an unbiased manner. That such ambiguous situations (without explicit instructions to be unbiased) are common in real life means that the literature might actually be overestimating the contribution of intelligence to rationality because many tasks in the experimental literature contain explicit instructions on the task requirements and how to reason in order to fulfill them. More intelligent people appear to reason better only when you tell them in advance what good thinking is! This makes little sense given the structure in Figure 3.2. It becomes more explicable from within the expanded model presented in Figure 3.4.

  The override capacity is a property of the algorithmic mind, and it is indicated by the arrow labeled A in Figure 3.4. However, previous dual-process theories have tended to ignore the higher-level cognitive function that initiates the override function in the first place. This is a dispositional property of the reflective mind that is related to rationality. In the model in Figure 3.4, it is represented by arrow B, which represents, in machine intelligence terms, the call to the algorithmic mind to override the Type 1 response by taking it offline. This is a different mental function from the override function itself (arrow A), and I have presented evidence indicating that the two functions are indexed by different types of individual differences—the ability to sustain the inhibition of the Type 1 response is indexed by measures of fluid intelligence, and the tendency to initiate override operations is indexed by thinking dispositions such as reflectiveness and need for cognition.

  Figure 3.4. A More complete Model of the Tripartite Framework

  Figure 3.4 represents another aspect of cognition somewhat neglected by previous dual-process theories. Specifically, the override function has loomed large in dual-process theory, but less so the simulation process that computes the alternative response that makes the override worthwhile. Figure 3.4 explicitly represents the simulation function as well as the fact that the call to initiate simulation originates in the reflective mind. The decoupling operation (indicated by arrow C) itself is carried out by the algorithmic mind and the call to initiate simulation (indicated by arrow D) by the reflective mind. Again, two different types of individual differences are associated with the initiation call and the decoupling operator—specifically, rational thinking dispositions with the former and fluid intelligence with the latter. Finally, the algorithmic mind receives inputs from the computations of the autonomous mind via so-called preattentive processes (arrow E).

  Don’t Forget the Mindware!

  The term mindware was coined by Harvard cognitive scientist David Perkins to refer to the rules, knowledge, procedures, and strategies that a person can retrieve from memory in order to aid decision making and problem solving.22 Perkins uses the term to stress the analogy to software in the brain/computer analogy. Each of the levels in the tripartite model of mind has to access knowledge to carry out its operations, as illustrated in Figure 3.5. As the figure indicates, the reflective mind not only accesses general knowledge structures but, importantly, accesses the person’s opinions, beliefs, and reflectively acquired goal structure. The algorithmic mind accesses micro-strategies for cognitive operations and production system rules for sequencing behaviors and thoughts. Finally, the autonomous mind not only accesses evolutionarily compiled encapsulated knowledge bases, but also retrieves information that has become tightly compiled and available to the autonomous mind due to overlearning and practice.

  It is important to note that what is displayed in Figure 3.5 are the knowledge bases that are unique to each mind. Algorithmic- and reflective-level processes also receive inputs from the computations of the autonomous mind (see arrow E in Figure 3.4). The mindware available for retrieval, particularly that available to the reflective mind, is in part the product of past learning experiences. And here we have a direct link to the Cattell/Horn/Carroll theory of intelligence mentioned earlier. The knowledge structures available for retrieval by the reflective mind represent Gc, crystallized intelligence (intelligence-as-knowledge). Recall that Gf, fluid intelligence (intelligence-as-process), is already represented in the figure. It is the general computational power of the algorithmic mind—importantly exemplified by the ability to sustain cognitive decoupling.

  The Gf/Gc theory is the most comprehensive theory of intelligence available that has extensive scientific validation. It is thus important to see how both of its major components miss critical aspects of rational thought. Fluid intelligence will, of course, have some relation to rationality because it indexes the computational power of the algorithmic mind to sustain decoupling. Because override and simulation are important operations for rational thought, Gf will definitely facilitate rational action in some situations. Nevertheless, the tendency to initiate override (arrow B in Figure 3.4) and to initiate simulation activities (arrow D in Figure 3.4) are both aspects of the reflective mind unassessed by intelligence tests, so the tests will miss these components of rationality.

  Figure 3.5. Knowledge structures in the Tripartite Framework

  The situation with respect to Gc is a little different. It is true that much of the mindware of rational thought would be classified as crystallized intelligence in the abstract. But is it the kind of crystallized knowledge that is specifically assessed on the tests? The answer is no. The mindware of rational thought is somewhat specialized mindware (it clusters in the domains of probabilistic reasoning, causal reasoning, and scientific reasoning, as I will discuss in later chapters). In contrast, the crystallized knowledge assessed on IQ tests is deliberately designed to be nonspecialized. The designers of the tests, in order to make sure the sampling of Gc is fair and unbiased, explicitly attempt to broadly sample vocabulary, verbal comprehension domains, and general knowledge. The broad sampling insures unbiasedness in the test, but it inevitably means that the specific knowledge bases critical to rationality will go unassessed. In short, Gc, as traditionally measured, does not assess individual differences in rationality, and Gf will do so only indirectly and to a mild extent.

  With this discussion of mindware, we have established that rationality requires three different classes of mental characteristic. First, algorithmic-level cognitive capacity is needed in order that override and simulation activities can be sustained. Second, the reflective mind must be characterized by the tendency to initiate the override of suboptimal responses generated by the autonomous mind and to initiate simulation activities that will result in a better response. Finally, the mindware that allows the computation of rational responses needs to be available and accessible during simulation activities. Intelligence tests assess only the first of these three characteristics that determine rational thought and action. As measures of rational thinking, they are radically incomplete.

  Scoping Out the President’s Brain

  Now that we have the sketch of a tripartite model of mind on the table, we can revisit the example that started this book—the thought processes of President George W. Bush. American politics is so polarized, however, that a quick caveat is needed. In such domains, people tend not to agree on the
facts of the matter. However, I would argue that at this late date—after eight years of the Bush presidency—we have such a wealth of consistent testimony and commentary that, in science, it would be called a convergence of evidence.

  In fact, no one doubts—not even the president’s supporters—that the aspects of his cognition that I will describe here are characteristic of him. His supporters, in numerous books, have described exactly these characteristics. In Chapter 1, I mentioned the characterizations of the president by David Frum and George Will, two conservative commentators not unsympathetic to many of Bush’s policies. Frum, the president’s onetime speechwriter, has a view of Bush’s intellect (“sometimes glib, even dogmatic; often uncurious and as a result ill-informed,” p. 272) that is exactly mirrored by that of John McCain, the Republican senator whom Bush defeated for the Republican nomination in 2000 but who was one of the president’s most important allies on the issue of the war in Iraq. McCain was asked if Bush ever asks his opinion. McCain replied, “No, no, he hasn’t. As a matter of fact he’s not intellectually curious” (Woodward, 2006, p. 419). Reporters Evan Thomas and Richard Wolffe in Newsweek magazine fill in other parts of the pattern in their coverage of how Bush handled the war in Iraq. One of our senior officials in Baghdad had observed Bush in various videoconferences and noticed how the president’s “obvious lack of interest in long, detailed discussions, had a chilling effect” (p. 37). The reporters note, “by all accounts, he is not intellectually curious. Occasionally outsiders brought into the Bush Bubble have observed that faith, not evidence, is the basis for decision making” (p. 37). Numerous other commentators echo these descriptions.23

  Nonetheless, many of the same commentators who criticize President Bush’s thinking insist that he does not lack intelligence. Ronald McCallum, a friend of Bush’s from Yale, says that Bush was “extraordinarily intelligent, but was not interested in learning unless it had practical value” (Kessler, 2004, p. 27). The prime minister of Great Britain, Tony Blair, found that Bush had a quality that President Clinton had lacked, reliability, and Blair repeatedly told his associates that Bush was “very bright” (Barnes, 2006, p. 56).

  What Blair is referring to with the phrase “very bright” is the same thing—fluid intelligence (Gf)—that allowed Bush to do well in his youth on tests that were intelligence proxies.24 However, the presence of that fluid intelligence did not prevent him from displaying irrational thought tendencies well known to psychologists. The president has only one of the three characteristics that determine rational thought—algorithmic-level cognitive capacity. He lacks two essential factors—the mindware that supports rational action and the thinking dispositions of the reflective mind that support rational thought. In fact, his case shows how important are the intellectual qualities that IQ tests leave out.

  There are published and well-investigated scales or tasks for most of the thinking dispositions suggested in the characterizations of George Bush’s thinking. The technology of rationality assessment is so far advanced that we could imagine, for example, testing President Bush (long before his presidency) and in fact predicting exactly the thinking attributes that are now known to be so tellingly characteristic of the him. Using the terms for the actual scales and tasks in the literature, formal tests of rational thinking might have revealed that the president is: overconfident; low in typical intellectual engagement; low in openness to experience; high in belief perseverance; high in confirmation bias; high in faith in intuition; high in impulsiveness; high in one-sided thinking; low in need for cognition; low in openness to experience; does not engage in counterfactual thinking; treats beliefs as possessions (has high belief identification); is high in need for closure, and low in thought flexibility.

  The directions of Bush’s score on all of these thinking dispositions is in the direction associated with lower rationality.25 Fluid intelligence provided no inoculation against a confluence of problematic intellectual traits. President Bush is an intelligent person—consistent with his test scores and consistent with what many people close to him insist. But he is not a very rational person.

  FOUR

  Cutting Intelligence Down to Size

  In U.S. society, cognitive skills have become practically equated with intellectual skills—the mental bases of intelligence. This equation is a mistake.

  —Robert J. Sternberg, Wisdom, Intelligence, and Creativity Synthesized, 2003b

  I totally agree with the epigraph from Robert Sternberg that leads this chapter. We are missing something important by treating intelligence as if it encompassed all cognitive abilities. I coined the term dysrationalia over a decade ago in order to draw attention to a large domain of cognitive life (rational thinking) that intelligence tests fail to assess. The idea that IQ tests do not measure all of the important human faculties is not new. This is precisely what broad theorists of intelligence1 have been emphasizing all these years, so in one sense I align myself with the critics who wish to stop the overvaluing of MAMBIT (the mental abilities measured by intelligence tests). However, my strategy for taming MAMBIT is different from that of critics such as Howard Gardner and Robert Sternberg.2 These critics want to broaden the term intelligence (practical intelligence, bodily-kinesthetic intelligence, etc.) in order to signal that MAMBIT is not all of intelligence in their view. Even though I am in sympathy with some of the goals of these critics, I think their strategy is mistaken. Here is why.

  Broad theorists inflate the concept of intelligence. By inflation I mean putting into the term more than what the IQ tests measure. One very strong tendency among broad theorists is to use adjectives to differentiate the more encompassing parts of their intelligence concept from the “IQ-test part.” Major theorists such as Sternberg and Gardner talk about practical intelligence, creative intelligence, interpersonal intelligence, bodily-kinesthetic intelligence, etc. In such usages, the word intelligence becomes a marker for “optimal or expert behavior in the domain of.” So, for instance, when Sternberg discusses high practical intelligence it can be translated to mean “optimal behavior in the domain of practical affairs” or when Gardner talks about high bodily-kinesthetic intelligence he means little more than high functioning in the bodily-kinesthetic domain. The word intelligence is actually superfluous. It is there merely to add status to the domain in question (to put it on equal footing with MAMBIT). The strategy seems to be something like the following: Because intelligence is a valued trait and we want bodily-kinesthetic talent to be valued too, we’ll fuse the term intelligence onto it in order to transfer some of the value from intelligence to bodily-kinesthetic talent. Indeed, this is why educators have been so enthusiastic about the “multiple intelligences” idea. Its scientific status is irrelevant to them. They use it as a motivational tool—to show that “everyone is intelligent in some way.” The same is true for the coinages of social intelligence or emotional intelligence.3

  However, there are unintended consequences—some of them quite ironic—of this strategy, consequences that have been insufficiently appreciated. Labeling different mental entities with the same name will encourage just the assumption that many broad theorists want to attack—it will inflate the esteem given to MAMBIT. In a sense, broad theorists seek to break a rule of construct validity—and of common sense: things that are named the same should go together. If these things really are separate mental faculties, and we wish to emphasize their separateness, then we should not suggest just the opposite by calling them all “intelligences.” However, by their profligate use of the term intelligence, the broad theorists subvert their very purpose of isolating “the IQ-test part of intelligence” (MAMBIT) as only one aspect of many cognitive virtues that we may wish to value (spatial ability, creative ability, fluency in practical affairs). People will continue to make the assumption that MAMBIT will correlate with all of these other things (in psychometric terms, an assumption of positive manifold).4

  By inflating the word intelligence, by associating it with more and more valued mental ac
tivities and behaviors, broad theorists will succeed in doing just the opposite of what many of them intend—cutting “the IQ-test part of intelligence” down to size. If you inflate the conceptual term intelligence you will inflate all its close associates as well—and 100 years of mental testing makes it a simple historical fact that the closest associate of the term intelligence is “the IQ-test part of intelligence.”

  Intelligence Imperialism

  In commenting on the history of his multiple intelligences theory, Howard Gardner relates that he considered other terms such as skills or capacities but then realized “that each of these words harbored pitfalls, I finally elected to take the bold step of appropriating a word from psychology and stretching it in new ways. . . . I was proposing an expansion of the term intelligence so that it would encompass many capacities that had been considered outside its scope” (1999, pp. 33, 34). Likewise, Robert Sternberg argues that “the time perhaps has come to expand our notion and everyone’s notion of what it means to be intelligent” (2003b, p. 69). Clearly one of the goals here is to emphasize that there are aspects of cognitive life that are important outside of MAMBIT. This is a goal that I share with many broad theorists.5 However, I do not see why everything in human nature, cognitively speaking, has to have the label intelligence—particularly when there are readily existing labels (both scientific labels and folk labels) for some of those things (rationality, creativity, wisdom, critical thinking, open-minded thinking, reflectivity, sensitivity to evidence).