What Intelligence Tests Miss Page 5
Incident C, on the other hand, does not involve such an algorithmic-level information processing error. The woman’s perceptual apparatus accurately recognized the edge of the cliff, and her motor command centers quite accurately programmed her body to jump off the cliff. The computational processes posited at the algorithmic level of analysis executed quite perfectly. No error at this level of analysis explains why the woman is dead in incident C. Instead, this woman died because of her overall goals and how these goals interacted with her beliefs about the world in which she lived.
In 1996, philosopher Daniel Dennett wrote a book about how aspects of the human mind were like the minds of other animals and how other aspects were not. He titled the book Kinds of Minds to suggest that within the brains of humans are control systems of very different types—different kinds of minds. In the spirit of his book, I am going to say that the woman in incident B had a problem with the algorithmic mind and the woman in incident C had a problem with the reflective mind. This terminology captures the fact that we turn to an analysis of goals, desires, and beliefs to understand a case such as C. The algorithmic level provides an incomplete explanation of behavior in cases like incident C because it provides an information processing explanation of how the brain is carrying out a particular task (in this case, jumping off a cliff) but no explanation of why the brain is carrying out this particular task. We turn to the level of the reflective mind when we ask questions about the goals of the system’s computations (what the system is attempting to compute and why). In short, the reflective mind is concerned with the goals of the system, beliefs relevant to those goals, and the choice of action that is optimal given the system’s goals and beliefs. It is only at the level of the reflective mind that issues of rationality come into play. Importantly, the algorithmic mind can be evaluated in terms of efficiency but not rationality.
This concern for the efficiency of information processing as opposed to its rationality is mirrored in the status of intelligence tests. They are measures of efficiency but not rationality—a point made clear by considering a distinction that is very old in the field of psychometrics. Psychometricians have long distinguished typical performance situations from optimal (sometimes termed maximal) performance situations.12 Typical performance situations are unconstrained in that no overt instructions to maximize performance are given, and the task interpretation is determined to some extent by the participant. The goals to be pursued in the task are left somewhat open. The issue is what a person would typically do in such a situation, given few constraints. Typical performance measures are measures of the reflective mind—they assess in part goal prioritization and epistemic regulation. In contrast, optimal performance situations are those where the task interpretation is determined externally. The person performing the task is instructed to maximize performance and is told how to do so. Thus, optimal performance measures examine questions of efficiency of goal pursuit—they capture the processing efficiency of the algorithmic mind. All tests of intelligence or cognitive aptitude are optimal performance assessments, whereas measures of critical or rational thinking are often assessed under typical performance conditions.
The difference between the algorithmic mind and the reflective mind is captured in another well-established distinction in the measurement of individual differences—the distinction between cognitive abilities and thinking dispositions. The former are, as just mentioned, measures of the efficiency of the algorithmic mind. The latter travel under a variety of names in psychology—thinking dispositions or cognitive styles being the two most popular. Many thinking dispositions concern beliefs, belief structure, and, importantly, attitudes toward forming and changing beliefs. Other thinking dispositions that have been identified concern a person’s goals and goal hierarchy. Examples of some thinking dispositions that have been investigated by psychologists are: actively open-minded thinking, need for cognition (the tendency to think a lot), consideration of future consequences, need for closure, superstitious thinking, and dogmatism.13
The literature on these types of thinking dispositions is vast, and my purpose is not to review that literature here. It is only necessary to note that the types of cognitive propensities that these thinking disposition measures reflect are: the tendency to collect information before making up one’s mind, the tendency to seek various points of view before coming to a conclusion, the disposition to think extensively about a problem before responding, the tendency to calibrate the degree of strength of one’s opinion to the degree of evidence available, the tendency to think about future consequences before taking action, the tendency to explicitly weigh pluses and minuses of situations before making a decision, and the tendency to seek nuance and avoid absolutism. In short, individual differences in thinking dispositions are assessing variation in people’s goal management, epistemic values, and epistemic self-regulation—differences in the operation of reflective mind. They are all psychological characteristics that underpin rational thought and action.
The cognitive abilities assessed on intelligence tests are not of this type. They are not about high-level personal goals and their regulation, or about the tendency to change beliefs in the face of contrary evidence, or about how knowledge acquisition is internally regulated when not externally directed. As we shall see in the next chapter, people have indeed come up with definitions of intelligence that encompass such things. Theorists often define intelligence in ways that encompass rational action and belief but, despite what these theorists argue, the actual measures of intelligence in use assess only algorithmic-level cognitive capacity. No current intelligence test that is even moderately used in practice assesses rational thought or behavior.
The algorithmic mind, assessed on actual IQ tests, is relevant in determining what happened in the case of lady B above, but it does not provide sufficient explanation of the case of lady C. To understand what happened in the case of lady C, we need to know about more than her processes of memory and speed of pattern recognition. We need to know what her goals were and what she believed about the world. And one of the most pressing things we want to know about lady C was whether there was some sense in her jumping off the cliff. We do not want to know whether she threw herself off with the greatest efficiency possible (an algorithmic-level question). We want to know whether it was rational for her to jump.
Moving toward a Tripartite Model of Mind
We have now bifurcated the notion of Type 2 processing into two different things—the reflective mind and the algorithmic mind. If we give Type 1 processing its obvious name—the autonomous mind—we now have a tripartite view of thinking that departs somewhat from previous dual-process views because the latter tended to ignore individual differences and hence to miss critical differences in Type 2 processing. The broken horizontal line in Figure 3.3 represents the location of the key distinction in older, dual-process views. The figure represents the classification of individual differences in the tripartite view, and it identifies variation in fluid intelligence (Gf) with individual differences in the efficiency of processing of the algorithmic mind. In contrast, thinking dispositions index individual differences in the reflective mind. The reflective and algorithmic minds are characterized by continuous individual differences. Continuous individual differences in the autonomous mind are few. Disruptions to the autonomous mind often reflect damage to cognitive modules that result in very discontinuous cognitive dysfunction such as autism or the agnosias and alexias.14
Figure 3.3 highlights an important sense in which rationality is a more encompassing construct than intelligence. To be rational, a person must have well-calibrated beliefs and must act appropriately on those beliefs to achieve goals—both properties of the reflective mind. The person must, of course, have the algorithmic-level machinery that enables him or her to carry out the actions and to process the environment in a way that enables the correct beliefs to be fixed and the correct actions to be taken. Thus, individual differences in rational thought and action can a
rise because of individual differences in intelligence (the algorithmic mind) or because of individual differences in thinking dispositions (the reflective mind). To put it simply, the concept of rationality encompasses two things (thinking dispositions of the reflective mind and algorithmic-level efficiency) whereas the concept of intelligence—at least as it is commonly operationalized—is largely confined to algorithmic-level efficiency.
The conceptualization in Figure 3.3 has two great advantages. First, it conceptualizes intelligence in terms of what intelligence tests actually measure. That is, all current tests assess various aspects of algorithmic efficiency (including the important operation that I have emphasized here—the ability to sustain cognitive decoupling). But that is all that they assess. None attempt to measure directly an aspect of epistemic or instrumental rationality, nor do they examine any thinking dispositions that relate to rationality. It seems perverse to define intelligence as including rationality when no existing IQ test measures any such thing! The second advantage is that the model presented in Figure 3.3 explains the existence of something that folk psychology recognizes—smart people doing dumb things (dysrationalia).
Figure 3.3. Individual differences in the Tripartite Framework
It is clear from Figure 3.3 why rationality and intelligence can come apart, creating dysrationalia. As long as variation in thinking dispositions is not perfectly correlated with intelligence, then there is the statistical possibility of dissociations between rationality and intelligence. Substantial empirical evidence indicates that individual differences in thinking dispositions and intelligence are far from perfectly correlated. Many different studies involving thousands of subjects have indicated that measures of intelligence display only moderate to weak correlations (usually less than .30) with some thinking dispositions (for example, actively open-minded thinking, need for cognition) and near zero correlations with others (such as conscientiousness, curiosity, diligence).15
Psychologist Milton Rokeach, in his classic studies of dogmatism, puzzled over why his construct displayed near-zero correlations with intelligence test scores. He mused that “it seems to us that we are dealing here with intelligence, although not the kind of intelligence measured by current intelligence tests. Apparently, intelligence tests do not tap the kinds of cognitive functioning we have been describing in this work. This seems paradoxical. For the current work is concerned with the very same cognitive processes with which intelligence tests are allegedly concerned” (1960, p. 407). The paradox that Rokeach was noticing was the drastic mismatch between the claims for the concept of intelligence and the cognitive processes that tests of the construct actually measure. In the current view, Rokeach’s measure of dogmatism is indeed an important thinking disposition of the reflective mind, but there is no reason to consider it an aspect of intelligence. Dogmatism/openness is instead an aspect of the reflective mind that relates to rationality.
It is important to note that the thinking dispositions of the reflective mind are the psychological mechanisms that underlie rational thought. Maximizing these dispositions is not the criterion of rational thought itself. Rationality involves instead the maximization of goal achievement via judicious decision making and optimizing the fit of belief to evidence. The thinking dispositions of the reflective mind are a means to these ends. Certainly high levels of such commonly studied dispositions as reflectivity and belief flexibility are needed for rational thought and action. But “high levels” does not necessarily mean the maximal level. One does not maximize the reflectivity dimension for example, because such a person might get lost in interminable pondering and never make a decision. Likewise, one does not maximize the thinking disposition of belief flexibility either, because such a person might end up with a pathologically unstable personality. Reflectivity and belief flexibility are “good” cognitive styles (in that most people are not high enough on these dimensions, so that “more would be better”), but they are not meant to be maximized.
Thinking Dispositions as Predictors of Rational Thought and Action
There is a further reason to endorse the tripartite structure I am proposing here—an empirical reason. In order to statistically predict rational thought and action to a maximum extent, one needs to take into account aspects of the reflective mind in addition to intelligence. For example, an important aspect of epistemic rationality is the ability to calibrate evidence appropriately to belief. One rule of such calibration is that ambiguous evidence should lead to tentative belief. People often violate this stricture, particularly when myside bias is operating. Research has found that the tendency to follow this stricture is more strongly related to two thinking dispositions—the tendency to believe in certain knowledge and the need for cognition—than it is to intelligence.
In my own laboratory, we have developed an argument evaluation task in which we derive an index of the degree to which argument evaluation is associated with argument quality independent of prior belief.16 Intelligence did in fact correlate with the ability to avoid belief bias in our task. Nonetheless, we have consistently found that, even after statistically controlling for intelligence, individual differences on our index of argument-driven processing can be predicted by a variety of thinking dispositions, including: measures of dogmatism and absolutism; categorical thinking; flexible thinking; belief identification; counterfactual thinking; superstitious thinking; and actively open-minded thinking.
It is likewise with other aspects of rational thinking. For example, researchers have studied situations where people display a particular type of irrational judgment—they are overly influenced by vivid but unrepresentative personal and testimonial evidence and are under-influenced by more representative and diagnostic statistical evidence.17 We have studied a variety of such situations in my own laboratory and have consistently found that dispositions toward actively open-minded thinking are consistently associated with reliance on the statistical evidence rather than the testimonial evidence. Furthermore, this association remains even after intelligence has been statistically controlled for. Similar results have obtained for a variety of other rational thinking tendencies that we have studied.18
Not only is rational thought itself predicted by thinking dispositions after intelligence is controlled, but the outcomes of rational thought are likewise predicted by variation in characteristics of the reflective mind.19 In an important study, Angela Duckworth and Martin Seligman found that the grade point averages of a group of eighth graders were predicted by measures of self-discipline (that is, indicators of response regulation and inhibition at the reflective level) after the variance due to intelligence was partialled out. A longitudinal analysis showed that self-discipline was a better predictor of the changes in grade point average across the school year than was intelligence. The personality variable of conscientiousness—which taps the higher-level regulatory properties of the reflective mind—has been shown to predict, independent of intelligence, academic performance and measures of performance in the workplace. Political psychologist Philip Tetlock studied expert political forecasters, all of whom had doctoral degrees (and hence were presumably of high intelligence), and found that irrational overconfidence was related to thinking dispositions that tapped epistemic regulation. Wandi Bruine de Bruin and colleagues recruited a sample of 360 citizens who resembled the demographics of the 2000 U.S Census for their area and administered to them a battery of rational thinking tasks similar to those to be discussed in this book. They formed a composite score reflecting overall rational thinking skill and found that it was correlated (negatively) with a composite measure of poor decision making outcomes (for instance, bouncing checks, having been arrested, losing driving privileges, credit card debt, eviction). Importantly, Bruine de Bruin and colleagues found that variance in their decision outcome measure was predicted by rational thinking skill after the variance due to cognitive ability had been controlled.
Across the range of tasks I have been reviewing here (and more that will be discus
sed in later chapters), it is the case that performance on the rational thinking tasks was moderately correlated with intelligence. Nevertheless, the magnitude of the associations with cognitive ability left much room for systematic variation to be explained by thinking dispositions. Furthermore, if anything, the studies I have reviewed over estimate the linkage between intelligence and rational thinking. This is because many of these studies have given the subjects helpful instructions—for example, that they are to put aside their prior opinion and reason in an unbiased manner. There is a pattern in the literature indicating that when subjects are not given such instructions—when they are left free to reason in a biased or unbiased manner according to their wish (as we all do in real life)—then the correlations between unbiased reasoning and intelligence are nearly zero (as opposed to the modest .30–.40 correlations that obtain when such instructions are given).20