The nature of the data

The measurement apparatus Lynn and his colleagues used in each country enabled them to make several kinds of reaction time measurement. This apparatus was a rigid box with a downward-sloping top face, in which face were embedded the buttons and lights used to test the subject. The subject began each reaction time trial by depressing a central home button. Around that button were eight more buttons arranged in a semicircle 30 cm (12 inches) wide, each of these eight outer buttons being next to a small light. At the start of each reaction time trial all of the lights were off; one or more of the lights would then turn on, indicating to the subject that they should lift their finger off the home button and press a particular outer button. The net reaction time was the time elapsed between the light(s) coming on and the subject pressing an outer button, and this time was in turn split into two components: the subject's decision time (DT – sometimes referred to by Lynn as ‘reaction time’), defined as the time the subject took to lift their finger from the home button in response to the light(s), and the subject's movement time (MT), defined as the time elapsed between their releasing the home button and their pressing an outer button. A computer connected to the measurement box recorded these timings, eliminating human error in recording the data.

The box could be set to test the subject in any of three testing regimes: simple, choice (sometimes called complex) or odd man out. In simple reaction time trials, a specific light was used for every trial and the other seven remain unused and unlit, so the subject needed only to watch a particular light and press its associated button when that light came on. In choice reaction time trials, any one of the eight lights would turn on, and the subject's task was to see which light it was and press its button accordingly. Finally, in odd man out (OMO) trials, three lights turned on simultaneously, and the subject was to press the button by the shining light furthest from the other two.

Each child was tested with a block of trials in each of the three regimes, so that for each regime every child had their median DT, median MT, standard deviation of DT and standard deviation of MT recorded. The mean of each of these four variables was then taken for the whole sample, so that ultimately Lynn presented four (variable means) × three (regimes) = twelve summary statistics for each sample. At the sample level, Lynn typically referred to the mean of the subjects' MT/DT standard deviations as ‘variability’, and for continuity's sake this paper adopts the same word to mean the same thing.

When summarizing the international reaction time data, Lynn picked out the (six – median and variability for each of the three regimes) DT means for five samples, each from a different country – Hong Kong, Japan, Britain, Ireland and South Africa – and juxtaposed them with his estimates of the countries' IQs in table 6.2 of Lynn & Vanhanen (Reference Lynn and Vanhanen2002, p. 67). He drew attention to the correlation across countries between IQ and each DT measure. For example, Lynn calculated a correlation of −0.94 between his estimates of national simple DT and his estimates of national IQ. It is true that for each of the six DT measures, a nation having a shorter mean DT does appear to be associated with that nation having a higher mean IQ. One might believe, as Lynn seems to, that this lends credibility to the contention that decision time serves as a good index of a nation's IQ and intelligence. Two psychologists took this apparent finding still further: Rushton & Jensen (Reference Rushton and Jensen2005, pp. 244–245) used Lynn's summary table (and reproduced its accompanying citations) as evidence for racial differences in intelligence, citing reaction time as ‘one of the simplest culture-free cognitive measures’ (p. 244).

Note that Lynn, Rushton and Jensen dedicate far more attention to DT than MT. However, Flynn (Reference Flynn1991) observes that MT, like DT, also correlates with IQ in Lynn et al.'s (Reference Lynn1991) results, and he proposes that MT be considered alongside DT when interpreting their data. A look at the correlations between Raven's Progressive Matrices scores and RT tabulated in Lynn et al. (Reference Lynn1991) and other papers suggests that Flynn's point is well taken. The median of the 48 correlations between DT measures and RPM score is −0.150 with a standard deviation of 0.098; the median of the 48 correlations between MT measures and RPM score is −0.135 with a standard deviation of 0.078. The medians are of similar magnitude, and so are evidence that MT and DT can correlate similarly with IQ in Lynn's samples, supporting Flynn's move to draw attention to MT as well as DT. Nonetheless, as Lynn and subsequent workers have drawn inferences primarily from the DT results, this paper likewise focuses on DT.

Lynn's summaries of the comparative reaction time data

Before accepting Lynn's analysis of DT, it is instructive to return to Lynn's original sources and check that he summarized them correctly. On the same page as their table, Lynn and Vanhanen cite Shigehisa & Lynn (Reference Lynn and Shigehisa1991) for the Japanese data, Chan & Lynn (Reference Chan and Lynn1989) for the data on Hong Kong and Britain, Lynn (Reference Lynn1991) for the Irish data and Lynn & Holmshaw (Reference Lynn and Holmshaw1990) for data on South Africa.

One of the citations is misleading: there are no reaction time data in the Chan & Lynn (Reference Chan and Lynn1989) paper, which appears to be a paper about IQ test scores (as its title suggests) rather than reaction times. The Hong Kong and Britain reaction time data are actually to be found in Lynn et al. (Reference Lynn1991), which one can confirm by comparing that paper's data with the numbers in the Lynn & Vanhanen (Reference Lynn and Vanhanen2002) table.

Comparing the sample size and decision time data stated in the 2002 table with those in the original papers reveals a number of discrepancies, which Table 1 lists. Table 1 also records discrepancies between the original reports and the data in table 7 of Lynn (Reference Lynn1991), which collects reaction time data from Lynn's earlier papers in much the same way as Lynn & Vanhanen's (2002) table. Indeed, the 1991 table is similar enough to the 2002 table that the latter could well be based on the former, so it is informative to include the 1991 table in the comparisons. (The author tried to confirm or falsify this possibility by asking Lynn via email whether the 2002 table ‘was written afresh from the original papers, or instead mostly derived from table 7 in [his] 1991 review’. Lynn replied that he could not remember (personal communication, 13th July 2009).)

Table 1. Inconsistencies in reported sample sizes and DT means for the countries in Lynn's (1991) table 7, Lynn & Vanhanen's (2002) table 6.2, and their original sources

All numbers other than the sample size (i.e. the DT statistics) have units of milliseconds, and numbers in bold and italics disagree with the original report.

There is another problem with both the Lynn (Reference Lynn1991) and Lynn & Vanhanen (Reference Lynn and Vanhanen2002) tables, but it is with what is absent from the tables rather than what is present. Both tables only include data from the four papers (Lynn & Holmshaw, Reference Lynn, Cooper and Topping1990; Shigehisa & Lynn, Reference Lynn and Shigehisa1991; Lynn et al., Reference Lynn, Chan and Eysenck1991; Lynn, Reference Lynn1991) and five countries mentioned above. However, at around the same time as those papers were published, other papers co-authored by Lynn that included more data on reaction time among nine-year-olds went into print, and these additional data are absent from the tables. As such they present a selective picture of Lynn's reaction time data, and scholars who rely on the tables – such as Rushton & Jensen – run the risk of drawing conclusions from an incomplete data corpus.

To allow a full assessment of Lynn's results, and so that future workers in this field can conveniently refer to all of Lynn's available data, here is a short description of the six relevant papers omitted from both summary tables.

Two of the omitted papers – Ja-Song & Lynn (Reference Ja-Song and Lynn1992) and Lynn & Ja-Song (Reference Lynn and Ja-Song1993) – discuss reaction times in South Korean nine-year-olds. The first of the papers reports reaction time data, as well as mean Raven's Standard Progressive Matrices and ‘Korean General Intelligence Test (GIT)’ scores, from ‘148 boys and 151 girls, aged 9.6 to 9.11 years (mean age, 117.2 months), attending a large middle-income primary school in the city of Pusan in South Korea’ (p. 422). The paper's table 1 has the results for the total sample and a breakdown of the results by sex. The second paper, Lynn & Ja-Song (Reference Lynn and Ja-Song1993), compares the South Korean reaction time results from the first paper with reaction time results from a ‘Britain’ sample, and here arises a potential confusion.

Lynn & Ja-Song's (Reference Lynn and Ja-Song1993) ‘Britain’ sample was of Northern Irish children, and the paper nowhere mentions that; it repeatedly refers to its Irish sample as ‘British’ or being taken in ‘Britain’. This is somewhat frustrating because, by contrast, Lynn's (Reference Lynn1991) table 7 and the 2002 table both present results from ‘Britain’ and results from ‘Ireland’ in separate columns as data from separate regions. Lynn & Ja-Song (Reference Lynn and Ja-Song1993) does not observe this careful distinction. To establish that the sample is of Northern Irish schoolchildren one must independently notice that the ‘Britain’ data are the same as data earlier published in Lynn et al. (Reference Lynn, Cooper and Topping1990), where they are explicitly described as coming from ‘205 children (93 boys and 112 girls) attending primary schools in small country towns and villages in Northern Ireland’ (p. 265). In spite of this, Lynn & Ja-Song failed to cite the original Lynn et al. (Reference Lynn, Cooper and Topping1990) paper as the source of the ‘Britain’ data, and did not cite Ja-Song & Lynn (Reference Ja-Song and Lynn1992) as the original publication of the Korean data. Thus Lynn & Ja-Song (Reference Lynn and Ja-Song1993) constitutes a double example of double publication of data.

The data also changed slightly between their first publication and the second. Lynn et al. (Reference Lynn, Cooper and Topping1990) avers that the Northern Irish sample contained 93 boys and 112 girls, and that the female subsample's median simple DTs had a standard deviation of 53.4 ms, but Lynn & Ja-Song (Reference Lynn and Ja-Song1993) asserts that it contained 90 boys and 112 girls, and that the corresponding standard deviation was 54.3 ms. Similarly, for the South Korean sample, Ja-Song & Lynn (Reference Ja-Song and Lynn1992) gives the standard deviation of the girls' median simple MT as 42.1 ms, the mean of the boys' choice MT as 180.3 ms, the standard deviation of boys' median OMO DT as 166.3 ms, and the standard deviation of the girls' median OMO DT as 180.7 ms. The corresponding numbers in Lynn & Ja-Song (Reference Lynn and Ja-Song1993) are 41.1 ms, 180.2 ms, 116.3 ms and 180.6 ms, respectively.

Lynn et al. (Reference Lynn, Cooper and Topping1990) is one of two publications about Irish datasets not incorporated into the 1991 and 2002 tables. Lynn et al. (Reference Lynn, Cooper and Topping1990) describes reaction time and IQ data from 93 boys and 112 girls, as quoted above, and Lynn & Wilson (Reference Lynn and Wilson1990) has reaction time and mental test scores from ‘95 boys and 96 girls aged between 9.00 and 9.11 years who were pupils from country primary schools in the Irish Republic’ (p. 331). Neither paper cites the other; in fact, there are no papers authored or co-authored by Lynn in either paper's reference list.

Lynn & Shigehisa (Reference Lynn and Shigehisa1991) is another double example of double publication. It compares the Japanese data from Shigehisa & Lynn (Reference Lynn and Shigehisa1991) with the ‘Britain’ data from Lynn & Holmshaw (Reference Lynn and Holmshaw1990), citing neither. Those last two papers are given as sources for Lynn & Vanhanen's (Reference Lynn and Vanhanen2002) table, though Lynn & Shigehisa (Reference Lynn and Shigehisa1991) is not.

Finally, Chan et al. (Reference Chan, Eysenck and Lynn1991) reports on ‘479 9-yr.-old Chinese children attending primary schools in Hong Kong’, 239 of them boys, and 240 girls (p. 428). The sample size and reaction time statistics differ from those in Lynn et al. (Reference Chan, Eysenck and Lynn1991), so presumably the latter paper's Hong Kong sample is completely distinct from Chan et al.'s, but it is impossible to be fully sure.

Reanalysing Lynn's comparative reaction time data

With all of Lynn's comparative reaction time data in hand, it is possible to consider his results in toto. His papers describe two samples of Hong Kong children, one from Japan, one from South Korea, one from ‘Britain’, three from Ireland and one from South Africa. (The analyses below assume that these samples are all entirely independent of each other, but this may not be true. Lynn's papers often do not provide enough detail to rule out overlaps between samples in the same country.) In his summary tables Lynn omitted without explanation one of the Hong Kong datasets, the South Korean data (although it is not inconceivable that the data were not in a usable form when he put together the 1991 table, as they were not published until 1992) and two of the three Irish datasets.

This left Lynn with one sample each from Hong Kong, Japan, ‘Britain’, Ireland and South Africa. Analysing only one dataset from each nation has the disadvantage of precluding testing for DT variation within nation, the presence of which could derail cross-country DT comparisons like those of Lynn and his co-workers. As there are two sets of means and standard deviations for Hong Kong and for Ireland, it is both straightforward and prudent to compare each country's pair of means with t-tests to pinpoint statistically significant within-country variation in DT. (The Irish results from Lynn's (Reference Lynn1991) table are unusable for this purpose, as the table has no standard deviations for the DT variables. The means from the other two Irish datasets are nonetheless comparable with t-tests.) Table 2 collects the results of these statistical comparisons (which were done with Welch's t-test, as none of the samples have equal sizes and there is little reason to expect equal population variances among them).

Table 2. Two-tailed Welch's t-test comparisons of pairs of DT means for Hong Kong and for Ireland

There is one statistically significant DT difference between the two Irish samples – a minor difference in complex DT variability with a p-value of 0.04. There is one DT difference with a p-value of 0.032 between the Hong Kong samples, suggesting statistical significance, as well as three more differences with borderline p-values of 0.050, 0.076 and 0.092. As none of the p-values are far smaller than the conventional 0.05 significance level, and as they cannot be treated as wholly independent due to intercorrelations between DT measures, they are not strong evidence of DT heterogeneity between the Irish and the Hong Kong subpopulations of nine-year-olds. This of course does not rule it out – only more comprehensive and more systematic studies could warrant that – but it is difficult to pick it out from the within-sample variation.

The level of intranational variation is not so high as to complicate international comparisons. It is more problematic that DT in general does not seem to function (at least on a national level) as a good proxy for IQ. If it did, the ordering of different samples on each DT variable should be the essentially the same, reflecting IQ differences between samples. This is not the case: Table 3 shows that the orderings vary appreciably across different DT variables. The samples are large enough that most of these fluctuations in rank cannot be explained by sampling error; the alternative explanation for the fluctuations is that the different DT variables do not all measure one and the same trait, and so they cannot all be unambiguous indices of national IQ or intelligence.

Table 3. Rankings, in ascending order (i.e. shortest/fastest DT first), of Lynn's samples of nine-year-olds on the six recorded DT variables

Lynn, Rushton and Jensen do not appear to acknowledge this issue. Instead of warning their readers of it, they simply attempted to use Lynn's papers to argue for racial differences in DT and hence intelligence (Lynn, Reference Lynn1991; Rushton & Jensen, Reference Rushton and Jensen2005). They also failed to adequately check for heterogeneity in DT averages within races. If a race, however one chooses to define it, is heterogeneous with respect to some statistic, attempts to represent it with a single number are likely to be ill-founded; a range may be appropriate, as that accounts for within-race variation, but a simple average should be regarded with scepticism. Heterogeneity would complicate racial comparisons because it could render unjustified a statement that one race has unconditionally longer or shorter mean DTs than another. More likely, one could at best state that a set of specific ethnicities within a race had longer or shorter DTs than another set of other ethnicities within another race.

Testing for heterogeneity within races requires definitions of racial groups. As Lynn's data (and secondarily Rushton & Jensen's interpretation of them) are the subject of this paper, the racial heterogeneity analysis uses Lynn's (and Rushton & Jensen's) choice of racial groups. Rushton & Jensen (Reference Rushton and Jensen2005, p. 245) use Lynn's summary to compare ‘East Asian children in Hong Kong and Japan’, ‘white children in Britain and Ireland’ and ‘black children in South Africa’. Lynn uses the different terms ‘Mongoloid’, ‘Negroid’ and ‘Caucasoid’ in his 1991 review, but he appears to use them synonymously with ‘East Asian’, ‘Black’ and ‘White’ respectively. Going by these categorizations, it seems fair to place the samples from Hong Kong, Japan and South Korea into the East Asian racial category, the Irish and ‘British’ samples into the white category, and the lone South African sample into the black category. Since Lynn's datasets include only one sample of black children, it is not possible to test for heterogeneity among black children, but the results from four East Asian samples and three white samples (Lynn's (Reference Lynn1991) Irish sample could not be included for want of reported standard deviations) are testable.

Having confirmed a racial categorization to use, and its compatibility with the Lynn–Rushton–Jensen categorization, applying Welch's F-test (Welch, Reference Cochran1951) to the means within each group serves to test for heterogeneity. Table 4 collects the tests' results, broken down by race and DT variable. On every DT variable, the East Asian samples exhibit significant heterogeneity, and there is also significant heterogeneity within the white samples for the three DT variabilities, although not for the means of the three median DTs themselves. Heterogeneity's ubiquity among the East Asian samples implies that one cannot try to calculate a single average representative of all of them for purposes of comparison; to compare them with other samples, one should compare the samples individually with others instead of aggregating them first.

Table 4. Welch's F-test results for within-race heterogeneity in DT means

All of these F-statistics' p-values are greater than 0.1 or less than 0.0001.

Asterisks denote F-statistics with p<0.0001, and so statistically significant heterogeneity.

Therefore, to test the Lynn–Rushton–Jensen hypothesis of systematic racial DT differences, all of the individual sample means were compared with each other on each DT variable. Note that although directly comparing races (as defined previously) is problematic because of within-race heterogeneity, an implicit test of the Lynn–Rushton–Jensen racial model remains possible. If statistically significant DT differences only occur between samples of different races, and rarely between samples from the same race, that would be evidence consistent with the Lynn–Rushton–Jensen view. Alternative patterns (or a lack of a pattern) of differences would be evidence against it, and would suggest that race per se is a relatively unimportant determinant of mean DT.

The author calculated 95% confidence intervals on the pairwise differences between sample DT means to compare the samples on each DT variable, using the ‘C’ (after Cochran (Reference Cochran1964)) procedure described in Dunnett (Reference Dunnett1980) to adjust the confidence intervals to account for multiple comparisons. (The Cochran procedure is more applicable for this case than the usual Tukey–Kramer procedure because the samples differ in size and there is little reason to presume that the samples have equal population DT variance.) Table 5 collects the results of these pairwise comparisons on each of the six DT variables; significant differences are those for which the pairwise mean difference's confidence interval included zero, and insignificant differences are those for which the confidence interval excluded zero.

Table 5. Comparison of DT means among samples of various races and countries: samples that share any letter do not differ at the 0.05 significance level (by two-tailed tests)

‘HK1’ and ‘HK2’ denote Hong Kong data from Lynn et al. (Reference Lynn1991) and Chan et al. (Reference Chan, Eysenck and Lynn1991) respectively; ‘J’ denotes Japan data from Shigehisa & Lynn (Reference Lynn and Shigehisa1991); ‘SK’ denotes South Korea data from Ja-Song & Lynn (Reference Ja-Song and Lynn1992); ‘B’ & ‘SA’ denote Britain data and South Africa data respectively from Lynn & Holmshaw (Reference Lynn and Holmshaw1990); ‘I2’ denotes sex-pooled Republic of Ireland data from Lynn & Wilson (Reference Lynn and Wilson1990); ‘I3’ denotes sex-pooled Northern Ireland data from Lynn et al. (Reference Lynn, Cooper and Topping1990).

The pairwise comparisons corroborate the conclusions of the previous statistical tests: there are no significant differences between the two Hong Kong samples or the two Irish samples, but there are significant differences among the samples in the East Asian category and among those in the white category. The pairwise tests further imply that the significant differences do not primarily fall neatly into the three racial categories. The South Korean sample tends to differ significantly from all of the other samples, not just the non-East Asian samples. Also, in the cases where the East Asian and white averages tend to cluster along racial lines, the white and black averages do not, and vice versa. For example, for median OMO DT, there is a clean split between the East Asian and white means, but there are no significant differences among the means for the white and black samples. Conversely, for complex DT variability, the South African mean stands apart from all of the others, but the East Asian and white means are broadly similar.

It may be tempting to argue that the heterogeneity of the East Asian means masks an underlying racial pattern to the results. Such an argument would be invalid: the white–black differences fail to weigh heavily in one direction even when considered alone. For three (median complex DT, median OMO and OMO variability) of the six variables, the South African mean does not differ significantly from at least one of the ‘British’ and Irish means. Among the other three variables, the South African mean is significantly more than the ‘British’ and Irish means in two cases, and is significantly less in the third (for simple DT, the South African mean is 298.0 ms, and the comparable white means range from 366.6 to 376.2 ms).

Acknowledgments

The author would like to thank the paper's reviewer for their helpful comments and Richard Lynn for his responses to the author's questions about his research.