Continuous variables

Examining the shape of the empirical distribution of a continuous variable using suitable displays can help you select appropriate statistics to describe its location and spread and to decide on an approach for statistical inference. The most common shapes that researchers will encounter for continuous variables are normal or skewed (both defined in Table 3). If the display reveals the data's empirical distribution is approximately normal, then the sample mean and s.d. are suitable measures of location and spread, and the normal distribution can be used for statistical inference about the population. This is no longer the case if the display reveals that the data's empirical distribution is highly skewed, where the median should be used as a measure of location and the inter-quartile range (25th–75th percentiles) and/or range as measures of spread. The effect of positive or negative skewness is to pull the mean above or below the median, respectively (i.e. the mean no longer reflects where the bulk of the data lies). Note that if the empirical distribution is approximately normal or symmetric then the mean should be similar to the median. The example in this tutorial deals with only normal and positively skewed data. Advanced statistical techniques are required if the data's empirical distribution has one of the other shapes mentioned in Table 3 or some other shape.

Table 3. Glossary of terms used to describe the shape of a variable's empirical distribution

The most accurate display shows all the data points collected in a sample, but this may be difficult to do in an uncluttered way when the sample is large. Accordingly, we recommend the use of different plots depending on sample size. The threshold used to define a small and large sample in the following sections is simply a guide.

Displays for small samples

If the sample is small (⩽30 individuals) then display all the data using a dotplot. Figure 2 presents dotplots of IgG titre in response to each Plasmodium blood stage antigen. Data are a random subset of 30 (15 parasite-free controls and 15 malaria infected cases) out of the 317 pregnant women. Each dot on the plot represents an observation for an individual, plotted along the y-axis is IgG titre in response to each antigen with observations for cases and controls plotted alongside each other for comparison. If women have the same IgG titre in response to a particular antigen the observations are stacked horizontally, so that the frequency of a particular observation is represented, for example, in antigen group Pf-IE merozoite (Fig. 2) five controls have an IgG titre of 0·04 OD.

Fig. 2. Dotplot of IgG titres in response to each Plasmodium blood stage antigen. Data presented are a random subset of 30 (15 non-infected controls and 15 malaria infected cases) out of the 317 pregnant women included in the example dataset.

How should we interpret Fig. 2 for a descriptive analysis and to inform statistical inference? Descriptive analysis: the dotplot clearly shows that IgG titre in response to each antigen tend to be lower with less spread for parasite-free controls compared with malaria infected cases. For both cases and controls, the IgG titres in response to each antigen tend to be bunched towards zero with a tail stretching towards the right, which indicates that the data's empirical distribution for each group is positively skewed (defined in Table 3). Accordingly, the location and spread for each group is best described and compared using the median and interquartile range (and/or range), respectively. Measures of location and spread can be included on dotplots, but make sure you have selected them based on the shape of the distribution. If the dotplot function in the statistical package you are using defaults to adding measures of location and spread to the plot, check what the default settings are and make sure they are suitable. Statistical inference: the descriptive analysis showed that the data's empirical distribution is positively skewed; therefore, statistical inference based on the normal distribution is not appropriate. The most straightforward approach to statistical inference for skewed data is to transform the data's empirical distribution so that the transformed values are close to a normal distribution. This approach will be adopted in this tutorial and is explained in detail in section ‘Appropriate displays for statistical inference ’.

Displays for large samples

If the sample is large (>30 individuals), histograms and box & whisker plots are appropriate displays. Figures 3 and 4 are histograms and box & whisker plots of IgG titre in response to each antigen for the 93 malaria infected cases and 224 parasite-free controls. A histogram is constructed by dividing the data range into several non-overlapping equally sized bins (categories) and the number of observations falling into each bin counted. The bins are displayed on the x-axis and the frequency (or percent or proportion) on the y-axis. A box & whisker plot consists of a box which represents the inter-quartile range, that is 25% of the IgG titres lie above the top of the box (i.e. 75th percentile), the box itself contains the middle 50% of the IgG titres and 25% of the IgG titres lie below the bottom of the box (i.e. 25th percentile). The horizontal line within the box represents the median IgG level (i.e. 50th percentile). The whiskers end at the largest and smallest IgG values excluding any outliers. The outliers are defined as those observations greater than 1·5 times the inter-quartile range from the top or bottom of the box, and are represented as points outside the whiskers. Of note, the criterion for determining an outlier may differ between statistical packages, the definition we have given is used by the common statistical packages (e.g. Stata, R, SAS and SPSS). Histograms should be used to identify the shape of the data's empirical distribution. If the histogram reveals that the shape of the data's empirical distribution is normal or skewed, then box & whisker plots are more efficient displays of these shapes and are particularly useful for comparing how the location and spread of normal and skewed distributions vary across several groups (compare Fig. 3 with 4). Both histograms and box & whisker plots are better displays of larger datasets than dotplots, where the latter can look rather messy for a large number of observations. For more details on box & whisker plots, dotplots and other displays see (Freeman et al. Reference Freeman, Walters and Campbell2009).

Fig. 3. Histogram of IgG titres in response to each Plasmodium blood stage antigen from 317 pregnant women (224 non-infected controls and 93 malaria infected cases) included in the example dataset.

Fig. 4. Box & whisker plots of IgG titres in response to each Plasmodium blood stage antigen from 317 pregnant women (224 non-infected controls and 93 malaria infected cases) included in the example dataset. Box represents the inter-quartile range and the horizontal line within the box represents the median IgG titre. The whiskers end at the largest and smallest IgG titre excluding any outliers, and the circles outside the whiskers are outliers.

How should we interpret Figs 3 and 4 for a descriptive analysis and to inform statistical inference? Descriptive analysis: similar to dotplots (Fig. 2) the histograms and box & whisker plots (Figs 3 and 4) also display the differences in the distribution of IgG titre in response to each antigen between cases and controls, that is the IgG titre for controls are lower and less variable (spread of histogram is narrower and the box & whisker plots have a narrower box and smaller whiskers) than cases. The histograms (Fig. 3) and box & whisker plots (Fig. 4) show a tendency for the observations below the median to be contained in a narrower range than the data above the median, which indicates the IgG titre in response to each antigen for cases and controls are positively skewed. Statistical inference: will proceed as outlined in the previous section ‘Displays for small samples’.

Discrete variables

Discrete numerical data (whole numbers such as counts) are also very common in parasitology, in the form of parasitaemia (e.g. number of asexual parasites μL⁻¹ of blood), gametocytaemia (e.g. number of gametocytes μL⁻¹ of blood). Displaying discrete data are more challenging than displaying continuous data. Bar charts (although not ideal for displaying the empirical distribution of continuous data – see Fig. 1), can be used to display the frequency with which values of a discrete variable occur.

In the example parasitaemia dataset (see section ‘Example dataset description’ for a description) there are a large percentage (23% or 39/168) of women who did not have a malaria infection (i.e. parasitaemia measurement recorded as zero) and such data is often referred to as zero-inflated data. For zero-inflated data, we recommend plotting the non-zero measurements (e.g. parasitaemia in those who had a malaria infection) and reporting the percentage of zero measurements. Figure 5 is a bar chart of the parasitaemia data for those women who had a malaria infection (i.e. had non-zero parasitaemia measurements recorded). On the horizontal axis are the numbers of parasites, going from a minimum of 16 parasites to maximum of 11 078 parasites, while on the vertical axis is the frequency with which these measurements occur. The vertical axis could also be rescaled to percentages, which facilitates the comparison of groups. The bar chart shows that the empirical distribution of the parasitaemia measurements is highly positively skewed. However, if the discrete variable has a large number of unique values then plots suitable for continuous data (dot plots for smaller datasets, box plots or histograms for larger datasets) would be appropriate. Since there are 49 unique values for the parasitaemia data plotted in Fig. 5, it would be reasonable to apply plots for continuous data to this discrete variable (see section ‘Continuous variables’ for further details). Note the extreme positive skew makes the mean a poor location statistic for these data (mean number of parasites per host is 555, which is much greater than the median of 64 – these sample statistics were calculated including the zero measurements). In the case when the discrete variable consists of a large number of unique categories/counts (and consequently is amenable to analysis with statistical methods for continuous variables), a possible approach to make the distribution approximately normal (or less skewed) would be to take a log transformation (as illustrated in section ‘Transformation of IgG titre values’), but this is often impossible if the discrete variable has a high frequency of zeroes (O'Hara & Kotze, Reference O'Hara and Kotze2010). A brief discussion of how to analyse discrete data and what summary statistics to use is provided under ‘Discrete variables’ in the next section.

Fig. 5. The distribution of 129 non-zero discrete parasitaemia measurements presented in a bar chart. There were 39 zero measurements out of 168 observations.

Continuous variables

As we described in the introduction, statistical inference is performed to draw conclusions about the distribution of population parameter based on its empirical distribution. The ‘good’ displays of the IgG titre's empirical distribution showed that its shape in both the small and large sample size examples was positively skewed. In the following sections we will use the complete dataset (n = 317) to show how taking the natural log-transformation of positively skewed data can result in an empirical distribution that is approximately normal.

We also recommend statistical inference plots which display estimates (statistics used to estimate a population parameter) and 95% CIs (a plausible range of values for the population parameter of interest) rather than the mean ± one s.e. which corresponds to the 67% CI for the population parameter (i.e. the 67% CI is less likely to contain the true population parameter than the 95% CI).

Transformation of IgG titre values

Data can often be transformed to remove skewness and make the data's empirical distribution resemble a normal distribution. The histograms (Fig. 3) and box & whisker plots (Fig. 4) showed that total IgG titre in responses to all three antigens were positively skewed for cases and controls. When data are positively skewed a log-transformation can often be applied to each data point to make the distribution of the data resemble a normal distribution (note a log-transformation cannot be applied to zero or negative values). As mentioned earlier the parameters that govern the shape of a normal distribution are the population mean and s.d., and the sample mean and sample s.d. are used to estimate the population parameters.

Box & whisker plots are used to display the log_e-transformed IgG titres (Fig. 6). The log_e-transformed total IgG titres appear to be normally distributed, for example each box looks relatively symmetric around the median value and there are considerably fewer outliers. The length of each box also appears similar for cases and controls across antigens, which shows that the spread or variance of the antibody levels (log-transformed) is similar for the malaria infected cases and parasite-free controls. Histograms are also very useful for determining whether data are approximately normally distributed data (if the histogram is symmetric and roughly bell-shaped, and the mean is similar to the median, then the assumption of normality is typically accepted). Quantile–quantile (Q–Q) plots are another useful display for examining normality (see Kohler & Kreuter, Reference Kohler and Kreuter2012 for more details).

Fig. 6. Box & whisker plots of natural log-transformed IgG titres in response to each Plasmodium blood stage antigen from 317 pregnant women (224 non-infected controls and 93 malaria infected cases) included in the example dataset. Box represents the inter-quartile range and the horizontal line within the box represents the median log_e-transformed IgG titre. The whiskers end at the largest and smallest log_e-transformed IgG titre excluding any outliers, and the circles outside the whiskers are outliers.

Statistical inference plots

Statistical inference plots depict estimates and 95% CIs (defined in Table 1) for the population parameters of interest. We now demonstrate how to make statistical inferences about the population mean log_e-transformed IgG titre to the Pf merozoite antigen between cases and controls in the example dataset. In the previous section we established that a normal distribution is an appropriate model for the population distribution of the log-transformed data. The estimates are the sample mean log_e-transformed IgG titre to Pf merozoite antigen for cases and controls, and the limits of the 95% CI are calculated from: sample mean ± (1·96 × s.e. of the sample mean) where the s.e. of the sample mean equals the sample s.d. divided by √n (n is 93 for malaria infected cases and 224 for parasite-free controls). The value 1·96 in the 95% CI calculation for the population mean is suitable for this example because the sample size is large, however, when the sample size is less than 60 the 1·96 should be replaced with the 97·5th percentile of the t distribution with the degrees of freedom (d.f.) equal to n – 1. Once the sample size is larger than 60 the 97·5th percentile of the t distribution is close to 1·96, whereas for smaller sample sizes this value increases with decreasing sample size (i.e. more conservative CIs are calculated for smaller sample sizes).

We can interpret the results on the original scale (as log-transformed variables do not retain the original units of measurement) by exponentiation (typically e^x on a calculator or the exp()function in most statistical packages) of the estimate and the limits of the CI calculated on the log_e-scale. The exponentiated estimate is the geometric mean and the exponentiated 95% CI is for the population geometric mean, which have the same units as the outcome (in our example IgG titre measured in OD units). The geometric mean should closely approximate the median of the untransformed data if the log_e-transformed data are normally distributed.

In Fig. 7 the x-axis represents malaria exposure group (malaria infected or parasite-free), and the y-axis is the mean log_e IgG titre in the left panel and the back transformed results (i.e. the exponentiated mean log_e IgG response and corresponding 95% CI for population mean log_e IgG titre) on the original OD units in the right panel, with the dots representing the sample mean (arithmetic left panel and geometric right panel) for controls, the squares representing the sample mean for cases and the error bars portraying the 95% confidence limits for the population mean. The statistical inference plots in Fig. 7 suggest the population geometric mean IgG level (or, alternatively, the population mean log_e IgG level) in response to each antigen is higher for cases than controls.

Fig. 7. Left panel: sample mean log_e IgG titre in response to antigen Pf merozoite between cases and controls, and the 95% confidence interval (CI, error bars) for the population mean. Right panel: geometric mean log_e IgG titre (OD) into antigen Pf merozoite between cases and controls, and the 95% CI (error bars) for the population geometric mean.

Some researchers like to add P-values examining the strength of evidence against the null hypothesis (which, in this example, is that the population mean IgG level for cases is the same as controls) to statistical inference plots. If it is the researcher's preference to include P-values on such plots, we recommend stating the exact P-value and not presenting stars or symbols to indicate whether the P-value is below a particular threshold, e.g. <0·05 (see Fig. 1). When the P-value is very small, then stating, for example, the P-value is <0·001 is acceptable.

In situations where continuous data (original data values) are not normally distributed and a transformation of the data cannot alleviate this problem, then statistical inference plots of medians and 95% CIs for the population median (using statistical methods not covered in this tutorial, such as those presented in (Campbell & Gardner, Reference Campbell and Gardner1988), an approach based on the beta distribution and the bootstrap method) or box & whisker plots along with P-values from an applicable non-parametric test (e.g. Mann–Whitney U (Hart, Reference Hart2001)) are recommended.

Statistical inference plots are not restricted to displaying statistical inferences regarding population means and medians, but statistical inferences concerning any population parameter of interest can be displayed (e.g. proportions, odds, rates etc.) assuming suitable estimates and 95% CIs can be derived (see online Supplementary Figure 1 as an example of a statistical inference plot for the population proportion).

Discrete variables

Statistical inference plots are also difficult for discrete data, due to the combination of skew and zeroes mentioned above. Generally the mean or expected count (discrete value) is estimated by modelling the counts with an appropriate probability distribution for discrete data such as Poisson or negative binomial, or zero-inflated versions of these distributions that can account for the high frequency of zeros (Bolker et al. Reference Bolker, Brooks, Clark, Geange, Poulsen, Stevens and White2009; O'Hara & Kotze, Reference O'Hara and Kotze2010). Such modelling is beyond the scope of an accessible article on data display.

Descriptive detonator plots vs box & whisker plots

Descriptive detonator plots can obscure the shape of the data distribution by the use of summary statistics that are not robust (i.e. interpretation of the statistic changes depending on the shape of the data distribution) to skewness/outliers (e.g. mean and s.d.) or through the use of too few robust summary statistics (displaying only the mean and maximum). For example, in Fig. 8 detonator plots comparing the distribution of the raw (i.e. untransformed) IgG titres with each antigen between cases and controls are presented. The descriptive error bars are the sample mean plus one s.d. in panel A and the maximum in panel B. No information is presented for the distribution of the data below the top of the bar (i.e. below the sample mean). Much of the discussion relating to Fig. 8, panel A, is applicable to Fig. 1.

Fig. 8. Detonator plot of the distribution of IgG titres in response to each Plasmodium blood stage antigen from 317 pregnant women (224 non-infected controls and 93 Malaria infected cases) included in the example dataset. In both panels the bar indicates the mean. Left panel: mean plus one s.d. is the error bar. Right panel: maximum is the error.

Our examination of appropriate displays for the data's empirical distribution revealed that the distribution of IgG titres to each antigen for cases and controls was positively skewed. As mentioned earlier the effect of positive or negative skewness is to pull the mean above or below the median, respectively (i.e. the mean no longer reflects where the bulk of the data lies). For example in Fig. 8, the mean of the untransformed IgG tires is plotted, and without our previous investigation using the ‘appropriate’ displays the reader would not know that the mean is not the most appropriate measure of location.

In addition, the reader would also conclude from Fig. 8 that the difference in IgG titres for cases and controls is much larger than it appears due to the effect of the positive skewness (e.g. difference in means between cases and controls is 0·26, 0·30 and 0·17 for Pf merozoite, Pf-IE and Pv merozoite groups). The median (a measure of central tendency that is robust to skewness), however, indicates that the difference in IgG titres to each antigen for cases and controls is not as large as that displayed in Fig. 8 (e.g. difference in medians between cases and controls is only 0·08, 0·22 and 0·08 for Pf merozoite, Pf-IE and Pv merozoite groups – the medians for the groups are also displayed in Fig. 4 and are provided in Table 2).

We know from the boxplots in Fig. 4 that there is less variability in measurements from controls compared with cases and the IgG titres to each antigen for cases and controls are positively skewed, but such useful information about the spread/distribution of the IgG titres cannot be easily ascertained from either (or both) of the detonator plots in Fig. 8. The detonator plot with error bars equal to one s.d. suggests that the distribution of IgG titres is normally distributed, and can be summarized using the mean and s.d. If the data were normally distributed then 67% of the observations should fall between the mean ± one s.d., which is not the case for the IgG titres to Pf merozoite antigen from cases (the mean minus one s.d. is −0·1 but the 16·5th percentile is 0·04 OD units) and raises questions that the assumption of normality may not be well supported by these data. Drawing this conclusion from Panel A of Fig. 8 requires the viewer to roughly calculate the mean minus the s.d. (or the lower limit of the error bar). Skewness can only be diagnosed using this plot if the data are positive valued or the error bars contain implausible values. In all other situations it is impossible to diagnose skewness or deviations from normality using a detonator plot with error bar equal to mean + one s.d. The detonator plot displaying the mean and maximum is also not very informative and only shows you how far the maximum is from the mean. Since the mean (not the median) is being displayed we cannot be certain whether 50% of the observations lie between the mean and the maximum (without assuming the data are normal or symmetric) and cannot unambiguously determine whether the data are skewed (even if the minimum were included it may be difficult to diagnose skewness or deviations from normality because these single minimum and maximum values may be outliers). The scale of detonator plots beginning at zero is also misleading as zero values may be biologically implausible or not observed in the study sample.

The need to make strong assumptions (e.g. normality) or perform additional calculations in order to draw conclusions from descriptive detonator plots about the shape of the data illustrates that descriptive detonator plots cannot faithfully display the data and that box & whisker plots are a superior alternative.

Inferential detonator plots vs statistical inference plots

Inferential detonator plots are essentially statistical inference plots with the lower half of the inferential error bar removed and the mean displayed using a bar extending from zero rather than a point. Omitting the lower half of the error bar impedes the viewer's ability to visualize the lower range of plausible values for the population parameter. As mentioned previously, the sample mean of the original data values may not represent the true centre of the empirical distribution, as is the case for our data example (see Fig. 9).

Fig. 9. Inferential detonator plots of the sample mean IgG titre in response to antigen Pf merozoite between cases and controls (bars). Left panel: mean plus s.e. is the error bar. Right panel: upper limit of the 95% confidence interval for the population mean is the error bar.