Model-based archaeology aims to use computer simulation to improve our understanding of the past. Simulation is widely used in other sciences as a major research tool but its adoption within archaeology has been not an easy one. The first archaeological simulations were developed almost four decades ago and provided interesting examples of its potential. Despite these benefits, simulation is still not popular in mainstream archaeology, in contrast with other computational tools such as Geographic Information Systems. However, the situation seems to be gradually changing as can be observed in the large number of recent reviews and special issues focused on this method (Madella et al., Reference Madella, Rondelli, Lancelotti, Balbo, Zurro, Rubio-Campillo and Stride2014; Wurzer et al., Reference Wurzer, Kowarik and Reschreiter2015).
The adoption of computer models in archaeology is rather unique. Being placed at the junction between the humanities and science, most archaeologists do not receive a strong mathematical training. This is radically different to the fields where simulation is typically found: physics, chemistry, and more recently biology. This unusual situation has raised serious issues because essential components of the modelling methodology are being generally ignored in archaeology. This is even more relevant for Agent-Based Modelling (ABM), a popular type of model that is theoretically closer to archaeological thinking than other models (i.e. equation-based models). Platforms such as NetLogo (Wilensky, Reference Wilensky1999) allow researchers with no mathematical training to develop a simulation in minutes. This is an excellent tool to learn how to create a model, but what about the rest of the method steps such as validation or experiment design? How can we analyse complex models without a proper mathematical background? It seems that sometimes we gladly embrace these tools without critically discussing their assumptions, complexities, and challenges or even thinking if they are equipped to deal with archaeological case studies.
The aim of this edited volume is to address these concerns. The book derives from a forum held in 2014 at the Meeting of the Society for American Archaeology (SAA) in Austin, Texas under the title Error, Sensitivity Analysis, and Uncertainty in Archaeological Computational Modeling. Sensitivity Analysis (SA) is the family of methods designed to study how the output of a model is linked to the input parameters. SA is typically performed to assess the degree to which each variable is affecting the result. This task combines tests exploring how slight variation on one parameter affects the final outcome of the model, or finding regions of the parameter space where the model gives specific outcomes. In essence, SA tries to assess the impact of uncertainty in the model itself. This uncertainty comes from numerous sources (e.g. biases in datasets, parameter values, stochasticity) and ideally it should be measured to better understand any model. This step of the modelling process is even more critical in archaeological simulations, considering the high levels of uncertainty and biases in the evidence we use. However, this discussion is not really happening. Why are we ignoring uncertainty? How can we address this situation? What methods are more interesting for model-based archaeology?
These are the questions that the contributors to this volume try to answer. The first two chapters set the context for the remaining papers. In the introductory Chapter 1, Brouwer Burg et al. argue that ‘issues of uncertainty and model validation have remained in the background, which suggests that we take this fact for granted, or that the issue is too delicate to handle’ (p. 6) and summarize how SA is used in other disciplines connected to archaeology (geosciences, ecological and social modelling). The discussion is complemented by the methodological challenges highlighted by William Lovis in Chapter 2 (‘Is There a Research Design Role for Sensitivity Analysis (SA) in Archaeological Modeling?’). This first section of the book provides a good starting point to explore what similarities and differences can be found between archaeological models and other disciplines. Especially relevant here is the nature of archaeological evidence: the observations we use are not collected from the object of study, but from the patterns in material culture caused by the object of study (i.e. social behaviour). The consequence of this difference is an increased sense of equifinality as we cannot be sure that the social dynamics we model are the ones that generated the evidence, even if there is an exact match between a model's outcomes and evidence. Both chapters end up with a cautionary note, as Lovis suggests that ‘we may be trying to do more with our models and modeling than our data most often allow’ (p. 33). This is certainly the case for a large percentage of archaeological simulations that seem to be over-parameterized and under-analysed; they include huge amounts of uncertainty that is not properly examined due to the curse of dimensionality.
The second section is an illustrative set of examples including GIS- and agent-based models. These are probably the most practical pages of the volume as each chapter examines the use of SA for specific case studies. One of the interesting ideas here is that most of these models are already published and were re-evaluated for this volume. Chapter 3 (Peeters & Romeijin) explores how variability in parameters can affect the result of a spatial model and the use of proper experiment design to understand these dynamics. Particularly relevant here is the discussion of the evaluation of multiple competing hypotheses and how uncertainty can affect this model selection. Chapter 4 (Brouwer Burg) extends this discussion to models integrating environmental and social dynamics. These socio-natural models have an added risk to the one posed by data uncertainty because they combine evidence from very diverse sources. How do you integrate these data sources if some of them have higher resolution than others? What if the modeller does not have enough data to define a specific component of the model? As Brouwer Burg points out, we should always be cautious regarding the idea of adding all available evidence to a model, because more data does not mean that the model is more realistic (it is only more detailed). The chapter also discusses specific SA methods that could help us identify potential issues concerning this difficult balance. Corner-test SA is probably the most useful one here as it can identify the boundaries where the model behaves as predicted.
Chapters 5‒7 give valuable insights into common aspects found in most agent-based models. Carroll (Ch. 5) explores the impact of stochasticity in a model of cultural transmission. This is an illustrative example of how a proper experiment design integrating sensitivity analysis can increase our understanding of the model. Watts (Ch. 6) discusses the concept of scale for a diversity of variables (time, space, and population size). He also uses correlation tests to provide an easy-to-use approach to computing the correct number of simulations that an experiment should ideally run. In the last chapter of this section (Ch. 7), White introduces the concept of population dynamics. Demography is an essential component in several agent-based models, but it is particularly difficult to calibrate because fertility and mortality are typically expressed as non-linear dynamics. The chapter presents an intriguing approach based on a regulatory mechanism designed to shift the system to an equilibrium point. From these three works a set of very practical guidelines can be extracted, including the exploration of multiple values for each parameter and the need to give details on every decision made during the entire modelling process, from the number of time steps to the spatial resolution.
The last section includes two chapters wrapping up the issues discussed thus far while suggesting some future challenges. Chapter 8, ‘Archaeological Simulation and the Testing Paradigm’, is the most thought-provoking piece in the edited volume. First, Thomas Whitley lists a collection of assumptions that any simulation has and that is often forgotten by the modellers. While this is an interesting discussion, I suspect that the author had specific types of simulations in mind when he compiled the list. For example, he suggests that decision-making needs multiple inputs because ‘a single criterion alone will not be sufficient to allow agency in a mechanistic decision’ (p. 140). I do not see why this assumption is needed as one could base a decision on a completely random variable (i.e. tossing a coin). Stochastic decision-making is effectively used to model several mechanisms of social learning such as random copy. Similar points can be raised for other listed assumptions such as the idea that decision-making must be probabilistic. Second, the author suggests here a new classification for archaeological simulations based on the goals that drive them. The division is based on three categories: a) simulations as re-creations; b) simulations as data mining; c) simulations as explanatory tools. Again, Whitley focuses the discussion on specific models and within this context the classification is potentially useful (see Lake, Reference Lake2014 for a more general classification of archaeological simulations). Finally, the author challenges one of the pillars of scientific modelling (the law of parsimony) by arguing that ‘We inherently want to believe the simplest explanation possible. But mechanisms do not operate that way’ (p. 150). I disagree with this critique because simpler models are better than equivalent detailed models if they have similar explanatory power. This is Occam's razor in action and it is a basic assumption of science for multiple reasons: simpler models are easier to understand, analyse, and replicate. This idea is also present across the rest of the papers in this volume, and the debate certainly provides interesting insights into the nature of archaeological simulations.
The last chapter, titled ‘Uncertainties’, by van der Leeuw, provides a summary of the book by extending the scope of the discussion beyond archaeology. The author links the examined topics to current general debates on complexity science, such as the existence of tipping points or the concept of risk. This is a very interesting conclusion that suggests the way these work can help us understand not only the past but also the present.
Overall this volume is a welcome addition to the current debate within model-based archaeology. It is surprising, though, that the authors did not provide any example of evolutionary archaeology, arguably the only field of the discipline where formal models are the standard method of hypothesis testing. Uncertainty is an essential component of any evolutionary framework and for this reason archaeologists working on cultural evolution have tackled similar issues to the ones present in this volume, such as time-averaging (Premo, Reference Premo2014), non-equilibrium systems (Kandler & Shennan, Reference Kandler and Shennan2013), and model selection (Crema et al., Reference Crema, Edinborough, Kerig and Shennan2014). This omission is probably a consequence of the growing adoption and diversity of simulations across the discipline. In this context, methodological discussions such as the ones addressed in this book are essential if we want to transform simulation into a useful tool for all archaeologists.
