Woodward's counterfactual theory of explanatory power

This paper subscribes to James Woodward's account of explanation where:

The structure of an explanation is an argument that employs a generalization in order to track physical dependencies. The quality of an explanation has to do with whether or not it can exhibit counterfactual dependency relations. Adequate explanations (according to Woodward) enable this by providing information that can be exploited to answer what-if-things-had-been-different questions (w-questions). Answers to w-questions enable us to see what the outcome would have been if initial conditions had been different. These counterfactual initial conditions oftentimes describe hypothetical situations that would result from changing (intervening on) the values of variables of the explanans. For Woodward, this ability to exhibit systematic patterns of counterfactual dependence is the criterion to establish whether or not an explanation is adequate. Woodward furthermore suggests that the more what-if questions an (adequate) explanation answers, the better it is (Woodward and Hitchcock, Reference Woodward and Hitchcock2003).

This paper endorses Woodward's counterfactual theory of explanatory power, but with the following qualification: better explanations are not simply ones that answer more what-if questions, but rather, following Ylikoski and Kuorikoski (Reference Ylikoski and Kuorikoski2010), better explanations answer more relevant Footnote ⁴ what-if questions than alternative explanations.

This criterion for explanatory relevance can be expressed in the following principle: The quality of an explanation is determined by how many relevant w-questions it can answer.

Structural equation modeling

SEM was developed in the beginning of the last century by geneticists and economists to combine qualitative cause-effect information with statistical data between variables of interest. It is heavily used in many social sciences because it can model latent variables and errors, while handling many types of relations between variables. A good introductionFootnote ⁵ on how to use and build these equations can be found in Pearl (Reference Pearl2009).

Typically, within the SEM literature, a functional causal model (representing an effect as a function of causes and noise) is a set of equations,Footnote ⁶ where some independent variables determine the value of a dependent variable, considering possible errors and/or non-represented factors. It is convention to use the “=” symbol, even though it is an asymmetrical relation that resembles the assignment function “:=”. Preferably, each equation in the system should represent what Woodward (Reference Woodward2003, p. 328) and Pearl (Reference Pearl2009, p. 27) call a (independent) mechanism.

The equations are to be understood as encoding information about how a variable changes if others were to change. By assigning values to the variables, we can understand how a system would (have) behave(d). In the simplest cases, variables are set to 1 if present and 0 if not, and through logical operators they lead to a certain value for the state of the phenomenon under consideration. Additionally, certain quantities (given by the solution of algebraic equations) can activate (or not) some qualitative variables (e.g., turn them into a 1 or a 0).

For instance, consider a toy example of a simple square table with separate legs. The table needs at least three legs to stand, and if it has a weight on it, it can stand as long as the weight is less than a certain amount. Let us idealize and suppose that the weight is always centered, and each leg can hold 5 kg before it breaks. We can model the possible states of the table as follows (where W is the weight (kg) placed on top):

$${\rm Leg}_{ i} = \{ {0, \;1} \} \,[ {\rm for}\,{i} = 1, \;2, \;3, \;4] \,( {\rm if}\,{\rm leg}\,{\rm present}, \;1, \;\,{\rm if}\,{\rm not}\,0).$$

$$\eqalign{&{\rm if}\left({\displaystyle{W \over {{\rm leg}_1 + {\rm leg}_2 + {\rm leg}_3 + {\rm leg}_4}} > 5} \right)\;{\rm then}\;{\rm Leg}_{i} = 0\; [ {\rm for}\,{i} \cr & \quad= 1, \;2, \;3, \;4] .}$$

$$\eqalign{{\rm Table}\_{\rm Stands}\,& = ( {\rm le}{\rm g}_1\vee {\rm le}{\rm g}_2) \wedge ( {\rm le}{\rm g}_3\vee {\rm le}{\rm g}_4) \wedge ( {\rm le}{\rm g}_1\vee {\rm le}{\rm g}_3) \cr & \wedge ( {\rm le}{\rm g}_2\vee {\rm le}{\rm g}_4) \wedge ( {\rm le}{\rm g}_1\vee {\rm le}{\rm g}_4) \wedge ( {\rm le}{\rm g}_2\vee {\rm le}{\rm g}_3) \cr &\quad ( {{\rm if}\,1, \;{\rm table\ stands}} )}$$

Using this scheme, we can explain the current state of the table (e.g., why it is standing or why it collapsed). This is because these equations represent physical dependencies; they represent causal interactions between weights and the breaking of legs. As a consequence, we can answer w-questions by filling-in the variables. For example, if there is no weight and there are three legs (say, the first, the second, and the fourth), then [Table_Stands = 1]. If there is a weight of 6 kg, and 3 legs, then [(6/3 = 2)<5], so [Table_Stands = 1], the table stands. Note that if the weight had been greater than 15, it would not stand (and yet it would, had it had 4 legs with a weight lesser than 20).

Glossary

• • Accuracy: The degree to which relevant (difference-making) factors are captured in a model.

• • Adequacy: The degree to which an explanation contains the information that can answer the explanatory requirements and aims of a particular audience.

• • Counterfactual: Hypothetical situation that would obtain if certain conditions were met.

• • Counterfactual dependence: An event E counterfactually depends on event C if and only if (i) if C had occurred, E would have occurred; (ii) if C had not occurred, E would not have occurred.

• • Explanandum: What is being explained within an explanation.

• • Explanans : The model or propositions from which one can derive the explanandum.

• • Robustness: The range of values that can be given to the variables of a model (while remaining valid).

• • w-questions: Questions that ask how something would have been different had a different set of conditions been the case.

In the following two sections, this procedure is applied to two case studies on failure analysis, one from Mechanical Engineering and one from Civil Engineering.

Explanation of the failure of a steering shaft and its transcription into structural equations

This section details the case studyFootnote ⁸ of a heavy road vehicle's steering shaft rupture, which looks at the investigation aimed at determining whether a fractured steering shaft was the cause or the consequence of an accident (Cleland and Jones, Reference Cleland and Jones1996). We can reconstruct the investigation in two steps: first, figuring out (through metallography analysis) whether the shaft had malfunctioned or, alternatively, that the rupture was a consequence of the accident. It turned out that the rupture was an effect of the accident. Given this result, the second step entailed explaining how the accident would transfer an amount of force large enough to break an otherwise “healthy” shaft. Their explanation is therefore not concerned with redesigning the shaft, but rather, with showing that the accident caused it to break. This means they take the characteristics of the shaft as fixed, that is, not in need of redesign, which would not happen in a redesign explanation (cf. the explanation of the section “Civil Engineering and accuracy”).

The shaft displayed unmistakable signs of shear failure (not only a visual inspection but a scanning electron microscope also confirmed shear failure and fibrous tensile failure at a micro level). Given that the metallography indicated that material properties did not contribute to the failure, they then pursued the second option: the accident had broken the shaft.

Since the question is how the shaft broke, given that the vehicle underwent an accident, and there were unmistakable signs of shear failure, the shaft's failure torque becomes the key variable. They considered the relations between several variablesFootnote ⁹ and failure torque. This allows to calculate the maximal force above which a rupture would happen: “the present analysis provides an upper-bound estimate for the failure” (1996, p. 17). Their calculations go as follows:

The equation for the torque required to cause shear fracture of a narrow concentric band is:

$${\rm d}\Gamma = 2\,\pi k_ur^2{\rm d}r,$$

where Γ is the torsional moment, k_u is the ultimate shear stress, and r is the radius.

Ultimate shear stress can be calculated from the empirical expression

$$k_u = \sigma _{{\rm TS}}/1.6,$$

where σ _TS is the tensile strength, which can be estimated from hardness data through

$$\sigma _{{\rm TS}}( {{\rm MPa}} ) = 3.2\,{\rm HV}.$$

And ultimate shear stress can be formulated as a function of distance to the center,Footnote ¹⁰ by an empirical equation of the form:

$$k_u( {{\rm MPa}} ) = 700 + 0.00878( {r/{\rm mm}} ) ^3.$$

Ultimate shear stress is integrated along the radius (from 0 to 23.5 mm) to get the maximal torque of the core:

$${\rm \Gamma } = 2{\rm \pi }\int_0^{23.5} {( 700 + 0.00878\cdot r^3) \cdot r^2\cdot {\rm d}r = 20.6\,{\rm MN}\,{\rm mm}}.$$

The shaft's case can be calculated by the formula of a cross-section:

$${\rm \Gamma } = \displaystyle{{2\cdot \pi \cdot \;k_u\cdot ( {r_1^3 -r_2^3 } ) \;} \over 3}.$$

Which gives a torque of 9.8 MN mm for the case. The total maximal torque is then 30.4 MN mm (9.8 + 20.6).

From here, it follows that

$$\eqalign{F_{{\rm max}}\ast {\rm length}\,{\rm of}\,{\rm steering}\,{\rm arm}\,( {250\,{\rm mm}} )\cr = {\rm total}\,{\rm torque}( {30.4\,{\rm MN}\,{\rm mm}} ).\quad\quad\quad}$$

Giving a maximum force of approximately 0.1216 MN, which is equivalent to 12.4 tf. This means that a force greater than 12.4 tf would likely break the shaft. The paper follows with the observation that a vehicle of 20 t with a (conservative) deceleration of 5 g has a collision force 100 tf (8 times more than is needed to break the shaft). In other words, the shaft broke as a result of the collision.

The authors conclude “The steering shaft had been subjected to a large axial torque, sufficient to cause gross yielding of the cross-section and fracture by a ductile mechanism. There were no indications that failure was promoted by prior defects or inadequate mechanical properties. If a small fraction of the likely collision force had been applied to the end of the steering arm, this would have been sufficient to cause failure. We therefore conclude that the failure was a consequence of the accident, and not its cause” (p. 18).

The explanation given by the authors can be regimented further by executing the first step in the procedure, namely the transcription of the variables and their relations into structural equations (which the engineers themselves did not provide) which can be used to set counterfactual scenarios by filing in the variables:

$$\eqalign{{\rm Total}\,{\rm torque} = {\rm core}\,{\rm torque}\left[{\int_0^{r_2} {( 2\pi \cdot \;k_u\cdot r^2\cdot \;{\rm d}r) } } \right] \cr+ {\rm case}\,{\rm torque\ }[ {2\pi \cdot k_u( {r_1^3 -r_2^3 } ) /3} ]\,\,\,\,\quad}$$

$${\rm If}\;( {\rm Force} > \displaystyle{{{\rm Total}\,{\rm torque}} \over {{\rm arm}\,{\rm length}}} + U) \;{\rm then}\;{\rm Shaft}\_{\rm Break} = 1; \;{\rm else} = 0$$

where U is error and omitted factors.

These structural equations provide a concise representation of the key ingredients of the explanation, highlighting what is epistemically relevant. It starts from the explanation (i.e., it assumes the validity of the explanation) in order to condense the relations that are of interest.

Identification of relevant w-questions

Moving to the second step of the procedure, we now need to identify the set of w-questions that (when answered) would satisfy the aim of the explanation. The original aim was to figure out whether the broken shaft caused or was a caused by the accident. An explanation at the very least should clarify this; however, once this minimum requirement is met, we can inquire whether it fulfills its aim optimally. To do so, it would need to satisfy as many relevant w-questions as possible. This means it should give us as much information as possible about when the collision would have caused the shaft to break and when a broken shaft would have caused an accident. The more scenarios (answers to w-questions) in which we know whether the shaft breaks or not, the better our explanation becomes. An example of such a relevant w-question would be “what-would-have-happened at half the collision force?”. More concretely, only the counterfactuals that consider changes in force are relevant.

If the aim had been to redesign the shaft, other counterfactuals would become relevant, such as ones considering how changes in the shaft's characteristics would affect the outcome, for example “what-would-have-happened if the total radius were increased by 2 mm”. Even though these questions provide redesign information, they are not relevant for the explanatory aim in the case at hand, which was to figure out whether the collision broke the actual shaft.

Exploration and corroboration of which w-questions can be answered

We need an explanation that can answer the set of w-questions related to different force values (“what-would-have-happened-if force was n”). The third step in the procedure is to figure out whether our explanation can indeed provide answers to this set of w-questions. The structural equations presented in the section “Explanation of the failure of a steering shaft and its transcription into structural equations” can help us answer many of the w-questions that can be formulated. By plugging values into the structural equations, we can obtain answers. For example, “what-would-have-happened-if the impact force = 2 tf”, where the answer would be 2 tf <12 tf, so Shaft_break = 0. We could also answer several irrelevant w-questions, that could nonetheless be interesting for other purposes (such as redesign). For instance, “what-would-have-happened if the arm length was 260 mm?”. If the arm length was 260 mm, then 100 tf > (30.4 MN/0.26 m), so Shaft_break = 1. This tells us that modifying the shaft length to 260 mm would not have prevented it from breaking. However, given the aim, we are only interested in w-questions about different force values.Footnote ¹¹

If the variable of force in our structural equations can capture a greater range of values, we will have more information about the conditions in which the shaft breaks or not (we will be able to specify a greater number of instances). Looking at the range of values force can take (while the model still outputs valid results), we can assess the number of scenarios that can be captured and hence how many w-questions can be answered. If a variable can take a set of integers ranging from 1 to 20, it will enable capturing more scenarios than a variable that can only be satisfied with a set of integers ranging from 2 to 5. This amount of range will be called robustness,Footnote ¹² which refers to how much change we can introduce in the variables of the equations of a model (or its assumptions) while keeping its results valid. This means that the greater the range the variables have, the more robust our model is; and by extension, the more robust the model, the greater the range of values its variables can capture (in this particular case, force).

By looking at the robustness of the model, we can learn about the size of the set of answerable w-questions, because the greater the range of force, the more w-questions (relevant to the aim) are answerable. Looking at the structural equations, we can see that the model is not well defined when Force ≈ (Total torque/arm length). When these two values are close, the result is uncertain (just how much will depend on U – the error). The model is valid in ranges where Force ≠ (Total torque/arm  length), so it is rather robust.Footnote ¹³ The model can tell us what happens to the explanandum when the variables take values in the ranges of [0, 12.3 tf) and (12.5 tf, ∞). If this range were shrunken, we would be able to capture fewer scenarios (see section “Validation and recommendations”, Figure 1 for a visual representation of the ranges), in the sense that the subset of scenarios would decrease (if U ₁ > U ₂, then the set of captured scenarios by the equations with error-term U ₁ is a proper subset of the captured scenarios with error-term U ₂. In other words, the set of scenarios captured – in this case – is inversely proportional to the error). By extension, if we could establish with precision what happens near the 12.4 range, we would increase the number of scenarios. In other words, an increase of robustness would increase the number of answerable (and relevant) w-questions. In explanations where the aim is similar to the current example, robustness is a good indicator of how good the explanation is. From this, we can assert that the explanation given by the engineers was good, since it does indeed answer a big array of relevant w-questions, which derives from the fact that their model is robust (it is possible to draw conclusions from substituting F with many values).

Fig. 1. Summary of the procedure applied to the case of a broken shaft.

Counterfactual comparison

The third step shows that our explanation is good (it satisfies its aim). The question now becomes whether it can be improved and how well it compares with others, which leads us to the fourth step: comparing our current explanation with possible hypothetical counterparts (where some factors are modified) to figure out whether the explanation can be optimized.

A model that maximizes the value range of force (enabling answering more relevant w-questions) is better; therefore, it is an advantage to have a maximally robust model. The question arises whether we can tweak the model to improve robustness further. There are several ways to do this. Theoretically, we could increase robustness by specifying more carefully the behavior of the system when the force approaches the breaking point (12.4 tf). But it is likely that improvements along those lines will come from improving external factors to the model, such as measuring capabilities. Another option is to try to increase the information relative to F. If each value of F captures a tuple of other values, its range (arguably) increases. Put differently, we can also increase robustness by enhancing the granularity of F, which can be done by expressing F in terms of new variables. If we expand F into the expression Kinetic energy (K)/distance (d), we cover more scenarios,Footnote ¹⁴ as each value of F can be obtained by a set of [K/d] pairs. By specifying  F = (K/d) = ((m ⋅ v ²)/(2 ⋅ d)) we can gain information about how the speed might affect the shaft failure (given the mass “m” of the vehicle and a deformation distance “d”). The distance will depend on the rigidity of the vehicle, and the velocity will depend on how fast it was going before impact. The reason expanding F provides more information is that we would know more about how different speeds might affect the breaking of the shaft. However, this is only useful if we can actually come to know the values of “d” and “m”. Including unknowable variables cannot improve the quality of the explanation, since one cannot set counterfactuals without knowing the value of said variables. Knowing which variables can be determined informs step 4 by telling us what modifications are sensible and which are not.

Increasing the granularity of F will improve the explanation, but only if this increase can be actualized (if we can determine and measure the variables that account for F). Such a decision needs to be considered by the engineering team working on the specific case. Ultimately the team must decide whether there are other (measurable) variables that can enhance granularity. However, should there be such a possibility, it is wise to implement it since it would improve the explanation.

Furthermore, we can illustrate how modifications that diminish robustness are detrimental to the quality of the explanation:

Suppose a hypothetical explanation of the same failure as follows. We know that the case's tensile strength is much higher than the interior, so we can set an unreasonably conservative upper bound limit by just taking k_u = 1760 MPa for all the shaft (instead of only the case), and calculate a torque that would certainly break the shaft by the following equation:

$${\bf \Gamma }\approx \displaystyle{{2\cdot {\pi }\cdot {k}_{u}\cdot {r}^3} \over 3}\approx 16.3\,{\rm MN}\,{\rm mm}.$$

This means that given an arm length of 250 mm, a force greater than 60 tf would surely break the shaft. Using the same back of an envelope calculation, the engineers used (20 t * 5 g of deceleration = 100 tf of impact force), we can explain how the shaft broke: 100 is much greater than 60 (which is already greater than the real limit). This represented by the following structural equation:

$${\rm If}\;\left({{\rm Force} > \displaystyle{{2\pi \cdot k_u\;( {r^3} ) /3} \over {{\rm arm}\,{\rm length}}}} \right),\;{\rm then}\,{\rm Shaft}\_{\rm Break} = 1.$$

While this also explains the failure, it seems of an inferior quality.Footnote ¹⁵ The first thing to notice is that robustness is reduced when using only the equation of the case instead of a more precise model that uses the equation of the case and the equation of the core (see Fig. 1 for a visual representation). All the scenarios that our hypothetical explanation captures can also be captured by the engineer's explanation, but their explanation also captures ones where the collision force is smaller. Put differently, our hypothetical explanation tells us that the shaft breaks if force is greater than 60 tf but cannot tell us much about numbers below 60. The engineer's explanation tells us what happens if the force is greater than 12.4 tf. This means that it captures a greater range (all the values between 12.4 and 60). Additionally, the engineer's explanation can also account for what happens below 12.4.

Validation and recommendations

Based on the results of steps 3 and 4, we can validate the explanation (robustness is maximized). If we applied the procedure to the hypothetical explanation from the previous section, it would not be validated, since its robustness could be improved. The illustration below summarizes the process (Fig. 1). The present analysis provides a means to engage with the explanatory content by considering how well suited it is to answer relevant w-questions and whether it can be improved. It, therefore, provides tools to gain understanding as to why the available explanation is adequate or not.

Consider, by mode of contrast, that the aim of the explanation had been to adjudicate responsibility for the accident. In such a case, the relevant w-questions would not be answered by the current explanation (and robustness would not be the virtue needing optimization). In such a scenario, an adequate explanation should have included other factors (e.g., the model would include variables of actors which could be set to 0 or 1 to check whether their involvement was conducive to the accident or not). This information is not present in the explanation (a fact that is easily observable by looking at the structural equations) and hence it is not adequate for the aim of adjudicating responsibility.

Given that the main aim was not redesign, the procedure need not be used to give any recommendations. Nevertheless, it would be possible to use the structural equations to question what the result would had been with a modified design. For example, we could change the values of length of the shaft and check if it would have broken under 12.4 tf. Other relations between variables can also be established by using the equations (i.e., by inputting certain values into variables of interest one can obtain the value of other variables). We now turn to a case in which redesign was the main aim of the explanation.

Collapse in a convention center and its transcription into structural equations

This section details an explanation given by the independent firm Wiss, Janney, Elstner Associates (WJE), hired by the owners of the David L. Lawrence Convention Centre to explain the collapse of an expansion joint in their convention center. The final aim of WJE's explanation was to redesign (and fix) the failed expansion, which was implemented by Thornton Tomasetti (a hire of the original architect) under their supervision.Footnote ¹⁶

The 4-story convention center was rebuilt in 2003. An expansion joint (with 25 slots) split the center in two. Joints are often one of the weakest points in structures. Their function is to connect while absorbing tension, temperature-induced size changes of the connected parts, vibration, etc. (Delatte, Reference Delatte2009). In 2007, a tractor-trailer parked on the second floor collapsed a concrete slab. The failure had occurred at an expansion joint exposed to an ambient temperature of −19°C. The colder a joint is, the more open it is, making it more likely to collapse under weight. Upon investigating the incident, it came to light that in 2005 a similar failure had occurred, resulting in a beam dropping onto a column. At the time, this incident had not been disclosed to the relevant authorities and was not considered relevant.

The reconstruction by WJE of the details goes as follows:

• Temperature-induced displacement can be calculated byFootnote ¹⁷: δT = α(ΔT)L.

• Additionally, the displacement caused by load can be calculated byFootnote ¹⁸: ${\rm \delta } = {{PL} /{AE}}.$

• And if thermal deformation is restrained, the force build up is: $P = \alpha ( {\Delta T} ) A E.$

They estimated the amount of free movement required to be 41 mm, assuming a temperature change of 28°C. This is because the α of steel is 10⁻⁵ mm/mm/°C and the (half) length of the building was 133 m. A finite element analysis showed that the distortion (of the high-strength steel angles) generated 630 kN of tension at the connection welds, with 8 mm of displacement. It was also noted that “[w]ith lower strength A36 steel, the force on the welds would have been reduced by 40%” (Delatte, Reference Delatte2009, p. 210).

Slots should be long and loose enough, and the bolts centered, allowing free movement that would not lock the joint or bear against the edge. Other possible factors for locking the joint might be corrosion, paint, and debris.Footnote ¹⁹ These, among other problems, are why the Manual of Steel Construction of the American Institution for Steel Construction suggests either double line of structural columns or low friction sliding connections for this type of joint (1998, part 8).

To make matters worse, the slots were welded only in the outer edges, making them weaker. When the tension grew too strong, they simply pulled free. The main investigation (WJE, 2008) concluded:

The fix consisted in welding 1-foot-square steel seats with Teflon pads underneath the joints of the replacement beam (and 25 others). This structural design element was indicated in early drawings but never executed (Rosenblum, Reference Rosenblum2007).

The explanation can be further regimented by executing the procedure's first step:

• If (Displacement > Room_Available), then Not_Enough_Room = 1.

• If (Total_Tension > Welding_Resistance − U), then Too_Much_Tension = 1.

• Pulls_Apart = Not_Enough_Room ^ Too_Much_Tension.

*Where Displacement is calculated by: δT = α (ΔT) L.

*Room_Available is a fixed design feature.

*Total_Tension is calculated by the tension generated from thermal change: P = α (ΔT) A E plus any weight there might be on top (such as a tractor trailer): P = m ⋅ g cosθ.

*And Resistance is a feature of the material where U accounts for imprecise factors such as corrosion, paint, debris, and maintenance.

Note that for Pulls_Apart to be 1 you need both conditions (if there is enough room, there will not be too much tension, if there is no room but not enough tension, it will not pull apart). These structural equations provide a concise representation of the key components (ingredients) of the explanation, highlighting what is epistemically relevant: the relation between variables (such as displacement and room available) and their connection to whether the joint fails or not.

Identification of relevant w-questions

In the second step of the procedure, we need to identify the set of w-questions that (when answered) would satisfy the aim. The aim of the explanation was to offer information that could be profitably used to redesign the malfunctioning element(s). This means that we need to answer the set of questions that tell us whether certain modifications of the current design would prevent future failure. We need information about the conditions in which the inclusion, exclusion, or modification of certain factors of the joint prevents failure. The aim of redesign is maximally fulfilled when the explanation can answer numerous w-questions that relate to how changes might prevent the future failure of the joint. An example of such a question would be “what-would-have-happened if instead of only steel ASTM A92 slots the joint would have used steel ASTM A36 and a Teflon supporting bracket underneath?” Note however, as with the previous case, that not all w-question are relevant. For example, “what-would-have-happened if the temperature the night of the accident was 0°C?” is not a relevant w-question. While it might tell us why it collapsed the day it did and not earlier, it does not give us any information about how certain modifications of the joint would prevent it from failing. This contrasts with the previous case study, where we kept the shaft structure fixed and considered how changes in force would or would not break it. Here, we consider the temperature drop fixed (to a certain extent) and consider how changes in the design would have averted the collapse. This is not to say that relevant questions should not consider worst-case scenarios in the background (such as even more dramatic temperature drops), but the focus should not be on tracking the dependencies between factors external to the design (like temperature) and joint failure. The focus should be on modifications of the design and how they affect the outcome (while keeping in mind possible roles played by external factors).

If the aim were (e.g.) to figure out why it collapsed when it did and not a day before, other counterfactuals would be relevant, such as ones considering variations in temperature and how they affect the success or failure of the joint (captured by 0,1 values of “Pulls_Apart”). But the aim is to redesign. In terms of our structural equations, the modifications that are informative are ones that modify the value of the variables “Welding_Resistance” and “Room_Available” (which could be further cashed out in terms of materials and dimensions if need be).

Exploration and corroboration of which w-questions can be answered

We need an explanation that can answer the set of w-questions related to modifying the current design. The third step in the procedure is to determine whether the explanation can indeed provide answers to this set of w-questions. The structural equations presented in the section “Collapse in a convention center and its transcription into structural equations” can help us answer many of the w-questions that can be formulated. By plugging values into the structural equations, we can obtain answers. For example, “what would have happened if instead of ASTM 92 steel the brackets would have used steel ASTM A36 and incorporated a Teflon coating low friction bracket?” The tension would be reduced dramatically, meaning Total_Tension < Welding_Resistance, So Pulls_Apart = 0. (It would also be possible to give these details in a fine-grained manner calculating relative values of tension and so on, but we choose latent variables to make the argument easier to read).

We could also answer several irrelevant w-questions. For instance, “what-would-have-happened if the maximum temperature was 28°C and the minimum 25°C?” Then Displacement would be: α (ΔT) L ≈ 4 mm, so Displacement < Room_Available, which means Pulls_Apart = 0. This would explain why it did not collapse on a given day (with those temperatures). However, even though the result is favorable (the joint does not collapse), this is not an interesting question (for our aim): we cannot intervene in order to modify temperature so as to avoid future collapses.

The quality of the explanation hinges on how well it answers relevant w-questions. In order to provide adequate information for answering such questions, the explanation must contain a detailed representation of factors that make (or could make) a difference to the outcome (failure or success of the joint). To see why, consider how if one ignores an operative factor, it might end up causing a failure in the future. In fact, this is what happened in 2005, when an incorrect evaluation of the difference making factors resulted in inaccurate engineering: not adding a low-friction support bracket ultimately led to the failure in 2007.

The importance of capturing elements that make a difference can be seen by how the engineers consider different factors as being causally relevant or not. For example, they consider insufficient room for thermal contraction as a difference maker, but they also consider other factors which were not difference makers, such as the length of the slot, limitation on bolt torque, whether the bolt was centered, paint, corrosion, and debris.

The greater the number of (causal) factors (i.e., factors that make a difference) that are identified, the better our explanation becomes, since it leaves fewer important things out. This virtue is called accuracy. By looking at the accuracy of the model, we can learn about the size of the set of answerable w-questions, because the greater the number of operative factors present in the explanation, the more w-questions (relevant to the aim) are answerable.

The number of relevant factors can be accounted for by looking at the structural equations, since they capture the factors that are deployed in the explanation. If we have a greater number of variables in our structural equations, we can capture more scenarios, hence we will have more information about the conditions in which modifying the joint in different ways leads to a collapse or not (we will be able to specify a greater number of relevant instances).

An increase in accuracy would increase the number of answerable (relevant) w-questions (having more factors enables formulating a greater number of counterfactuals). Furthermore, it is because the explanation is accurate that a considerable number of w-questions can be answered. This means that in explanations where the aim is similar to this one (redesign), accuracy is a good indicator of the quality of the explanation.

WJS's explanation does indeed answer a big array of relevant w-questions, because their model is accurate (it contains several variables that make a difference and considers many possible candidates that upon further evaluation are not considered to make a difference).

The accuracy of their explanation results in the possibility of repairing the failure (redesigning), which was the main aim of the explanation: “A more reliable detail for this type of connection is a low-friction supporting bracket ( … ) the bolts were removed and Teflon-coated supporting seats were added” (Delatte, Reference Delatte2009, p. 210). The recommendations (and actions) to improve the problem were based on the accuracy of the explanation, namely the need to lower friction and augment room for displacement. Furthermore, the accuracy of the explanation allows to know how the fixes should be implemented (e.g., the dimensions of the supporting seats will depend, among other things, on the expected thermal expansion).

Counterfactual comparison

The third step shows that the explanation is good. The question now becomes whether it can be improved and how it might compare to other alternatives, which leads us to the fourth step: comparing the current explanation with possible hypothetical counterparts (where some factors are modified).

The failure analysts aimed for accuracy in their investigations as can be seen in their quite thorough consideration of what factors played a role and which ones did not contribute. What needs to be analyzed is whether accuracy can be further improved.

The decision as to whether all relevant bases were covered (i.e., accuracy was optimized) relies on the team of engineers with access to the whole information about the case. However, what can be said is the following: if the aim of the explanation is redesign, should there be a factor that affects the displacement, it needs to be included in the explanation in order to make the explanation more accurate (and the redesign more effective). What the procedure can establish is how many factors are indeed included in the model and compare competing explanations to show that the one that is more accurate will perform better. To see why, we can use a hypothetical explanation that is less accurate.

Contrast the engineers’ explanation with an (worse for this aim) explanation that would pursue simplicity. To do so, one could for instance remove parts of the structural equations or leave certain details of the explanation out. Consider ignoring the first equation, that refers to having enough room. In theory, the failure could just be accounted for by simply referring to the tension overload. It would be possible to build an explanation that only focussed on the tension generated in the slots, without paying attention to the available room. For example:

• If (Total_Tension > Welding_Resistance − U), then Pulls_Apart = 1; else = 0.

This would in theory explain the failure: if the tension is greater than it can hold, it will break. It does tell us to some extent that if the tension had been smaller or the welding resistance larger it would not have broken. However, this is very limited. The simplicity of the explanation could in some cases benefit the explanatory aims of certain audiences (e.g., in a court room or a classroom), but it certainly would not aid the implementation of sound changes to the construction.

Our hypothetical explanation misses the point, since it would only recommend changing the material of the bolts, which would not address the underlying issue of thermal expansion (leading to possible complications down the line). This thought experiment highlights that in order to have an explanation that provides the right sort of information toward redesign, one needs accuracy.

We could also compare it to the explanation of 2005 that did not address the problem in the first place:

Had their explanation been more accurate (i.e., identified the factors that would cause a problem later on) and had they acted upon the knowledge it would have provided, it would be fair to say that there would not have been a collapse in 2007. If they had not only considered the tension but also the room required for thermal expansion, they would likely have re-evaluated the design as needing a support bracket (as was done after the collapse in 2007).

Validation and redesign recommendations

The results of step 3 show that the explanation is adequate (it is a good explanation). The results of step 4 show that it is optimal for the aim and is better than the alternatives considered. Given that accuracy is maximized in the engineer's explanation, it can be validated. We can easily see that both our hypothetical explanation and the explanation given in 2005 are not validated (a visual summary can be found in Fig. 2). The procedure, therefore, delivers insight as to why certain explanations are adequate and others are not, while allowing to improve those which are not. Similarly, it provides resources to justify the preference of certain explanations over others, while encouraging critical reflection on the results of failure analysis.

Fig. 2. Summary of the procedure applied to the case of the convention center.

Furthermore, the procedure can be used to provide information useful toward redesign. Accuracy is key to give the right recommendations (in this case, the addition of a low friction supporting seat); but it is also important for the correct implementation of the recommendations. Once the solution of a low friction supporting seat is proposed, there are other things to consider such as what materials to use (e.g., Teflon bearing vs. other elastomeric pads), the dimensions of the support bracket, etc. All these details need to be adapted to the case at hand, meaning that the implementation of the solution will benefit from an improved accuracy that takes into account the specific details and factors of the explanation. The information encoded through the procedure allows to make inferences about what type of redesign implementations are reasonable. For example, to know the dimensions of the supporting brackets, one must know not only the dimensions of the joint, but also how much it is expected to expand in unfavorable weather conditions. Using the structural equations, we can plug in such unfavorable conditions (e.g., a 40° drop in temperature) to see what the expansion would be, and by extension how large the support should be. A similar argument can be made for the resistance of the material needed for the supporting seat.