2.1. Problem assumptions

Distributed design principles provide a model to analyze the behavior of decomposed design problems with noncooperating or noncoordinated subsystems. In decomposition approaches, either the overall design system is naturally decomposed or a design manager delegates design responsibilities to different subsystems.

A distributed design process is iterative, and it possesses a nonhierarchical design structure with design subsystems that selfishly minimize their own objective function. Design subsystems have responsibility to determine the value of their assigned design variables. The major advantages of distributed design, as compared to other multidisciplinary optimization approaches, are that it is straightforward to implement for a design system and requires limited oversight in its execution. Such approaches include game theoretic formulations (Vincent, Reference Vincent1983) and other multidisciplinary design optimization approaches of independent subspaces (Martins & Lambe, Reference Martins and Lambe2013).

The disadvantages of the distributed design approach include the potential for the design process to diverge and that there is no guarantee the converged solution is optimal. In spite of this, distributed design models provide insight into many design problems. The noncooperative subsystem properties that characterize distributed design include the following: designers have knowledge of only their own objective function; designers act unilaterally to minimize their own objective function; designers have complete control over specific local design variables; and designers communicate by sharing the current value of their local design variables.

Distributed design problems exist as stand-alone problems and as subproblems in a larger design task. As long as these subproblems fulfill the four assumptions of noncooperation, distributed design techniques can effectively model the system behavior. For example, when the number of iterative loops in a design structure matrix cannot be reduced to zero, the subproblem can often be modeled as a distributed design problem (Denker et al., Reference Denker, Steward and Browning2001).

Distributed design problems have two characteristics of primary concern for system analyzers. The first is the equilibrium stability, which determines if the system equilibrium is convergent, divergent, or a saddle point. The second is the transient response, which determines the time required to reach the equilibrium solution for convergent systems. We discuss these characteristics in the next sections.

2.2. Equilibrium stability

Vincent (Reference Vincent1983) established the foundation for stability in distributed design using a game theoretic approach. Chanron and Lewis (Reference Chanron and Lewis2004, Reference Chanron and Lewis2006) extended Vincent's game theoretic model by representing the subsystem objective functions using a state space to apply control theoretic principles to determine system stability. Another approach analyzed equilibrium and convergence rates of coupled systems using only subsystem input/output information found in a design structure matrix representation (Smith & Eppinger, Reference Smith and Eppinger1997). Chanron et al. (Reference Chanron, Singh and Lewis2005) demonstrated that some design systems have multiple equilibria, each of which may be stable, unstable, or a saddle point. The approach in Chanron et al. (Reference Chanron, Singh and Lewis2005) included nonquadratic objective functions using nonlinear system theory and Lyapunov functions to identify convergent regions of the design space. While this paper considers systems with linear rational reaction sets, Chanron's characterization of nonlinear systems demonstrates the broad applicability of these techniques and the ability to expand our research to more general nonlinear cases.

Initial research in distributed design assumed that the structure of the design problem, called the solution process architecture, played no role in determining system stability (Smith & Eppinger, Reference Smith and Eppinger1997; Chanron & Lewis, Reference Chanron and Lewis2004). However, Devendorf and Lewis (Reference Devendorf and Lewis2011) demonstrated that process architecture changes can influence the stability, and they presented an approach to determine the stability for a set of parallel/sequential architectures. The influence of solution process architecture on transient response was first investigated by Devendorf and Lewis (Reference Devendorf and Lewis2013), and they showed that the rate and characteristic shape (e.g., exponential decay or decreasing amplitude sinusoidal) of convergence could be changed with architecture changes. Before examining the broader role of solution process architecture in determining the properties of distributed design systems, we first discuss transient response and then process architecture in Section 2.3.

2.3. Transient response

The transient response of a system describes the convergence rate and the shape of the convergence curve for the entire system. We refer to convergence rate in this work as the settling time as measured by the number of iterations. An iteration represents a single step through the entire system optimization process with each subsystem solving its individual optimization problem once. We do not consider the algorithmic convergence related to the solution of each subsystem's individual optimization problem. In this paper, we examine the settling time aspect of transient response. Our definition of settling time, consistent with control theory, is the number of iterations required for a system to converge to within 2% of its final value as measured from its initial value (Ogata, Reference Ogata2005). We use settling time as a metric for our experiments to provide a repeatable definition for convergence and to remain consistent with previous work examining transient response. Previous research (Devendorf & Lewis, Reference Devendorf and Lewis2013) demonstrated the dependence of settling time on process architecture. From a heuristic perspective, social network theory has shown that the settling time is dependent on the relationship between the process architecture and subsystem coupling (Devendorf et al., Reference Devendorf, Devendorf and Lewis2010).

Taken collectively, previous work examining equilibrium stability and transient response has demonstrated that process architecture is fundamental to the analysis of distributed design systems. In the context of distributed design, the term architecture refers to the ordering of the solution process. It does not refer to product architecture, which is an independent and significant area of design research (Ulrich & Eppinger, Reference Ulrich and Eppinger2012). Instead, it refers to the ordering in which the design subsystems are solved. This ordering can include both parallel/sequential and simultaneous elements. We show an example of a purely parallel or sequential and a purely simultaneous architecture in Figure 1 along with a hybrid architecture that incorporates aspects of both.

Fig. 1. Solution process architectures.

A change to the process architecture modifies design variable sharing in the system. When subsystems iterate in parallel with one another, they cannot share their design variable determination until the next iteration. For example, in the case of the hybrid architecture in Figure 1, Subsystem 3 will perform its optimization based on the new value of y before Subsystem 1 has the opportunity to incorporate y into the determination of its own design variable values.

From a structural perspective, specifying the process architecture is the only way to modify the behavior of a distributed design system without changing its constituent subsystems or violating the underlying noncooperative assumptions. Understanding the underlying deterministic behavior of distributed design systems remains important, but it neglects those cases when the system experiences stochastic inputs, interruptions, and changes. We discuss and define these design impulses in Section 3.

3.1. Local impulses

The first type of impulse, a local impulse, describes impulses that originate within a decision network and are defined as

• impulses caused by an event whose origin can be attributed to design resources controlled by, and whose initial impact occurs in, a single design subsystem.

Local impulses propagate across decision networks as subsystems share and exchange decision information across the network. The propagation path depends on the coupling that exists between subsystems in decentralized decision networks.

In this paper, we consider the local impulses that occur due to bounded rational designers. In bounded rationality, decision makers have deterministic preferences but make stochastic choices (Thurston, Reference Thurston2001). Gurnani and Lewis (Reference Gurnani and Lewis2008) used bounded rationality to model mistakes in distributed design and to analyze the equilibrium properties of bounded rational decision makers. The model we use in this paper is consistent with Gurnani and Lewis's work and is identical to the model used by Devendorf and Lewis (Reference Devendorf and Lewis2010) to investigate the role of local impulses in transient response. Figure 2 provides a general formulation for a bounded rational decision maker.

Fig. 2. Bounded rational decision maker.

In Figure 2, X* is the expressed preference of the decision maker for decision variable X, V(p,q) is a value function that represents the precise preference of the decision maker, while ε represents the stochastic portion of the decision. The value function V(p,q) is dependent on p, the local variables controlled by the designer, and q, the nonlocal variables in the system not controlled by the designer. Together, p and q yield the designer's rational reaction set, which embodies the value function V and is a direct representation of the fully rational and precise preference of the decision maker. We represent ε as a normal distribution centered at the origin, and the standard deviation of ε captures the level of irrationality the decision maker exhibits. As the distance from the precise preference in Figure 2 increases, the decision becomes more irrational.

3.2. External impulses

In contrast to local impulses, we define external impulses accordingly:

• an impulse caused by an event outside the decision network whose origin cannot be traced to any design subsystem in the system.

In the same way coupling between subsystems can cause changes from a local impulse to propagate across a decision network, changes from an external impulse can have a similar impact. For example, a common example of an external impulse for an automotive manufacturer would be a change in global petroleum prices, affecting demand for certain types of vehicles. As noted earlier, formally modeling these impulses is not the focus of this paper.

3.3. Architectural impulses

Local and external impulses both describe changes to individual subsystems in a distributed design system. Architectural impulses address broader scenarios where the structure of the decision network itself changes. We define an architectural impulse as

• an impulse caused by events that originate either locally or externally and cause a restructuring of the process architecture.

In contrast to local or external impulses, which we define as affecting the specific parameters internal to the subsystem design problem, architectural impulses change the coupling links between subsystems and are not restricted in their origin. We distinguish architectural impulses from local and external impulses based on their influence on the actual structure of the process.

Examples of potential architectural impulses might be a breakdown or enhancement in communication between subsystems, the removal or addition of a subsystem from or to a decision network, or the development or degradation of ties between subsystems. We further subdivide architectural impulses into three classifications: dissolution, combination, and division, and discuss them in Sections 3.3.1, 3.3.2, and 3.3.3.

3.3.1. Dissolution impulse

Dissolution impulses eliminate one or more design subsystems from the aggregate design system. Strategies to handle the design variables associated with the dissolved subsystem include distributing them to the remaining design teams, assigning them a new subsystem, or freezing them at their previous value. Freezing design variables is an attractive option to respond to a dissolution impulse because reassigning them can destabilize the system due to design variable coupling effects. Investigating the difference between the strategies used to mitigate this impulse is a topic of future work. An event that may result in a dissolution impulse is the bankruptcy of the organization controlling the design of one of the subsystems. Figure 3 provides a graphical representation for a five-subsystem distributed design system undergoing a dissolution impulse.

Fig. 3. Dissolution impulse.

The representation in Figure 3 shows five individual subsystems, D1 to D5. The arrows connecting the subsystems represent their dependence on design variable information from another subsystem. For example, in Figure 3 the arrow from D4 to D5 indicates that a design variable from D4 appears in the objective function of D5. The dissolution impulse shown by Figure 3 illustrates the removal of D1 from the design process. This removal leaves two subsystems, D2 and D3, previously linked to D1 without a source to specify those design variables. D4 does not directly require input from D1, but it now has less influence on the decision process because it can no longer indirectly control the other subsystems through inputs passed through D1. To continue the design process, a design manager or the subsystems must agree on a strategy to deal with the loss of D1, including reassigning or freezing design variable values.

3.3.2. Division impulse

While dissolution impulses remove subsystems from a design system, a division impulse increases the number of subsystems. Division impulses occur when an event divides a design subsystem into two or more separate and distinct subsystems. Each subsystem has its own design variables, objective function, and constraints, which must encompass at least the original shared design variables and may create new shared variables. An event that may cause a division impulse is the sale of a division of an organization working on a project still in process by the parent company. We graphically represent division impulses in Figure 4.

Fig. 4. Division impulse.

We represent a division impulse in Figure 4 as D4 splitting into two subsystems. One challenge associated with division impulses is partitioning design responsibilities to the emerging subsystems. Even when dividing design responsibilities is easy, dividing the design variables themselves presents a challenge. Furthermore, new design variable coupling from the now independent subsystems may also emerge due to the division.

3.3.3. Combination impulse

In contrast to division impulses, combination impulses occur when two or more design teams merge into a single subsystem. Combination impulses reduce the number of subsystems in a design process like a dissolution impulse, but the affected subsystem retains a presence in the design system. The combined subsystem retains control of design variables associated with each subsystem and may have a new aggregate objective function. An example of an event causing a combination impulse is a buyout or a merger. Figure 5 shows a combination impulse for a five-subsystem design problem.

Fig. 5. Combination impulse.

We represent a combination impulse in Figure 5 by combining D3 with D5. This combination requires the merging of objective functions for D3 and D5. The resultant subsystem can influence the stability and transient response of the overall system. We examine all three types of architectural impulses in conjunction with local impulses in Section 4 using a five-subsystem case study.

4.1. Case study problem structure

Our experiment examines the same case study used in the initial study of large distributed design systems as well as previous individual studies of local impulses and architectural impulses. We use this case study because it is large enough to have a variety of potential architectures and the behavior of the system is largely well understood because previous research has used this case study to demonstrate more general insights into distributed design systems (Chanron et al., Reference Chanron, Singh and Lewis2005; Chanron & Lewis, Reference Chanron and Lewis2006). The five-subsystem example shown conceptually in Section 3.3 to illustrate the types of architectural impulses has couplings consistent with this case study. We explicitly give the objective function associated with each subsystem in Table 1 and provide the design variables controlled by each subsystem. The objective functions shown in Table 1 are all quadratic and to be minimized, and each subsystem controls at least two design variables. These design variables couple every subsystem to at least one other subsystem as shown in the functional dependence table in Table 2.

Table 1. Five subsystem case study

Table 2. Functional dependence table

The breadth of different process architectures for a five designer problem means a wide range of potential settling times exist for the system. Figure 6 demonstrates the wide range of potential settling times across the design system.

Fig. 6. Five-subsystems deterministic convergence time.

In Figure 6 the architectures are grouped into bins based on the number of iterations required to converge. The height of the bars indicates the number of process architectures with a specified range of convergence times shown on the x axis. The mean convergence time for the simulated architectures is 25 iterations. The two fastest converging architectures are shown in Figure 7a. The slowest converging architecture is shown in Figure 7b and took 39 iterations to reach an equilibrium.

Fig. 7. (a) Fastest converging architectures for 14 iterations and (b) slowest converging architecture for 39 iterations.

One interesting result of the architectures shown in Figure 7 is the relative strength in the link between the different subsystems, roughly measured by the number of connections to other subsystems in Table 2. Conventional wisdom has held that subsystems with strong links to one another should be arranged sequentially while subsystems with weak links are arranged in parallel (Rogers, Reference Rogers1996). For example, the other four subsystems are all strongly linked to SS2. However, in the fastest converging architectures, SS2 is not arranged sequentially as previous work has suggested is appropriate.

For the study of local impulses in (Devendorf et al., Reference Devendorf, Devendorf and Lewis2010), it was found that one of the fastest converging architectures, [1,2,4] → [3,5], was also the most robust to impulses. This was because the local impulse propagated most quickly through that architecture, and that in turn enabled each subsystem to adjust to its presence. This is important, because it means that the same propagation property that enabled the system to remain robust to impulses also enables it to converge quickly to its equilibrium.

4.2. Statistical testing of the parallel character hypothesis for dissolution and division impulses

Strategies for mitigating architectural impulses as investigated in Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012) yielded a preference for systems that featured the fewest number of serial connections when applying dissolution or division impulses, which in this work we term as the parallel character hypothesis. That is, architectures that featured increasingly parallel arrangements generally yielded the fastest convergence times after the application of dissolution or division impulses. Combination impulses did not uphold this hypothesis, because of the inherent instability that these impulses cause, which is discussed in further detail by Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012).

Before investigating whether the parallel character hypothesis also applies to our evaluation of aggregated design impulses, we evaluate its validity via statistical testing. For these tests, we generate architectures and measure their transient response using the same procedure as described in Ghosh et al. Reference Ghosh, Lewis and Devendorf2012), except that each architecture is now run for a total of 500 trials as opposed to 10. This sample size allows us to assume that the mean iteration count produced for each architecture is representative of that architecture's performance.

Using the mean convergence time data, we categorize each architecture into subpopulations according to the number of sequential connections it exhibits, bounded by zero for a fully parallel architecture and four for a fully sequential architecture. Because we draw these results from the final population of the genetic algorithm, they represent the best architectures from within their subpopulation. We then conduct two-tailed t tests between each subpopulation to test whether the number of sequential connections is a significant predictor of convergence time (i.e., whether the subpopulations are from the same distribution or are distinct). We summarize the paired t tests that yielded statistical significance in Table 3.

Table 3. Parallel character hypothesis testing for dissolution and division impulses

*p < 0.01. **p < 0.001.

Overall, we see that the majority of subpopulation pairs tested uphold the parallel character hypothesis, asserting that reducing the number of sequential connections does reduce postimpulse mean convergence time for architectures that undergo dissolution and division impulses. We observe that 4-connection architectures are not present among the significance tests. It is very difficult to generate many purely sequential, 4-connection architectures using a genetic algorithm consistent with the experimental procedure of Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012) and related previous studies (Devendorf et al., Reference Devendorf, Devendorf and Lewis2010), because these architectures perform poorly enough to be eliminated from consideration from the final population of architectures produced by the genetic algorithm. The lack of results for 4-connection architectures is specific to this case study, because four connections represent the extreme case of a purely sequential architecture for 5-subsystem systems. Generalizing these results to systems of n subsystems with n > 5 will result in similar extreme cases when the architecture exhibits n – 1 connections. Thus, the lack of statistical power for tests concerning 4-connection architectures due to their underrepresentation in the final population agrees with our assertion of their poor performance relative to architectures that feature fewer sequential connections.

The remaining subpopulation paired test not represented is 0 versus 1 for division experiments. In this case, paired t tests demonstrated the opposite relationship, where 1 sequential connection architectures yielded a statistically significantly lower time than a purely parallel 0-connection architecture. Given the way division impulses are modeled in Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012), where the child subsystems are inserted in parallel at the location of the original parent subsystem, and with child subsystems sharing identical objective functions and design variables as the parent, a 0-sequential architecture under division can be thought of as a purely parallel architecture subjected to no impulses at all. Thus, because the fastest performing architectures in the case study used in Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012) yielded one sequential connection (see Fig. 7), this result agrees with our current intuition of division impulse behavior. Conducting sensitivity analysis on 0-connection architectures and their child architectures to determine whether our intuitive understanding of this behavior is correct is a topic of future study.

Having validated the parallel character hypothesis for a majority of dissolution and division impulse cases, we now turn our attention to experiments studying the simultaneous application of local and architectural impulses.

4.3. Experimental design

In this section, we describe the simulation parameters, the implementation details of local and architectural impulses, and specifics regarding the experimental design used.

4.3.1. Simulation parameters

Each architecture is evaluated as part of a genetic algorithm that determines the fastest converging architecture. We use the results of the genetic algorithm to identify the properties of the fastest converging architectures and to differentiate between the best architectures with different numbers of stages. The parameters of this genetic algorithm are shown in Table 4. We cap the maximum number of iterations at 250, which is a sufficiently large number to allow for effective observation of the system behavior. After a system reaches 250 iterations, we assume it to be divergent or an extremely slow architecture.

Table 4. Genetic algorithm parameters

A critical aspect to evaluating the convergence time is the actual convergence criteria. We define convergence for these simulations to occur when all the design variables for a system have progressed to within 2% of their equilibrium value, as measured from the initial starting location. To maintain consistency across the experiments, we normalize the equilibrium point for each system to the origin and recalculate the convergence criteria when an architectural impulse occurs that modified the equilibrium. The stability of the studied architectures depends on the coupling between the subsystems and the independence of the design variables controlled by a single subsystem.

4.3.4. Experimental design

For each experiment described in the following section, 90 architectures are randomly generated in accordance with previous studies using this five-subsystem example to provide a wide range of architectures for an initial population. We then run the architectures through the genetic algorithm described in Section 4.1 for a total of 10 independent trials. During the simulations, the number of iterations is recorded, and the resulting mean and standard deviation across the 10 trials for each architecture are collected, representing our measures for system transient response in this work. Depending on the results from these experiments, we determine which experiments require further study with larger populations and statistical analysis.

In the next section, we investigate the amalgamated effects of architectural and local impulses on distributed design systems. The set of experiments investigated are summarized in Table 5. The experiment column displays the shorthand name of the experiment being run. For instance, division-(20,10) refers to an experiment with the simultaneous application of division impulses and local impulses with severity and probability parameters set to 20% and 10%, respectively, which were chosen to mirror previous studies.

Table 5. Experimental design

4.4. Experimental results

In this investigation of amalgamated local and architectural design impulses, we use results from our prior investigation of architectural impulses in Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012) as a control and for comparative purposes to inform our conclusions in this work. The three baseline experiments that are used as a standard of comparison in this work are random dissolution, random division, and random combination. These experiments were conducted identically to what is described in Section 4.2, except without the simultaneous application of local impulses.

4.4.1. Dissolution experiments

The results from the random dissolution experiment from Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012) along with the results from the dissolution-(10,10) and dissolution-(20,10) experiments are displayed graphically in Figure 8a–c, respectively. These figures show the relationship between mean convergence time and number of sequential connections for each of the 90 architectures investigated for each experiment. The number of sequential connections is a measure for the parallel character of a system architecture, displaying a count of how many sequential steps an architecture has. For instance, while a purely parallel architecture has zero sequential connections, a purely serial one has four.

Fig. 8. Sequential connections versus convergence time for dissolution experiments.

In Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012), we note that for systems that are subjected to dissolution impulses only, there is a strong preference for systems with a low number of sequential connections. This trend is evident in Figure 8a, where each of the 11 total generated architectures that have four parallel subsystems in sequence with the remaining subsystem outperforms each of the 22 total generated architectures with three sequential steps.

In Figures 8b and 8c, we see the results of the dissolution-local impulse experiments. Our first observation is that there appears to be a similar preference for parallel architectures in these experiments, albeit less resolute than in the random dissolution experiment. For instance, in dissolution-(10,10) and dissolution-(20,10), we find at least one three-sequential-step architecture among the top 10 performing architectures of both experiments: one in dissolution-(20,10) and three in dissolution-(10,10). Nonetheless, the exhibited parallel preference is evident when considering that the top performing architectures in each experiment show the least number of sequential connections among the randomly generated architectures for those experiments, where the best performing architectures for dissolution-(10,10) and dissolution-(20,10) are [2,3,4,5] → [1] and [1,2,5] → [3,4], respectively. Furthermore, the top 10% of architectures for dissolution-(20,10) all exhibit only one serial connection, the minimum encountered among the 90 randomly generated architectures.

An additional observation to make concerns the magnitude of iterations required for dissolution-(10,10) and dissolution-(20,10) architectures to converge, which are notably greater than the random dissolution experiment. This finding is consistent with results from Devendorf et al. (Reference Devendorf, Devendorf and Lewis2010), where the addition of a local impulse had been shown to delay convergence time for systems.

Thus, treating the random dissolution experiment as a baseline, we find that the introduction of local impulses has produced two notable effects: to unilaterally delay the convergence of architectures and to somewhat dampen the strength of the parallel architecture preference exhibited by systems subjected to architectural impulses. These two effects can be considered somewhat complementary, because the stochastic nature of mistakes induced by local impulses not only serves to postpone system convergence but also can be somewhat disproportionate, affecting some architectures more by chance than others. This may explain the dampening of the parallel preferences for dissolution-local impulse systems, an effect whose underlying mechanics we discuss as a topic of future work.

Finally, this discussion highlights an intriguing observation: that local and architectural impulses have an additive effect on one another. In other words, it would seem that adding local impulses to dissolution experiments has not produced a novel aggregated behavior; rather, the results above would seem to suggest that the effects of local impulses act independently of the effects of architectural impulses.

4.4.2. Division experiments

The results from the random division experiment along with the results from the division-(10,10) and division-(20,10) experiments are displayed graphically in Figure 9a–c, respectively. Figure 9a displays a similar trend to dissolution, where systems subjected to division impulses prefer increasingly parallel arrangements to mitigate the effects of the impulse.

Fig. 9. Sequential connections versus convergence time for division experiments.

The division-local impulse results of Figure 9b and 9c exhibits similar relationships to Figure 9a as Figure 8b and 8c show to Figure 8a, though with a caveat: while there appears to be a similar delayed convergence and dampened strength of parallel architecture preferences, the effects seem to be less pronounced than when local impulses were introduced to dissolution impulses. This trend reveals itself over Figure 9a–c, where bands of architectures on each gridline for the number of sequential connections occupy roughly the same ranges of mean convergence time, whereas for the dissolution experiments, there was a visible offset between random dissolution and dissolution-(10,10) and dissolution-(20,10) in terms of mean convergence times.

Tracking differences in performance for a given architecture for the dissolution and division experiments reveals this trend. For dissolution, while the top five architecture [1,5] → [2,3,4] converged in an average of 15.00 iterations for random dissolution, the same architecture required an average of 18.38 iterations for the dissolution-(20,10) experiment.

Conversely, the top five random division architecture [1] → [2,3,4,5] required an average of 15.54 iterations to convergence, while the division-(20,10) experiment required an average of 15.67 iterations, delaying convergence only marginally (by about 0.8% of the nonlocal impulse value) when compared to the dissolution experiments (which exhibit a 22% increase in convergence time). Thus, we conclude that systems subjected to division impulses are better at handling the application of local impulses than are similar systems subjected to dissolution impulses.

4.4.3. Combination experiments

The results from the random combination experiment along with the results from the combination-(10,10) and combination-(20,10) experiments are displayed graphically in Figure 10a–c, respectively. As noted in Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012), there is no visible correlation between the parallel character of a system architecture and its mean convergence time with respect to random combination impulses, and we see a similar case for the combination-(10,10) and combination-(20,10) experiments in Figure 10b and 10c, respectively. This is in contrast to the results from dissolution and division experiments, suggesting that parallel structures are not universally better for all types of architectural impulses (with or without the addition of local impulses). This may be a symptom of the inherent stability associated with subsystem combinations (a previously noted caveat to these experiments is that subsystems to be combined must be sufficiently decoupled).

Fig. 10. Sequential connections versus convergence time for combination experiments.

An additional observation regarding these experiments can be found in Table 6, which displays the worst performing architecture for each combination experiment considered in this work. One common feature among these systems is that they all feature the [1,3] subsystem parallel pair, set in series with the remaining subsystems. The inherent instability of this subarchitecture had been well documented in Ghosh et al. (Reference Ghosh, Lewis and Devendorf2012), and we find that the combination-(10,10) and combination-(20,10) experiments support this prior finding. Thus, we find a similar trend to the dissolution and division experiments where the application of local impulses does not necessarily remove the character of behavior of systems subjected to architectural impulses alone.

Table 6. Worst performing architecture by experiment considered

4.5. Statistical testing of the parallel character hypothesis for aggregated impulses

In this section, we describe the hypothesis testing used to validate the correlations observed in Section 4.4. In particular, we examine the parallel character hypothesis in the dissolution-(10,10), dissolution-(20,10), division-(10,10), and division-(20,10) experiments. We do not conduct such hypothesis testing on the combination experiments, because the results of Section 4.3.3 do not present correlations that frame this investigation for statistical significance.

For these tests, we generate architectures and measure their transient response using the same procedure as in Section 4.3.4. Similar to the testing of the parallel character hypothesis conducted earlier for dissolution and division impulses alone in Section 4.2, each architecture is now run for a total of 500 trials instead of 10, enabling us to assume the mean iteration count for each architecture to be representative of that architecture's performance.

As in Section 4.2, we categorize each architecture into subpopulations according to the number of sequential connections it exhibits. As before, these results are drawn from the final population of the genetic algorithm, representing the best architectures from within their subpopulation. We then conduct two-tailed t tests between each subpopulation to test whether the number of sequential connections is a significant predictor of convergence time. We summarize the significant results of the t tests in Table 7.

Table 7. Parallel character hypothesis testing for simultaneous local and architectural impulses

*p < 0.01. **p < 0.001.

From Table 7, we observe that a majority of sequential connection subpopulation pairings bear statistically significant results, thus supporting the parallel character hypothesis. Thus, we see an overall trend where an increase in the number of sequential connections for dissolution-local and division-local experiments (outlined in Table 7) is associated with statistically significant increases in mean convergence time. Similar to the analysis in Section 4.2, many of the pairings that were not statistically significant feature 4-connection, purely sequential architectures. As we note earlier, these are generally poor candidates for the final population generated for analysis by the genetic algorithm, inherently performing worse in terms of convergence time when compared to fewer sequential connection architectures. In addition, the 0-connection case for the division-(10,10) and division-(20,10) experiments remain somewhat limited in the insights they can offer based on how division impulses are currently modeled, as noted earlier in Section 4.2. Notably, however, the majority of cases support our earlier observations that the mitigation strategy for architectural impulses remains dominant, despite its direct contradiction with local impulse mitigation strategy.