1. INTRODUCTION
The idea generation phase is a crucial part of the design process in which concepts are developed either intuitively or through systematic processes. There are many approaches to idea development, but we focus in this paper on factors that influence the use of analogies. Designers frequently retrieve and use solutions from analogous designs to help them create innovative solutions to new problems (Casakin & Goldschmidt, Reference Casakin and Goldschmidt1999; Leclercq & Heylighen, Reference Leclercq, Heylighen and Gero2002; Christensen & Schunn, Reference Christensen and Schunn2007). Indeed, studies of the evolution of technologies frequently cite analogies as an important force in the development of product classes (Basalla, Reference Basalla1988). One recent example is a retractable mast with sail designed after studying bird and bat wings (BBC, 2000; Fig. 1). This sail is also useful for cargo ships to harness wind power and reduce fuel costs. The sails are easily raised and lowered and are very compact (Reed, Reference Reed2006).
Fig. 1 The design of the sails of the cargo ship are based on analogies to a bat's wing (left and middle panels, reprinted with permission of Richard Dryden; right panel, courtesy of iStockphoto.com and Gijs Bekenkamp, 2008). [A color version of this figure can be viewed online at www.journals.cambridge.org]
Although observational studies of designers at work demonstrate the use of analogy (e.g., Christensen & Schunn, Reference Christensen and Schunn2007), there are many open questions surrounding the factors that promote the retrieval and use of analogies. For the above example, what modalities and representations make this type of innovation more likely? How do different modalities and representations influence a designer's abilities? What will make designers more successful? What tools do designers need to support this process? The paper uses a fundamental experimental approach to explore the effects of representation within the modality of sketching and the effects of coupling the modalities of functional modeling and sketching. We begin by reviewing previous research in cognitive science on analogical reasoning. This review serves as the foundation for the research questions and experimental approach described in the following sections. Then we present an experiment that examined the use of analogies in mechanical engineering design, and we discuss the implications of this work for automated design.
2. MOTIVATION AND PREVIOUS WORK
In this section, we review related research on analogical reasoning and design. In this paper, we focus on design by analogy in the modality of sketches. Much work within design research has investigated the use of sketches (Ullman et al., Reference Ullman, Wood and Craig1990; Goldschmidt, 1991; Goel, Reference Goel1995; Suwa & Tversky, Reference Suwa and Tversky1997; Purcell & Gero, Reference Purcell and Gero1998; Stahovich et al., Reference Stahovich, Davis and Shrobe1998; Shah et al., Reference Shah, Vargas-Hernández, Summers and Kulkarni2001; Nagai & Noguchi, Reference Nagai and Noguchi2002; Yaner & Goel, Reference Yaner, Goel, Barker-Plummer, Cox and Swoboda2006a, Reference Yaner, Goel and Gero2006b, Reference Yaner and Goel2007; Yang & Cham, Reference Yang and Cham2007). There is other research looking at modalities such as the use of physical models in design, and we believe that the work we present here is relevant to these other modalities (e.g., Vidal et al., Reference Vidal, Mulet and Gómez-Senent2004; Christensen & Schunn, Reference Christensen and Schunn2007). Understanding the design process requires understanding both the internal mental representations of designers as well as the external representations (e.g., sketches, function, and flow basis diagrams) that are used during the design process.
2.1. Representation
A representation is a physical or mental construct that stands for some other physical or mental construct. Analyses of the concept of representation suggest that there are four necessary parts to a mental representation: the physical or mental construct serving as the representation, the domain being represented, rules (usually implicit) that map parts of the representation onto the item represented, and a set of processes that makes use of the information in the representation (Markman, Reference Markman1999). Understanding the design process requires understanding both the internal mental representations of designers as well as the external representations (e.g., sketches, function, and flow basis diagrams) that are used during the design process.
The study of mental representations makes clear that people represent relationships among items, and that these relationships play an important role in analogical reasoning. Theories of analogy often posit that mental representations have a structure akin to that of predicate–argument structures used in logic, artificial intelligence, and linguistics (Gentner, Reference Gentner1983; Holyoak & Thagard, Reference Holyoak and Thagard1989). Using this representational notation, a predicate is a statement that is asserted of a subject or subjects, and arguments are the subjects of which predicates are asserted. For example, Brown(x) is a predicate capable of representing the property that some object x is brown. The variable x serves as an argument to this predicate and delimits the scope of the predicate. Thus, the proposition Brown(boot) is a statement that has the gloss “The boot is brown.”
By convention, a predicate (like Brown[boot]) that takes one argument is called an attribute. Attributes are typically used to describe objects in a domain. Predicates that take two or more arguments are called relations. For example, Larger_than(x, y) takes two arguments and represents the relation that some object x is larger than some other object y. This distinction is important, because analogies typically involve similarities between two domains in the set of relations that describe them (see Falkenhainer et al., Reference Falkenhainer, Forbus and Gentner1989). We discuss analogical reasoning in more detail below.
2.2. Cognitive memory representation
Cognitive models of memory propose that there are many different modalities of representation that play an important role in cognitive processing. One distinction of interest is between perceptual (i.e., nonverbal) representations and verbal representations (Loftus & Kallman, Reference Loftus and Kallman1979; Barrlett et al., Reference Barrlett, Till and Leavy1980; Paivio, Reference Paivio1986). The distinction between perceptual and verbal representation is supported by findings such as the verbal overshadowing effect in which talking about perceptual information can interfere with the later retrieval of that information from memory (Schooler et al., Reference Schooler, Fiore, Brandimonte and Medin1997). One implication of these kinds of verbal overshadowing effects is that verbal idea generation techniques may suppress or interfere with perceptual information in memory that may be the source of important analogies. Thus, sketching techniques may be particularly useful for supporting the retrieval of perceptual information. Finally, although perceptual and verbal representations appear to be psychologically distinct, there is good reason to believe that there are relational structures of the sort described in Section 2.1 in both perceptual and verbal modalities (Barsalou, Reference Barsalou1999).
2.3. Cognitive process model for design by analogy
We know that analogies are important in the design process, because designers frequently report using analogies when generating novel solutions to design problems (Basalla, Reference Basalla1988; Dunbar, Reference Dunbar, Ward, Smith and Vaid1997; Christensen & Schunn, Reference Christensen and Schunn2007). Thus, it is important to describe what is known about analogical reasoning processes in more detail. The consensus view of analogical reasoning in cognitive science is that analogy involves the mapping of relational knowledge from one situation to another (Gentner, Reference Gentner1983; Falkenhainer et al., Reference Falkenhainer, Forbus and Gentner1989; Holyoak & Thagard, Reference Holyoak and Thagard1989; Chiu, Reference Chiu2003). The problem domain is typically called the target of the analogy. A domain of prior knowledge that provides a potential solution to the problem is called the base of the analogy. Research on analogy suggests that people first find a mapping between the relations in the base and the target. On the basis of this mapping, aspects of the target may be rerepresented to make them more similar to the base. Furthermore, inferences about the target (such as potential solutions) may be made based on the similarity of the target to the base. The potential for creative problem solving is clearest when the two domains being compared are very different on the surface, although the same process of comparison can also be used for domains that share significant surface similarity (Gentner & Markman, Reference Gentner and Markman1997).
Research has been carried out in the field of psychology to understand the cognitive processes people use to create and understand analogies (Falkenhainer et al., Reference Falkenhainer, Forbus and Gentner1989; Gentner & Markman, Reference Gentner and Markman1997; Hummel & Holyoak, Reference Hummel and Holyoak1997; Gentner, Holyoak & Kokinov, Reference Gentner, Holyoak and Kokinov2001; Blanchette & Dunbar, Reference Blanchette and Dunbar2001). Figure 2 shows the basic process steps involved in reasoning by analogy, the most cognitively challenging step, and the design methods that are available to support each step.
Fig. 2 The steps in human reasoning by analogy and the current methods available to support those processes. [A color version of this figure can be viewed online at www.journals.cambridge.org]
Analogy has traditionally been viewed as a comparison between two items in which their relational, or causal structure, but not the superficial attributes, match (Gentner, Reference Gentner1983; Gentner & Markman, Reference Gentner and Markman1997). For example, an airplane wing and a hydrofoil can be viewed as analogous because both generate lift using flow over their surfaces. The fact that airplane wings involve air flow and hydrofoils involve water flow does not affect the analogy (nor does other potential surface detail such as the colors that the items are painted).
In the psychology literature, there has been a great deal of interest in the roles of analogy and expertise in problem solving when working with undergraduate students who have no specialized domain knowledge. A classical finding is that analogies are helpful in solving insight problems, but they are difficult to retrieve from memory (Gick & Holyoak, Reference Gick and Holyoak1980). Conversely, naturalistic research with experts typically finds that analogies are often used (e.g., Dunbar, Reference Dunbar, Ward, Smith and Vaid1997; Casakin & Goldschmidt, Reference Casakin and Goldschmidt1999; Leclercq & Heylighen, Reference Leclercq, Heylighen and Gero2002). This dichotomy may reflect that experts can see the deeper, logical structure of situations, whereas those without domain expertise are mainly aware of only the superficial features (cf. Chi et al., Reference Chi, Feltovich and Glaser1981; Gentner & Landers, Reference Gentner and Landers1985; Novick, Reference Novick1988).
To clarify and more fundamentally understand these issues, laboratory research, which affords good experimental control, needs to be conducted with burgeoning domain experts. Such individuals are capable of recognizing the causal structure of products, but may also be distracted by superficial features. These characteristics make them an appropriate test bed for determining the role of base representation in analogical reminding. Moreover, it has been suggested that implicit processes could mediate analogical problem solving (Schunn & Dunbar, Reference Schunn and Dunbar1996). That is, problem solving may occur based on analogy even when the problem solver is not aware that the analogy is being used. Therefore, in studies of analogical reasoning, it is important to look separately at when a solution based on a prior analogy is found and when an individual because aware of the analogy between two domains.
2.4. Retrieving analogies
Thus far we have discussed cognitive processes that allow a problem domain to be augmented by analogy to some base domain. A central problem in developing innovative solutions to problems, however, is that domains that are analogous to the problem are difficult to retrieve, particularly before the designer recognizes that the base domain is relevant for solving a problem.
The core principle of human memory retrieval is encoding specificity (Tulving & Thompson, Reference Tulving and Thomson1973). In essence, this principle states that a memory will be retrieved to the extent that the context at retrieval is similar to the context at encoding. The context consists of the representation of information at the time of retrieval as well as other factors like emotional state and physical location. Much research in cognitive psychology suggests that people tend to retrieve information based on attribute similarities between domains (e.g., Holyoak & Koh, Reference Holyoak and Koh1987; Gentner et al., Reference Gentner, Rattermann and Forbus1993; Catrambone, Reference Catrambone2002). Good analogies are ones that have primarily relational similarities. Paradoxically, then, people find good analogies useful, but they have difficulty retrieving them when they need them. On the encoding specificity view, this difficulty in retrieving analogies occurs, because people are typically focused on the specific situation they are in at the time of encoding. That is, representations of specific situations have a lot of attribute information in them. Consequently, they tend to be reminded of those situations only in new contexts that also share those attributes (see Forbus et al., Reference Forbus, Gentner and Law1995, for a computational model of analogical retrieval).
What would this view of analogical retrieval suggest if we wanted to improve people's ability to retrieve known situations that could be used to solve a new problem? One clear prediction is that, for any given target domain, a relationally similar base domain is more likely to be retrieved if it has few attributes than if it has many, because those attributes can only interfere with relational retrieval. In addition, this view predicts that a base domain will generally be easier to retrieve when it is represented using general relational terms (e.g., fill or travel) than when it is represented using specific relational terms (e.g., inflate or walk). When a domain is represented using specific relational terms, it will only be similar to other domains that also use related relational terms. In contrast, a domain that is represented using general relational terms, will be similar to problems expressed with a wider variety of more specific relational terms. For example, a domain represented using the relation walk will only be similar to domains that use some kind of locomotion, but a domain represented using the more general relation move will also be similar to relations like drive or fly.
It is less clear how design by analogy should be affected by the specificity of the problem representation. On the one hand, a general representation of a problem will minimize the attributes in the description and will create a description focused on relations. On the other hand, a problem domain does not contain any relations that are part of the solution to the problem (otherwise it would not be a problem). Thus, it may actually be better to have a specific representation of the problem being solved, because this representation will contain much of the detail that will be necessary for constraining the solution to the problem. The study we present here will examine the influence of the level of specificity of the base and problems domains on the retrieval and use of analogies.
2.5. Formal design by analogy methods
A few formal methods have been developed to support design by analogy such as Synectics, French's work on inspiration from nature (French, Reference French1988, Reference French1996), biomimetic concept generation and analogous design through the usage of the function and flow basis. Synectics is a group idea generation method that uses four types of analogies to solve problems: personal (be the problem), direct (functional or natural), symbolic, and fantasy (Gordon, 1961). Synectics gives little guidance on finding successful analogies. Other methods also base analogies on the natural world. French (Reference French1988, Reference French1996), highlights the powerful examples nature provides for design. Biomimetic concept generation provides a systematic tool to index biological phenomena (Hacco & Shu, Reference Hacco and Shu2002; Tinsley et al., Reference Tinsley, Midha, Nagel, McAdams, Stone and Shu2007; Vakili et al., Reference Vakili, Chiu, Shu, McAdams and Stone2007). From the functional requirements of the problem, key words are derived. The key words are then referenced to an introductory college textbook and relevant entries can be further researched.
Analogous concepts can be identified by creating abstracted functional models of concepts and comparing the similarities between their functionalities. Analogous and nonobvious products can be explored using the functional and flow basis (McAdams & Wood, Reference McAdams and Wood2000). A case study of a pickup winder for an electric guitar developed using this approach is shown in Figure 3. A guitar pickup is an electromagnetic device with thousands of small-gauge wire windings used to electrically transmit the vibration from the strings. Obvious analogies for the pickup winder include a fishing reel and a bobbin winder on a sewing machine. In addition to the obvious analogies, the abstracted functional model for the pickup winder identifies the similarity to the vegetable peeler. The analogy to a vegetable peeler leads to an innovative design (prototype shown in Fig. 3). Developing a systematic approach to search for and evaluating the utility of functionally similar concepts is critical to the successful implementation of design by analogy as is enhancing natural human capability.
Fig. 3 An innovative analogy that was discovered based on a functional model and using the representation of the function and flow basis. [A color version of this figure can be viewed online at www.journals.cambridge.org]
Other design by analogy methods have been recently developed, including both electronic tools and sketching-based approaches. A representative example of such recent tools is the work by Chakrabarti et al. (Reference Chakrabarti, Sarkar, Leelavathamma and Nataraju2005a, Reference Chakrabarti, Sarkar, Leelavathamma and Nataraju2005b). In this case, an automated tool exists to provide inspiration to designers as part of ideation process. Chakrabarti has tested the automation tool and its analogy representations with student participants as part of university design courses.
2.6. Previous research on design by analogy
Human-based design methods require a deep understanding of the processes people use and the areas in which guidance or assistance could improve the process. This knowledge is gained largely through experimental work. Even though design by analogy is a well-recognized method for design, few human experiments have been done that focus on the role of analogy in design. Important work in this line has been done by Casakin and Goldschmidt (Reference Casakin and Goldschmidt1999), Ball et al. (Reference Ball, Ormerod and Morley2004), Kolodner (Reference Kolodner1997), and Kryssanov et al. (Reference Kryssanov, Tamaki and Kitamura2001). Casakin and Goldschmidt (Reference Casakin and Goldschmidt1999) found that visual analogies can improve design problem solving by both novice and expert architects. Visual analogy had a greater impact for novices compared to experts. Ball et al. (Reference Ball, Ormerod and Morley2004) investigated the spontaneous use of analogy with engineers. They found that experts use significantly more analogies than novices do. The type of analogies used by experts was significantly different from the type used by novices. Novices tended to use more case-driven analogies (analogies where a specific concrete example was used to develop a new solution) rather than schema-driven analogies (more general design solution derived from a number of examples). This difference likely reflects that novices have more difficulty retrieving relevant information when needed and have more difficulty mapping concepts from disparate domains because of a lack of experience (Kolodner, Reference Kolodner1997).
A structured design by analogy methodology would be useful for minimizing the effects of experience and for enhancing experts' abilities. The cognitive analogical process is based on the representation and processing of information, and therefore can be implemented systematically given appropriate conceptual representations and information processing tools (Goldschmidt & Weil, Reference Goldschmidt and Weil1998; Kryssanov et al., Reference Kryssanov, Tamaki and Kitamura2001).
Prior research in analogical reasoning found the encoded representation of a source analogy (the analogous product) can ease retrieval if it is entered into memory in such a way that the key relationships apply in both the source and target problem domains (Clement, Reference Clement1994; Clement et al., Reference Clement, Mawby and Giles1994). This work shows that the internal representations in memory play a key role in retrieval. The analogies and problems used in these experiments were not specific to any domain of expertise and used fantasy problems relying only on linguistic descriptions.
Little work has been carried out based on a strong psychological understanding of analogical reasoning combined with the design knowledge of analogies for high-quality designs. This paper takes a distinctive interdisciplinary route to combine these threads of research to develop a more complete understanding of the use of analogy in engineering design and to provide the basis for formal method development. Designers rely on both internal mental representations and numerous external representations ranging from sketches to specialized diagrams such as black box models. The use of various representations and modalities in the design process warrants further understanding. The following experiment further investigates visual and semantic representation effects on design by analogy, and lead to a deeper understanding of how to enhance the design by analogy process.
3. EXPERIMENTAL APPROACH AND RESEARCH QUESTIONS
Designers need predictable methods and supporting automated tools for developing innovative solutions to difficult design problems. Prior work has shown that general representations of analogous products in a designer's internal memory increase the chances the product will be used to solve a novel design problem (Linsey et al., Reference Linsey, Murphy, Wood, Markman and Kurtoglu2006). Open questions remain regarding the effects of the design problem representation and the modality of sketching.
To further explore the effects of representation on analogy use for real-world problems and to expand the knowledge base from which a design by analogy method will be created, we ran a study that controlled how participants learned about a series of products and therefore also controlled how the products were represented in their memories. This allowed the predictions from psychological models of analogical reasoning and analogical retrieval to be evaluated. These models, along with additional knowledge gained from experimentation, can be used as the basis for tools and methods development. The experiment uses a combination of visual and semantic information to represent the source design analogy.
In this context, we seek to answer the following research questions:
• Question 1: Designers frequently base their solutions to novel design problems on prior analogous solutions they have stored in memory. As designers learn about and store products in memory with either a general sentential representation that applies across multiple domains or in more domain-specific terms, how does the linguistic representation affect their ability to later use the analogous product to solve a novel design problem within the modality of sketching?
• Question 2: How does the representation of the problem statement affect the ability of a designer to retrieve and use a relevant analogous product to expose a solution to a new design problem within the modality of sketching?
• Question 3: Does the additional modality of functional models facilitate solving a novel design problem?
3.1. Overview of the experiment
This experiment controls the way in which a designer learns about an analogous product (represents it in memory) and also how a design problem is stated. This setup allows the effects of representation in memory and of the design problem to be observed. Throughout the experiment, participants used the modality of sketching and words to both reason and document their ideas. These participants were made up of senior-level mechanical engineering students. These students ranged in age from early 20s to early 30s, and experience level from minimal industrial experience, to internship and coop experiences, to multiple years of experience obtained before returning for a higher education degree.
The choice of participants is appropriate for this study for a number of reasons. A key characteristic of the experiment concerns the use of domain knowledge for multimodal reasoning with different types of representations. The choice of experimental subjects clearly meets this characteristic. In addition, the use of college student participants allowed us to gather a sample of engineers with a range of demographic backgrounds without being affected by the scheduling constraints involved in running engineers from industry. Finally, our chosen participant group provides the opportunity to explore the effect of ideation methods as part of a higher education curriculum.
The experiment consists of two tasks: memorize the analogous products and solve the design problems with a week in between for most participants. Normally, when faced with a design problem, a useful analogous product has not been seen immediately beforehand, but the analogous product is stored in a person's long-term memory. A week was chosen as a relevant time period for the experiment because any analogies retrieved will clearly be taken from long-term memory. This time frame has been used in previous experiments (Thompson et al., Reference Thompson, Gentner and Loewenstein2000). Results from the first task were matched to the second task. Participants were senior mechanical engineers with instruction in design methodology including idea generation. Multiple solutions were encouraged for all phases. Participants were told that the experiment evaluated various skills used in the design process. The effects of the design problem and the analogous product representation were evaluated. A 2 × 2 factorial experiment design was employed which resulted in four different experimental groups (Table 1). For both the analogous product and the problem description, two levels of participants were compared, a domain specific description group and a general description group. In each task, participants received linguistic representations using either very domain specific wording or in more general terms (Table 2).
Table 1. Overview of the factorial experiment design
Table 2. An example of the domain-specific and general device descriptions given to participants for task 1
Sentences are general (G) or domain specific (D).
Table 3. Domain-specific and general problem statements
3.2. Procedure
For the first task, memorize the analogous products, participants were given five short functional–textual descriptions of products along with a picture (Fig. 4) and were asked to spend 30 min memorizing the descriptions. The products were functionally described in a few short sentences either with a more general description that applied in both the source analogy and target design problem domains or with a domain-specific description. An example of the descriptions used for the flour duster device is shown in Table 3. The product descriptions and the design problems included meaningful pictures. The semantic descriptions of the devices were varied, but the pictures were identical for both conditions. The focus of these experiments was on the linguistic representations of the devices, but visual information was also present.
Fig. 4 Analogous products and solutions based on the analogies. [A color version of this figure can be viewed online at www.journals.cambridge.org]
Both groups were then given up to 15 min to answer a quiz, requiring them to write out the memorized descriptions. Finally, the groups spent up to 10 min to evaluate their results. Two of the products acted as source analogies for the design problems in the second task, solve the design problems, and three were distracter products that shared surface similarities with the design problems. An example of the descriptions used for the air mattress is shown in Table 2.
All time limitations throughout this experiment were based on a pilot experiment with graduate students in mechanical engineering in which they were given no time limits. Time limits were set to be longer than the amount of time required by most participants in the pilot experiment. For certain tasks and phases, it was clear participants were not spending enough time on the task, so the time limits were actually extended well beyond the time required for the participants in the pilot experiment.
In the second task, solve the design problems, participants were given two design problems to solve. Each design problem was staged in the following seven phases:
phase 1: open-ended design problems, few constraints
phase 2: highly constrained design problems
phase 3: identify analogies and try using analogies
phase 4: continue using analogies
phase 5: try to use a function structure to help you find a solution (Fig. 5)
phase 6: informed task 1 products are analogous
phase 7: target analogous product is given
Fig. 5 The functional model for design problem 2: flour sifter.
Phases 1 and 2 were completed for the two design problems followed by phases 3–6. Throughout all phases, participants were given the general idea generation guidelines to generate as many solutions as possible with a high quality and large variety and to write down everything even if it did not meet the constraints of the problem including technically infeasible and radical ideas. Participants were also instructed to use words and/or sketches to describe their ideas. They were asked not to discuss the experiments with their classmates until all the experiments were completed.
In phase 1 the problems were initially presented with few constraints. Participants were given 11 min to generate ideas for the open-ended design problems and then given an additional 11 min to create more solutions to the same problem with additional constraints. The additional constraints limited the design space, thus increasing the chance the participants would retrieve the desired source analogy. Next they had a 5-min break.
In phase 3 participants spent 10 min listing any analogies they had used and also using analogies to develop additional solutions. An open question from one of our prior experiments (Linsey et al., Reference Linsey, Murphy, Wood, Markman and Kurtoglu2006) was whether participants would be more likely to find the source analogy from task 1 if they were given more time to use analogies. Therefore, following the initial phase using analogies, participants were given an additional 10 min to continue to use analogies to create solutions.
Next, participants were shown a series of six function structures and asked to develop more solutions to the constrained design problem. This phase provided a foundation for evaluating the effectiveness of function structures for generating novel design solutions. Function structures are representations used in engineering design (Stone & Wood, Reference Stone and Wood2000; Otto & Wood, Reference Otto and Wood2001; Kurfman et al., Reference Kurfman, Stock, Stone, Rajan and Wood2003; Hirtz et al., Reference Hirtz, Stone, McAdams, Szykman and Wood2002). They are a particular form of functional representations, where a number of such representations have been studied as part of the design process (Chandrasekaran et al., Reference Chandrasekaran, Goel and Iwasaki1993; Qian & Gero, Reference Qian and Gero1996; Goel, Reference Goel1997; Umeda & Tomiyama, Reference Umeda and Tomiyama1997; Balazs & Brown, Reference Balazs, Brown, Cugini and Wozny1998, Reference Balazs and Brown2002; Kitamura et al., Reference Kitamura, Sano, Namba and Mizoguchi2002; Gero & Kannengiesser, Reference Gero and Kannengiesser2003; Chandrasekaran, Reference Chandrasekaran2005; Stone & Chakrabarti, Reference Stone and Chakrabarti2005). When function structures are created for novel design problems, process choices must be made. Process choices include using human energy to actuate the device as opposed to a battery and electric motor or a gasoline engine. The process choices for the function structures were made to be consistent with the solution based on the analogous product and were expected to improve participants' ability to generate a solution. This phase of the experiment addresses whether an appropriate functional representation will assist participants in solving a difficult design problem. This experiment does not address how these particular functional representations with appropriate process choices can be developed by participants.
In phase 6 the participants were told that products from the first task were analogous, and were asked to mark their solutions that used the analogy and to generate additional solutions using analogies. Finally, participants were given the target analogy for each problem, and were asked to place a check where they had used it and to generate more ideas if they had not used the described analogy. These final two phases serve as a control to verify that the analogies being used are sensible, are useful for these particular design problems, and facilitate data evaluation. At each phase the participants used a different color of pen, which made it easier for the experimenters to identify the phases of the study at which information was added. A short survey at the conclusion of the experiment evaluated English language skills, work experience, if the participant had heard about the experiment ahead of time, functional modeling experience, if they felt they had enough time, and prior exposure to the design problem solutions. During one of session of task 2, a fire alarm occurred during phase 2. This caused a break in the middle of the experiment. The data were reviewed, and little impact was observed. These four participants are included in the results. The entire experiment required about 2 h.
3.3. Metrics for evaluation
Each analogy produces a set of solutions, not a single solution. Participants also created a large number of solutions that were not based on the analogies provided. We were primarily interested in the phase of the study at which participants produced a solution to the constrained design problem based on the targeted analogy and also the phase at which they identified the analogy that they used. As we will see, people often show evidence of being influenced by an analogous product without explicitly recognizing where the idea came from. Two evaluators judged the data independently, recording when the analogous solution was found. Initial agreement was approximately 80% across the experiments, and disagreements were readily resolved through discussion. The most common reason for the initial differences was the participant referenced solutions that appeared on different pages of the results.
4. RESULTS
Figures 6a and 7a show the percentage of participants at each phase who were able to generate the solution to the design problems based on the analogous product. Figures 6b and 7b show when participants both generated the solution and then also explicitly the analogous product from task 1. Both sets of graphs are based on participants' indication of the solution being based on the desired analogous product. Results based on evaluators' judgements of the correct features being mapped from the analogous product to the solution show a very similar pattern of results. Examples of participants' solutions based on the analogous product are shown in Figure 8. Figure 8 also contains models of the participants' ideas built by the authors for illustration and clarification. The analogous product representation and the problem representation had a clear influence on the designers' ability to use the analogy to generate a solution to the design problems. The trends are similar across the two design problems. Participants who had previously seen the solution to the design problems based on the analogous product were removed from the data set. This included 21 participants for design problem 1 and 3 participants for design problem 2. Participants who only completed one task of the experiment were also not included in the results. Participants who memorized the analogous product in a general form had the highest rate of success. This result is shown by the top (general/domain) line in the figures, where the success rate increased by up to 40%.
Fig. 6 (a) The percentage of participants with a solution based on the target analogous product at each phase for design problem 1, and (b) the percentage of participants who had a solution based on the target analogous product and also identified the analogy at each phase for design problem 1. [A color version of this figure can be viewed online at www.journals.cambridge.org]
Fig. 7 (a) The percentage of participants with a solution based on the target analogous product at each phase for design problem 2, and (b) the percentage of participants who had a solution based on the target analogous product and also identified the analogy at each phase for design problem 2. [A color version of this figure can be viewed online at www.journals.cambridge.org]
Fig. 8 Example solutions found by the participant and models built by the authors for illustration of the participants’ ideas. [A color version of this figure can be viewed online at www.journals.cambridge.org]
A two-predictor logistic model (Kutner et al., Reference Kutner, Nachtsheim, Neter and Li2005) was fit to the data for problem 1 at phase 4 to evaluate the statistical significance of the effects. A multivariate approach could not be used because too many of the participants had scores for only one of the design problems because a fairly large number had previous experience with the solution for design problem 1. The logistic model for problem 1 at stage 4 shows no significant interaction between the two predictors, and therefore, the interaction was removed from the model (p > 0.4). The remaining predictors show the design problem representation to be a statistically significant predictor (β = −1.6, p < 0.06) and the analogous product representation to be nonsignificant (β = 1.0, p > 0.2). The sample size is fairly small, because of participants having seen the targeted solution, and therefore, the statistical power to detect difference is low. As the graph clearly shows, the general/domain condition is different from the other three conditions. Using a binomial probably distribution with pairwise comparisons between the conditions, the general/domain condition is statistically significantly different from the other three conditions (all p < 0.01; Devore, Reference Devore1999). Statistical analysis based on evaluators' judgment of an appropriate mapping between the analogous product and the solution instead of the participant evaluation are consistent but with slightly higher probabilities (p < 0.015, <0.01, <0.015). The representation of the design problem has a large effect on the analogies designers retrieve to assist in developing a solution. The representation of the design problem and the representation in memory significantly impact the designers' abilities.
A two-predictor logistic model (Kutner et al., Reference Kutner, Nachtsheim, Neter and Li2005) was also fit to the data for problem 2 at phase 4 to evaluate the statistical significance of the effects. None of the predictors were statistically significant. Clearly, from the plots, the general/domain condition is different from the other three conditions. Using a binomial probably distribution with pairwise comparisons between the conditions, the general/domain condition is statistically significantly different from the domain/general condition (p < 0.01; Devore, Reference Devore1999). Statistical analysis based on evaluators' judgment of an appropriate mapping between the analogous product and the solution instead of the participant evaluation show the same results.
Figures 6b and 7b show when participants found a solution based on the analogy and also explicitly referenced which product from task 1 was analogous. Participants could have labeled the analogy as early as phase 2 when they were told to try using design by analogy to try to solve the design problem, but none of the participants explicitly identified the analogous product until phase 5, when they were given a functional model. Designers frequently use previous solutions without realizing it. This effect will be discussed in detail in Section 6.
4.1. Effect of the functional models
Figure 9 shows the percentage increase with the addition of the functional models in the number of participants who found the targeted solution to the problem. Figure 9 shows the percentage increase from phase 4 to phase 5, the addition of the functional models. Across the experimental conditions the effect is similar, with the exception of the general analogous product representation with a general problem statement for design problem 1.
Fig. 9 The functional models assisted the designers who had not been able to find the solution using the problem statement and trying to find analogies. [A color version of this figure can be viewed online at www.journals.cambridge.org]
5. EVALUATION OF POSSIBLE LIMITATIONS TO THE EXPERIMENT SURVEY RESULTS: DID PARTICIPANTS HAVE ENOUGH TIME?
To evaluate whether the participants felt they had enough time to generate ideas, two Likert scale questions were asked. The questions asked participants to agree or disagree with the statements, “I ran out of time before I ran out of ideas,” and “I ran out of ideas before I ran out of time.” Over 75% of the participants felt they had plenty of time, and they ran out of ideas before they ran out of time (Fig. 10).
Fig. 10 Almost all participants felt that they had plenty of time and that they ran out of ideas. [A color version of this figure can be viewed online at www.journals.cambridge.org]
The length of each of the phases for this experiment is based on the results of a pilot experiment. However, we are interested in whether participants might have generated more analogous solutions if they had been given more time. To address this issue, we gave participants a survey after the study asking them if they had run out of time or ideas first. Overall, 76% of participants stated that they ran out of ideas first, but only 14% felt that they ran out of time before they were able to state all of their ideas (Fig. 10). It is possible that even though participants felt they had enough time that they would actually have a greater likelihood of generating the analogous solutions if they spent more time engaged on the problem. To assess this possibility, the total time for participants to search for solutions through analogies was doubled compared to one of our prior experiments (Linsey et al., Reference Linsey, Murphy, Wood, Markman and Kurtoglu2006) and corresponds to phases 4 and 5. During this second time period, only one additional participant found the solution for either of the two design problems. Simply spending more time attempting to use analogies has very little effect, at least within our experimental setup, process, and conditions. The time periods were long enough for these basic yet novel problems. Although the increased time period did not facilitate retrieval of the analogous product from the first task, participants did continue to find additional analogies and solutions. Methods that help designers to spend more time searching for analogies by preventing designers from feeling they have run out of ideas will also enhance the process.
6. ADDRESSING THE RESEARCH QUESTIONS
The data illuminates the effects of problem representation and representation of analogous products on design by analogy within the modality of sketching. The following discussion provides further insights based on the results.
6.1. Research question 1
General linguistic representations, which apply both in the analogous product and design problem domain, increase the success rate more than domain-specific representations. General linguistic representations are more likely to be retrieved from memory. If a designer retrieves analogous products from memory with more general representations, then they are more likely to later use these analogies to solve novel design problems (Figs. 6a and 7a). This result has very important implications for the way we should teach designers to think about and remember design solutions they encounter. If they seek representations that apply across more domains and in more general forms, they will be much more likely to be able to use the design in the future. For example, framing an air mattress as “a device that uses a substance from the environment it is used in,” rather than “a device that is filled with air” makes it much more likely to be used in future design problems that seek innovative solutions.
6.2. Research question 2
The representation of design problems clearly influences a designer's ability to generate analogous solutions (Figs. 6a and 7a). The representation that will give the designer the highest probability of exposing or generating an analogous solution depends on how the analogous solution is stored in memory. This experiment evaluated cross-domain analogies; the products and the design problems were not in the same domain. Retrieving solutions to a design problem within a domain is much easier than cross-domain analogies but results in less novel solutions presumably because both the product and design problem are represented in the same domain specific form. For the case of cross-domain analogies, if the analogous product is stored in a general form, then a domain specific representation is the most efficient means to retrieve it. For products that are committed to memory in more domain-specific terms, it is unclear what representation is best. Generally, it is not known in advance what representation is most likely to retrieve the desired information. This means that the best approach for seeking analogous solutions is to use multiple representations that vary across the range of domain specific to domain general.
6.3. Research question 3
There is a clear increase in the number of participants who found a solution based on the analogy during phase 5, when participants used the function structures to assist in generating solutions. This result is exciting and a validation of anecdotal claims about an important role of functional modeling in design. Function structures are another potential representation that will enhance the design process and should be included in the search for analogous solutions. It is important for us to point out, however, that participants were given function structures with process choices that were consistent with the analogous solutions we hoped that they would find. These function structure also included linguistic functional descriptions that were different from the given problem statements. This experiment does not address the way participants would go about developing these particular function structures on their own. Instead, it suggests that if designers create an appropriate function structure, it will increase the likelihood that they will generate the analogous solution. Further research must explore the kinds of function structures that designers generate spontaneously and the influence of these function structures on the analogies retrieved. However, clear implications from this work is that functional representations are important, and in turn, verb constructs (active functions) from the English language should be exploited to assist in the retrieval or search for analogies.
7. DISCUSSION OF ADDITIONAL RESULTS
This experiment addresses the research questions and provides additional interesting results that are further discussed in this section.
7.1. Analogy identification and implications for naturalistic analogy research and evaluation of automated tools that provide analogous solutions
Designers frequently use analogies to solve design problems without realizing the source of the idea. The participants used analogies to solve the design problems, but did not mention that they were using analogies and/or did not realize that their solutions were analogous to previously experienced products until a later phase (Figs. 6 and 7). Instructing subjects to use analogies and list the analogies they had used caused little effect. Our findings replicate the work of Schunn and Dunbar (Reference Schunn and Dunbar1996), except for an independent data set and in the engineering domain. Schunn and Dunbar found that participants often used analogies to solve difficult insight problems, but the subjects did not realize they were doing this. One implication of this result is that analogies play an important role in problem solving, but they do so, at least in part, outside of awareness. Another implication is that, in naturalistic observation studies or when evaluating an automatic design tool that facilitates analogies, simply recording how often people say they are basing their solutions on analogies is likely to underestimate their true frequency. For example, imagine an investigator who seeks to determine how important analogies are in generating new designs. This researcher decides to observe expert designers at their workplace generating novel designs and counts the number of times the experts say “this is just like (some other product).” Intuitively, this procedure seems reasonable, but our data suggest that it will underestimate the role of analogies. These results also indicate that designers frequently use analogy without recognizing it. This implies that design by analogy has an even greater impact on the design process than what is currently indicated by the anecdotal evidence.
7.2. Implications for automated or semiautomated design tools
Automatic tools have great potential to support and enhance conceptual design and design by analogy. Designers need more tools that assist in searching and retrieving analogous design solutions, especially far-field solutions. Some tools have been and are currently being developed to assist designers in finding analogies. Chakrabarti et al. (Reference Chakrabarti, Sarkar, Leelavathamma and Nataraju2005a, Reference Chakrabarti, Sarkar, Leelavathamma and Nataraju2005b) have created a tool that searches a biological database and retrieves possible solutions. Hacco and Shu (Reference Hacco and Shu2002) created a tool that cross-references a functional description in engineering terms to the related biological phenomena thereby retrieving possible solutions. Computational tools need to be able to search other representations (shape, form, dynamic motion, etc.) other than linguistic (Yaner & Goel, Reference Yaner, Goel, Barker-Plummer, Cox and Swoboda2006a, Reference Yaner, Goel and Gero2006b, Reference Yaner and Goel2007). Computational tools can also support engineering design by creating multiple function structures with different processes choices. It would be useful for automatic tools to a transition from one representation (functional model to problem statement) and to present information in multiple representations.
8. CONCLUSIONS
Design by analogy is a powerful tool in a designer's toolbox, but few designers have the methods to harness its full capacity. Simply recognizing its potential and attempting to search mentally for analogies is not enough. Designers need methods and tools to support this process. They need approaches for when they feel they have run out of ideas and methods to represent the problem in a multitude of representations. Automated tools need to be developed to support and enhance this process. The right representations have the potential to increase a designers' probability of success by up to 40%. These methods need to be built on a solid understanding of human capacity combined with scientific design knowledge. The linguistic representation profoundly impacts a designer's ability to find an appropriate analogy in memory as they reason within the modality of sketching. This experiment demonstrates, at least foundationally, the impact the right representation within a modality has on the design by analogy process.
The coupling of modalities has significant potential to enhance the design by analogy process and support innovation. This study shows that the addition of a function structure, or more generally functions stated as active verbs, to the sketch-based concept design process improved a designers' ability to find an innovative solution to a novel design problem. Additional representations and modalities are likely to also augment the process and warrant further investigation.
A deeper understanding of the mechanism behind analogical reasoning and their implications within design will guide the development of drastically improved design by analogy methods and tools for design innovation. Methods and tools to create multiple representations of a design problem will increase the probably a designer will find an analogy for an innovative solution. Automation tools can assist the designer in finding analogous solutions and automatically creating multiple representations. Representation clearly matters, and seeking improved representations has great potential for significantly enhancing the innovation process.
8.1. Future work
Future work must focus on developing new design approaches and methods to increase the quantity and quality of innovative solutions based on the knowledge gained from the experiments presented in this paper and other relevant literature. Greater exploration of the use of functional models and other types of representation for assisting in the design process will also be investigated. Additional studies must also explore other influences on the design by analogy process including expertise, physical models, visual information, and a wider variety of design problems. New methodologies will be validated through controlled experiments and with professional, practicing designers.
ACKNOWLEDGMENTS
The authors acknowledge the support provided from the Cullen Endowed Professorship in Engineering, The University of Texas at Austin, and the National Science Foundation under Grant CMMI-0555851. This research was also supported by a Fellowship in the IC2 Institute given to Dr. Arthur Markman. The authors also thank Emily Clauss for her assistance in data evaluation and analysis. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the sponsors.
Julie S. Linsey is an Assistant Professor in the Mechanical Engineering Department at Texas A&M University. She earned a PhD and MS in mechanical engineering from The University of Texas at Austin and a BS in mechanical engineering from the University of Michigan. Her research focus is on systematic methods and tools for innovative design with a particular focus on concept generation and design by analogy. She has authored over 15 technical publications, including 2 book chapters, and holds 2 patents.
Kristin Wood is the Cullen Trust Endowed Professor in Engineering and Distinguished University Teaching Professor at The University of Texas at Austin in the Department of Mechanical Engineering. Dr. Wood obtained a BS in engineering science from Colorado State University (1985) and MS (1986) and PhD (1989) in mechanical engineering from the California Institute of Technology. He has published over 200 scholarly works, including a textbook on product design. Dr. Wood's current research interests focus on product design, innovation, development, and evolution. The current and near-term objectives of this research are to develop design strategies, representations, and languages that will result in more comprehensive design tools, innovative manufacturing techniques, and design teaching aids at the college, precollege, and industrial levels.
Art Markman is Annabel Irion Worsham Centennial Professor of Psychology and Marketing at The University of Texas at Austin. His research examines analogical reasoning, categorization, motivation, and the influence of these processes on innovation and creativity. He has published over 100 scholarly works including 7 books. He is a past executive officer of the Cognitive Science Society and is currently the Executive Editor of Cognitive Science.
