Introduction
The ability to understand and feel the needs and circumstances of others, also known as empathy, has been found to help designers develop a deeper understanding of the design problems they solve (Walther et al., Reference Walther, Miller and Kellam2012). Empathy could be particularly important in the early conceptual stages [i.e., concept generation and selection (Toh and Miller, Reference Toh and Miller2016a)] of the design process (McGinley and Dong, Reference McGinley and Dong2011) as it involves a designer's attempt to “relate to [the user] and understand the situations and why certain experiences are meaningful to these [users]” (Battarbee, Reference Battarbee2004, p. 67). As such, the design community has been invested in devising and assessing empathic design activities, such as simulating empathy evoking scenarios (Raviselvam et al., Reference Raviselvam, Hölttä-Otto and Wood2016, Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017), in the design process (Lin and Seepersad, Reference Lin and Seepersad2007; Strobel et al., Reference Strobel, Hess, Pan and Wachter Morris2013; Raviselvam et al., Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018a, Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b; Tang, Reference Tang2018). While empathy has been established as an essential component of design (McGinley and Dong, Reference McGinley and Dong2011; Walther et al., Reference Walther, Miller and Kellam2012; Raviselvam et al., Reference Raviselvam, Hölttä-Otto and Wood2016, Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017), the role of empathy on impacting creative design outcomes is still unclear, especially during concept generation and selection. Formalizing the role of empathy in these earlier conceptual stages can save costs and effort (Mattson and Messac, Reference Mattson and Messac2005), as the success of a product can be linked to the early conceptual stages of the idea's emergence (Goldenberg et al., Reference Goldenberg, Lehmann and Mazursky2001), and being empathic in those stages can be gateway to creative solutions to the design problem (McGinley and Dong, Reference McGinley and Dong2011).
Empathy can be particularly important in the context of artificial intelligence (AI) since the main goal of using AI in design is to create better AI assistants that could help designers along the different stages of the design process (Gero, Reference Gero2007). However, the “fuzzy front end” (Calabretta and Gemser, Reference Calabretta, Gemser, Luchs, Swan and Griffin2015) of the design process is challenging to translate into AI terms (Achiche et al., Reference Achiche, Appio, McAloone and Di Minin2013). In this fuzzy-front end, designers are involved in numerous decisions that require cognitive effort (Toh and Miller, Reference Toh and Miller2019). Understanding if and how empathy is important in that fuzzy-front end, particularly during concept generation and selection, is critical in order to better build AI assistants.
At its current state, the literature identifying the role of empathy in design is in conflict. A group of researchers, see Hess et al. (Reference Hess, Fila, Purzer and Strobel2015), Hess and Fila (Reference Hess and Fila2016), Genco et al. (Reference Genco, Johnson, Holtta-Otto and Seepersad2011), Johnson et al. (Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014), and Raviselvam et al. (Reference Raviselvam, Hölttä-Otto and Wood2016, Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017), are advocating for the role of empathy in design and are invested in devising empathy invoking interventions, particularly at the concept generation stage. In contrast, other researchers (Mattelmäki et al., Reference Mattelmäki, Vaajakallio and Koskinen2014) warn designers from engaging in empathic design activities, as these empathy invoking activities might end up in the “empathy trap”; their attempt to be empathic might trigger popular directed reflections from the users instead of providing radical innovations to the existing problems (Mattelmäki et al., Reference Mattelmäki, Vaajakallio and Koskinen2014).
While the prior work investigating the role of empathy in concept generation is in conflict, the role of empathy during concept selection has been scarcely researched. This is problematic since the concept selection stage is when designers narrow down the ideas generated during concept generation (Toh and Miller, Reference Toh and Miller2016a) and has been identified as one of the most critical stages that determine successful design (Pugh, Reference Pugh1996; Rietzschel et al., Reference Rietzschel, Nijstad and Stroebe2010). Studying designers’ creativity during concept generation solely is not representative of the designers’ creativity since the “availability of creative ideas is a necessary but insufficient condition for innovation” (Rietzschel et al., Reference Rietzschel, Nijstad and Stroebe2006, p. 48).
Taken as a whole, prior work investigating the relationship between empathy and creative design outcomes during concept generation provides conflicting interpretations. Additionally, the role of empathy in concept selection has been scarcely researched. Thus, formalizing the role of an individual's trait empathy in driving design outcomes in the concept generation and selection stages of the design process could bring great clarity to the existing research. Without this knowledge, the design community cannot be sure if, when, or how empathy is important in the design process. As such, the main goal of this paper is to identify the role of trait empathy in creative concept generation and selection in an engineering design student project. The results from this research can form the basis of computational models in design.
Related work
In order to establish the framework for the current investigation, this section highlights prior work on (1) the role of empathy in engineering design and (2) measuring trait empathy, which serve as the basis for the current study.
The role of empathy in the design process
Empathy has been defined commonly in the psychology literature as, “a social and emotional skill that helps us feel and understand the emotions, circumstances, intentions, thoughts, and needs of others such that we can offer sensitive, perceptive, and appropriate communication and support” (McLaren, Reference McLaren2013). Empathy has been identified by psychologists as an emotional intelligence skill (Riemer, Reference Riemer2003; McDonald and Messinger, Reference McDonald and Messinger2012; Tekerek and Tekerek, Reference Tekerek and Tekerek2017) that allows individuals to distinguish and deal with others successfully (Badea and Pană, Reference Badea and Pană2010). Notably, the psychology literature discretizes empathy into the following two components: a cognitive component and an affective component (Duan and Hill, Reference Duan and Hill1996; see Figure 1). The cognitive component indicates that one's empathy is dependent on the situation, while the affective component characterizes one's empathy as an emotional response (Duan and Hill, Reference Duan and Hill1996), see Figure 1 (Hess and Fila, Reference Hess and Fila2016). Hoffman (Reference Hoffman1977), Shantz (Reference Shantz1975), and Strayer (Reference Strayer1987) view empathy to involve both cognitive and affective components. According to Batson (Reference Batson2009), there are eight components to empathy, including the following: knowing another person's internal state; imagining how others think and feel; intuiting or projecting oneself into another's feeling distress at witnessing another person's suffering; and feeling for another person who is suffering (Batson, Reference Batson2009).
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_fig1.png?pub-status=live)
Fig. 1. A summary of the cognitive and affective processes involved in the four interpersonal reactivity subscales from Davis (Reference Davis1980).
Notably, some scholars have considered empathy as a personality trait or general ability (Kerr, Reference Kerr1947; Danish and Kagan, Reference Danish and Kagan1971; Buie, Reference Buie1981; Hoffman, Reference Hoffman1982; Davis, Reference Davis1983; Book, Reference Book1988). For example, Davis (Reference Davis1983) believed that empathy is a dispositional trait, or a stable ability. On the same line of research, there has been a discussion in the literature that the basis of empathic thinking is heritable (Melchers et al., Reference Melchers, Montag, Reuter, Spinath and Hahn2016) with behavioral and imaging genetic studies providing evidence for a genetic basis for empathy (Anckarsäter and Cloninger, Reference Anckarsäter and Cloninger2007; Knafo et al., Reference Knafo, Zahn-Waxler, Van Hulle, Robinson and Rhee2008, Reference Knafo, Zahn-Waxler, Davidov, Van Hulle, Robinson and Rhee2009; Chakrabarti and Baron-Cohen, Reference Chakrabarti, Baron-Cohen, Baron-Cohen, Tager-Flusberg and Lombardo2013; Melchers et al., Reference Melchers, Montag, Reuter, Spinath and Hahn2016). However, researchers argue that the environmental context could impact an individual's empathy (Knafo et al., Reference Knafo, Zahn-Waxler, Van Hulle, Robinson and Rhee2008, Reference Knafo, Zahn-Waxler, Davidov, Van Hulle, Robinson and Rhee2009; Abramson et al., Reference Abramson, Uzefovsky, Toccaceli and Knafo-Noam2020), highlighting the role of particular parental contexts in promoting prosocial behavior (Fortuna and Knafo, Reference Fortuna, Knafo, Padilla-Walker and Carlo2014). Furthermore, other researchers have also reported that mental health was found to be related to an individual's empathic behavior (Knafo et al., Reference Knafo, Zahn-Waxler, Davidov, Van Hulle, Robinson and Rhee2009; Apter-Levy et al., Reference Apter-Levy, Feldman, Vakart, Ebstein and Feldman2013; Mitchell et al., Reference Mitchell, Sheppard and Cassidy2021).
In the context of engineering design, empathy has been found to help designers better understand the needs of users that are different from themselves (Gray et al., Reference Gray, Yilmaz, Daly, Seifert and Gonzalez2015; Reference Schmitt and MorkosSchmitt & Morkos). Prior research in engineering design has shown that developing empathy can help develop a deeper understanding of the design problem (Walther et al., Reference Walther, Miller and Kellam2012) and the stakeholders (Reference Schmitt and MorkosSchmitt & Morkos) and encouraged an employment of a more targeted user research (Gray et al., Reference Gray, Yilmaz, Daly, Seifert and Gonzalez2015). In a qualitative study, Fila and Hess (Reference Fila and Hess2016) found empathy to be positively related to engineering students’ problem contextualization and individual design inspiration.
In terms of design effectiveness, Genco et al. (Reference Genco, Johnson, Holtta-Otto and Seepersad2011) and Johnson et al. (Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014) found that empathetic design experiences were effective in driving creative outcomes (originality and quality) in the conceptual design stages. On the same line of research, simulating extraordinary user scenarios was effective in enhancing students’ empathic self-efficacy as well as the novelty, quantity, and variety of ideas generated by students (Raviselvam et al., Reference Raviselvam, Hölttä-Otto and Wood2016). While prior work found a relationship between empathy and creativity, researchers have found that the creativity of solutions generated by a designer can hinge on the nature of the design task (Starkey et al., Reference Starkey, Toh and Miller2016), and the designer's personal connection with the end-user (Raviselvam et al., Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017). Similarly, Hess and Fila (Reference Hess and Fila2016) explored designers’ selection of empathic techniques in two design tasks: a service-learning course and a decontextualized design problem; they found that students in the service-learning course made use of a higher variety of other-oriented empathic techniques which suggested the significance of the context of the design problem in impacting design outcomes, which this work controlled for.
In contrast to the previous research that highlighted the effectiveness of empathic design techniques in the concept generation stage, engagement in empathic design experiences have also received criticism in the literature. For example, Chung and Joo (Reference Chung and Joo2017) found that engaging designers with an empathic instruction task (watching a video on the end-user) decreased their concept evaluation scores, suggesting a “dark” side to empathy. Breithaupt (Reference Breithaupt2019) discusses some of the dark sides of empathy, empathic vampirism, where individuals might over-identify with others, without necessarily having the best interests of those others in mind (Breithaupt, Reference Breithaupt2018). In the context of design, that line of research suggests that the designer would end up designing for themselves if they over empathize (Breithaupt, Reference Breithaupt2018).
While this prior work provides conflicting interpretations on the role of empathy in concept generation, little research has explored the role of empathy in concept selection. During this stage, designers narrow down the ideas generated during concept generation (Toh and Miller, Reference Toh and Miller2016a). Studying designers’ creativity during concept generation solely is not representative of the designers’ creativity since generating creative ideas does not necessarily guarantee the final design's creativity (Rietzschel et al., Reference Rietzschel, Nijstad and Stroebe2010). One way of assessing designers’ creativity in the concept selection stage is through their propensity for selecting creative ideas (Toh and Miller, Reference Toh and Miller2015, Reference Toh and Miller2016b; Zheng et al., Reference Zheng, Ritter and Miller2018). While concept selection has been found to be an important component of creativity of the design process (Rietzschel et al., Reference Rietzschel, Nijstad and Stroebe2006) that requires a different cognitive skillset that concept generation (Toh and Miller, Reference Toh and Miller2019), the relationship between empathic tendencies and creative concept selection has not been established in the literature.
This existing research provides conflicting interpretations on the role of empathy in design and the scarcity of research on the role of empathy in concept selection. Therefore, formalizing the role of an individual's trait empathy in driving design outcomes in the earlier conceptual stages (e.g., concept generation and selection) of the design process could bring great clarity to the existing research. Without this knowledge, we cannot be sure if, when, or how empathy is important in the design process.
Measuring trait empathy
Trait empathy can be defined as “the reactions of one individual to the observed experiences of another” (Davis, Reference Davis1983, p. 113). Trait empathy is broken down into a cognitive component and an affective component (Duan and Hill, Reference Duan and Hill1996). The cognitive component defines an individual's empathy as dependent on the situation, while the affective component characterizes an individual's empathy by their emotional response and feeling (Shantz, Reference Shantz1975; Strayer, Reference Strayer1987; Duan and Hill, Reference Duan and Hill1996).
One of the widely used measures of trait empathy is Davis’ Interpersonal Reactivity Index (IRI; Davis, Reference Davis1980). The IRI defines trait empathy with four empathic tendencies: (1) perspective-taking measures the ability “to adopt the perspectives of other people and see things from their point of view” (Davis, Reference Davis1980, p. 12); (2) fantasy measures “the tendency to transpose themselves imaginatively into the feelings and actions of fictitious characters in books, movies, and plays” (Davis, Reference Davis1980, p. 12); (3) empathic concern measures “the degree to which the respondent experiences feelings of warmth, compassion and concern for the observed individual” (Davis, Reference Davis1980, p. 12); and (4) personal distress measures an “individual's own feelings of fear, apprehension and discomfort at witnessing the negative experiences of others” (Davis, Reference Davis1980, p. 12).
While there are numerous instruments for assessing trait empathy (Davis, Reference Davis1980; Baron-Cohen and Wheelwright, Reference Baron-Cohen and Wheelwright2004), IRI is one of the few measures in the literature that encompasses both the cognitive and affective components of empathy (Duan and Hill, Reference Duan and Hill1996). In engineering design, Hess and Fila (Reference Hess and Fila2016) argued that both components are needed to allow designers to better understand the end-users’ needs. While IRI has been used in prior work to assess the empathic tendencies of engineering students (Hess et al., Reference Hess, Fila and Purzer2016; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b), it has not been used in relation to creative concept generation and selection. Due to its rigorous development and acceptance in diverse communities of research, this study used IRI (Davis, Reference Davis1980) to model designers’ trait empathy and examine its relationship with driving designers’ creative design outcomes.
Research objectives
Based on this prior work, the main objective of this study was to determine if or how engineering student trait empathy impacts their ability to generate and select creative concepts in an engineering design project, see Figure 2. Specifically, the following research hypotheses were devised:
-
h1: Participants with higher trait empathy would generate more ideas.
-
h2: Participants with higher trait empathy would generate more creative concepts.
-
h3: Participants with higher trait empathy would select more creative ideas.
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_fig2.png?pub-status=live)
Fig. 2. Theoretical framework.
These hypotheses are based on prior work with engineering graduate students that found that trait empathy was related to their innovative self-efficacy (Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b).
Methodology
In order to answer this research objective, a study was conducted with 103 first-year engineering design students who were in four sections of an introduction to engineering design course taught by three instructors at a large Northeastern university in the United States. The remainder of this section summarizes the methodological approach taken in this study.
Participants
Participants were recruited from four sections of an introduction to engineering design course taught by three instructors at a large Northeastern university. Notably, the first-year course studied has received national awards due to its ability to successfully incorporate team-based projects (Ritter and Bilen, Reference Ritter and Bilen2019). In all, 103 first-year engineering design students (73 males and 30 females) participated in the study.
Procedure
The study was completed over the course of an 8-week design project that included the following design stages: (1) Introduction to the Design Process and Team Formation, (2) Problem Formulation and Customer Needs Assessment, (3) Idea Generation, (4) Concept Selection, (5) Detailed Design, Manufacturing & Prototyping, and (6) Final Design. Thus, the data presented here is part of a larger data collection effort geared at understanding the role of empathy in engineering design education (Alzayed, Reference Alzayed2019). However, only the aspects of the study pertinent to the current investigation (concept generation and concept selection) are described here.
At the start of the semester, the researchers presented the study to each of the four sections of the course according to the Institutional Review Board guidelines set forth at the university. Participation in the study was voluntary, and informed consent was gathered prior to the start of the study. Participants were then divided into 3–4-member design teams by the course instructor in their respective sections, and they were assigned the eight-week design project. The project focused on addressing the United Nation's Sustainable Development Goal 3 (Nam, Reference Nam2015), which aims at “ensuring healthy lives and promoting well-being for all at all ages.” Specifically, teams were asked to select between the following challenges: (1) lack of safe water, sanitation, and hygiene services, (2) access to hepatitis-B vaccinations, (3) indoor and ambient air pollution, and (4) road traffic injuries. While participants in all four sections were allowed to select from these four design challenges, the design context of these challenges varied across the sections. Specifically, two of the sections focused on designing for the developed world (n = 50 participants), while the remaining two sections were tasked with designing for the developing world (n = 53 participants) [see “Problem Statements - Sustainable Development Goal 3” (2020) for the detailed problem statements].
The participants then continued to work on the project per the timeline presented in Figure 3. In week 1, participants were asked to complete an extreme user research activity where they were encouraged to use reputable online sources to develop a 1–2-page memo about their chosen user group. In week 2, as a team, participants completed an empathy map (Ferreira et al., Reference Ferreira, Silva, Oliveira and Conte2015) using the information they gathered in their user research. Teams were encouraged to answer the following questions: (1) what does the user say? (2) what does your user think? (3) how does your user act? and (4) how does your user feel? Next, participants were tasked with developing personas for their intended user and formulating point-of-view statements (Dam and Teo, Reference Dam and Teo2017). Toward the end of week 2, participants were tasked with creating a journey map to help them visualize key moments in the daily life of the user (Howard, Reference Howard2014).
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_fig3.png?pub-status=live)
Fig. 3. Timeline of the project.
During the concept generation stage (week 4), participants were involved in two brainstorming sessions: reverse brainstorming (Hagen et al., Reference Hagen, Bernard and Grube2016), where they were given 15 min to brainstorm bad ideas that would make the problem worse; and then individual brainstorming where they were asked to generate concepts for 20 min. Specifically, participants were asked to come up with as many ideas as possible by completing idea generation cards (see Fig. 4 for example). Participants were asked to sketch the idea in addition to writing a short description of the idea. This form of task during idea generation has been used in prior studies in design research (Starkey et al., Reference Starkey, Toh and Miller2016; Toh and Miller, Reference Toh and Miller2016a; Miller et al., Reference Miller, Hunter, Starkey, Ramachandran, Ahmed and Fuge2021).
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_fig4.png?pub-status=live)
Fig. 4. An example of an idea generation card completed by participant 32.
During the concept selection stage (week 5), participants were asked to individually filter out the concepts generated by their team by completing a concept screening matrix where they categorized each idea as “consider” or “do not consider.” Ideas falling in the “consider” category are ideas that will most likely satisfy the design goals and that they would like to prototype immediately (Toh and Miller, Reference Toh and Miller2015, Reference Toh and Miller2016a, Reference Toh and Miller2016b). Meanwhile, ideas that fall under the “do not consider” category have little to no likelihood of satisfying the design goals and you fin minimal value in these ideas (Toh and Miller, Reference Toh and Miller2015, Reference Toh and Miller2016a, Reference Toh and Miller2016b).
Finally, in weeks 6–8, participants were asked to prototype their solutions and report their final conceptual design in a written report. Of importance to the current study, participants were asked to complete the 28-item Trait Empathy survey (see “Metrics”) at the end of week 4, immediately after the concept generation activity, and at the beginning of week 5, immediately after the concept selection activity. The survey was completed electronically in-class via Qualtrics.
Metrics
In order to explore the factors critical to achieving the research objective, the following metrics were used:
Trait Empathy : Participants’ trait empathy was measured using the IRI (Davis, Reference Davis1980), a 28-item survey answered on a 5-point Likert scale ranging from “does not describe me well” to “describes me very well.” This instrument assesses an individual's cognitive and affective components of their empathy (Duan and Hill, Reference Duan and Hill1996); those two components have been deemed as necessary to understand the users’ needs in engineering design (Hess and Fila, Reference Hess and Fila2016). The instrument was utilized in prior research in assessing the empathic tendencies of engineering students (Hess et al., Reference Hess, Fila, Purzer and Strobel2015, Reference Hess, Fila and Purzer2016; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b). The IRI includes four subscales (perspective taking, fantasy, empathic concern, and personal distress) each made up of seven different items. For example, an item in empathic concern is “I often have tender, concerned feelings for people less fortunate than me.” Due to previous research that shows that trait empathy changes between the design stages (concept generation, concept selection) ( Alzayed et al., Reference Alzayed, McComb, Menold, Huff and Miller2021), we have tested the hypotheses with participants’ empathy at those different time points.
The four-factor structure of the IRI has been validated (Davis, Reference Davis1983) and has been implemented to assess individuals’ empathic tendencies (Siu and Shek, Reference Siu and Shek2005; Péloquin and Lafontaine, Reference Péloquin and Lafontaine2010; Kokkinos and Kipritsi, Reference Kokkinos and Kipritsi2012; Gilet et al., Reference Gilet, Mella, Studer, Grühn and Labouvie-Vief2013), including engineering students (Hess et al., Reference Hess, Fila, Purzer and Strobel2015, Reference Hess, Fila and Purzer2016; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b). A reliability analysis was conducted to evaluate the internal reliability of the subscales of the IRI, and a high Cronbach's α was observed for fantasy (problem formulation α = 0.82, concept generation α = 0.83, and concept selection α = 0.91), perspective-taking (problem formulation α = 0.76, concept generation α = 0.78, and concept selection α = 0.82), empathic concern (problem formulation α = 0.77, concept generation α = 0.80, and concept selection α = 0.80), and personal distress (problem formulation α = 0.78, concept generation α = 0.83, and concept selection α = 0.85).
Number of Ideas: The number of ideas was calculated for each participant by counting the number of idea sheets completed by each participant during the individual brainstorming session. This aligns with the quantity metric from the work of Shah, Vargas-Hernandez, and Smith (Shah et al., Reference Shah, Kulkarni and Vargas-Hernandez2000).
Consensual Assessment Technique (CAT): The Consensual Assessment Technique (Amabile, Reference Amabile1983) was used to assess the effectiveness of the ideas generated by the 103 participants. This technique has been widely used in prior research in engineering design (Christiaans and Venselaar, Reference Christiaans and Venselaar2005; Nikander et al., Reference Nikander, Liikkanen and Laakso2014) and has been identified as a global measure of creativity (Fischer, Reference Fischer, Sannino and Ellis2013; Cseh and Jeffries, Reference Cseh and Jeffries2019). The CAT defines that an idea is creative when judges independently agree that it as creative (Amabile, Reference Amabile1982). Using a 6-point Likert Scale, the ideas were rated on the following criteria: overall creativity, usefulness, uniqueness, and elegance (Besemer and O'Quin, Reference Besemer and O'Quin1999). Specifically, (1) overall creativity relates to experts’ judgment of the overall creativity of an idea, (2) uniqueness relates to overall perceptions of how original and surprising the idea was (Besemer and O'Quin, Reference Besemer and O'Quin1999), (3) usefulness relates to the overall perceptions of value, logic, and how understandable the ideas were, while (4) elegance refers to the idea's “simplicity, insight shown, and conciseness of [the idea's] presentation” (Besemer and O'Quin, Reference Besemer and O'Quin1999, p. 288) The four metrics have been previously used in design research to assess ideation effectiveness (Klein et al., Reference Klein, DeRouin and Salas2006; Buelin-Biesecker and Wiebe, Reference Buelin-Biesecker and Wiebe2013; Sinha et al., Reference Sinha, Chen, Meisel and Miller2017; Cseh and Jeffries, Reference Cseh and Jeffries2019; Prabhu et al., Reference Prabhu, Miller, Simpson and Meisel2019; Zheng and Miller, Reference Zheng and Miller2019). Additionally, we asked the raters to rate the drawing abilities possessed by each idea to control for that factor, since drawing abilities have been found to influence ratings of creativity (Chan and Chan, Reference Chan and Chan2007).
The CAT method uses experts to rate 20% of the complete idea set to provide a training set for quasi-experts to rate the remaining set based on the experts’ mindset in rating the ideas (Kaufman and Baer, Reference Kaufman and Baer2012; Cseh and Jeffries, Reference Cseh and Jeffries2019). Two faculty members experienced in engineering design research independently rated 20% of the ideas. Additionally, two quasi-experts (PhD candidate and third-year undergraduate student, both studying Industrial Engineering) independently rated the 20% overlap of ideas to ensure agreement with the expert judges (Landis and Koch, Reference Landis and Koch1977). Each of the quasi-experts’ ratings had high agreement (α > 0.7) (Koo and Li, Reference Koo and Li2016) with the expert raters on each of the five metrics, see Table A1 in the Appendix.
Once inter-rater reliability was achieved, the two quasi-experts rated the remaining 80% of the ideas independently and high inter-rater reliability (α > 0.7) (Koo and Li, Reference Koo and Li2016) was achieved between the two quasi-expert raters for each of the five metrics, see Table A1 in the Appendix. An average of the scores from the two quasi-expert raters was calculated for each metric (overall creativity, elegance, usefulness, uniqueness, and drawing abilities), as per recommendations by Silvia (Reference Silvia2011); see Figure 5a,b for examples of ratings.
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_fig5.png?pub-status=live)
Fig. 5. (a) An idea from participant 53 that received a score of 4 on overall creativity, 3 on usefulness, 5 on uniqueness, 4 on elegance, and 3 on drawing abilities. (b) An idea from participant 81 that received a score of 1 on overall creativity, 3.5 on usefulness, 1 on uniqueness, 3 on elegance, and 4 on drawing abilities.
Propensity for Selecting Creative Ideas: To assess simulated teams’ propensity for selecting creative concepts, we used the propensity to ward creative concept selection metric, PC (Toh and Miller, Reference Toh and Miller2015), a metric that has been used in engineering design research (Toh and Miller, Reference Toh and Miller2015, Reference Toh and Miller2016b; Zheng et al., Reference Zheng, Ritter and Miller2018). Specifically, PC measures the “…tendency towards selecting (or filtering) creative concepts during the concept selection process” (Toh and Miller, Reference Toh and Miller2015, p. 118). For instance, the formula to calculate participants’ propensity toward selecting unique concepts (P Uniqueness) can be summarized as the following:
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_eqnU1.png?pub-status=live)
Similarly, an individual's propensity toward concept selection of ideas rated high in (1) overall creativity, (2) usefulness, (3) elegance, and (4) drawing abilities was also assessed in the same manner. For example, an individual can receive a value (P Uniqueness) greater than 1 if the average uniqueness of the selected ideas is higher than the average uniqueness of the available ideas, indicating a propensity for selecting unique ideas, while a value on P Uniqueness that is less than 1 indicated an aversion for selecting unique concepts (Toh and Miller, Reference Toh and Miller2015). Toh and Miller's paper (Reference Toh and Miller2015) provides further details on the scoring methodology.
Data analysis and results
In order to answer the research objective, statistical analyses were computed using SPSS 25.0, and a significance level of 0.05 was used in all analyses. The results are presented as mean ± standard error (SE) unless otherwise denoted. In addition, effect sizes were classified according to Cohen (Reference Cohen1988). The data used in the analyses of the three research questions is available upon request, and a sample of the data used in these analyses is presented in Tables A1–A3 in the Appendix.
Hypothesis 1: participants with higher trait empathy would generate more ideas
Our first research hypothesis was that trait empathy would be positively related to the generation of more ideas (Duncan et al., Reference Duncan, Boisjoly, Levy, Kremer and Eccles2003; Roberge, Reference Roberge2013; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b). To address this research hypothesis, a hierarchical regression model was computed with the dependent variables being the number of ideas generated by each participant. In addition, since the design context, design problem, and course instructor have been shown to influence creativity (Alsager Alzayed et al., Reference Alsager Alzayed, Miller and McComb2020, Alzayed et al., Reference Alzayed, McComb, Menold, Huff and Miller2021), we controlled for these factors as they were not the focus of the current investigation. To account for this, the independent variables were entered in two blocks: (i) design context (developing, developed), course instructor, and design problem, and (ii) perspective-taking, fantasy, empathic concern, and personal distress. A visual schematic of the hierarchical regression analyses used for this RQ is shown in Figure 6.
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_fig6.png?pub-status=live)
Fig. 6. Schematic representation of the two-step hierarchical regression model for RQ1.
Prior to the analysis, statistical assumptions were checked. The results showed linearity of the independent variables as assessed by partial regression plots and a plot of studentized residuals against the predicted values. Visual inspection of a plot of studentized residuals revealed that the assumption of homoscedasticity was met. There was no multicollinearity in the independent variables, as assessed by tolerance values greater than 0.1 (Fox, Reference Fox1991). As assessed by the studentized deleted residuals greater than ±3 standard deviations, there were 4 outliers. The outliers were found to have no significant impact on the significance of the results and, therefore, the full analysis is presented here. Additionally, there were no leverage values greater than 0.2 (Huber, Reference Huber1981), and no values for Cook's distance above 1 (Cook and Weisberg, Reference Cook and Weisberg1982). Finally, normality was confirmed by visually inspecting the histograms and Q-Q plots. Based on these results, the analysis proceeded as planned.
The results from the hierarchical regression model showed that the design context and problem, and the course instructor, significantly predicted number of ideas, R 2 = 0.124, F(3, 98) = 4.48, p < 0.01, which is considered a small effect. However, the design context and problem, as well as the course instructor did not significantly contribute to the model, p > 0.05, see Table 1. The addition of trait empathy (fantasy, perspective-taking, personal distress, and empathic concern) to this model also led to a statistically significant model F(7,98) = 2.99, p < 0.01, with an R 2 change of 0.063. From the four empathic tendencies, only personal distress (p = 0.047) and empathic concern (p = 0.037) significantly contributed to the model. Specifically, personal distress negatively impacted the number of ideas generated by participants, while empathic concern positively impacted the number of ideas generated by participants.
Table 1. Summary statistics of the regression model on the relationship between the number of ideas and trait empathy
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_tab5.png?pub-status=live)
The findings from this research question partially support our hypothesis that trait empathy positively impacted the number of ideas generated during concept generation. Empathic concern positively impacted the number of ideas generated by participants. This finding partially corroborates a qualitative investigation with engineering students (Fila and Hess, Reference Fila and Hess2016) that found that empathic concern tendencies motivated students to work harder on an engineering task. However, personal distress was found to impact the number of the ideas generated by the participants, while perspective-taking and fantasy tendencies did not have an impact on the number of ideas generated. This finding is congruent to the discussion in the literature that note how being empathic may restrict the designer from coming up with creative innovations to the existing problem (Mattelmäki et al., Reference Mattelmäki, Vaajakallio and Koskinen2014).
Hypothesis 2: participants with higher trait empathy would generate more creative concepts
While the first research hypothesis investigated the impact of trait empathy on the number of ideas generated by participants, the second research hypothesis was that trait empathy would be positively related to the generation of ideas that are rated high in overall creative, elegant, useful, or unique ideas (Duncan et al., Reference Duncan, Boisjoly, Levy, Kremer and Eccles2003; Roberge, Reference Roberge2013; Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b). To address this research hypothesis, four hierarchical regression models were computed with the dependent variables being the average overall creativity, average elegance, average usefulness, and average uniqueness of the teams’ generated ideas. In addition, since the design context, design problem, and course instructor have been shown to influence creativity (Alsager Alzayed et al., Reference Alsager Alzayed, Miller and McComb2020, Alzayed et al., Reference Alzayed, McComb, Menold, Huff and Miller2021), we controlled for these factors as they were not the focus of the current investigation. Additionally, we controlled for the drawing abilities of each participant as the drawing abilities have been found to influence ratings of creativity (Chan and Chan, Reference Chan and Chan2007). To account for this, the independent variables were entered in two blocks: (i) participant's average drawing abilities, design context (developing, developed), course instructor, and design problem, and (ii) perspective-taking, fantasy, empathic concern, and personal distress. A visual schematic of the hierarchical regression analyses used for this RQ is shown in Figure 7.
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_fig7.png?pub-status=live)
Fig. 7. Schematic representation of the two-step hierarchical regression model for RQ2.
Prior to the analysis, statistical assumptions were checked. The results showed linearity of the independent variables as assessed by partial regression plots and a plot of studentized residuals against the predicted values. Visual inspection of a plot of studentized residuals revealed that the assumption of homoscedasticity was met. There was no multicollinearity in the independent variables, as assessed by tolerance values greater than 0.1 (Fox, Reference Fox1991). As assessed by the studentized deleted residuals greater than ±3 standard deviations, there were 0, 6, 4, and 1 outliers for the first, second, third, and fourth regression models, respectively. The outliers were found to have no significant impact on the significance of the results and, therefore, the full analysis is presented here. Additionally, there were no leverage values greater than 0.2 (Huber, Reference Huber1981), and no values for Cook's distance above 1 (Cook and Weisberg, Reference Cook and Weisberg1982). Finally, normality was confirmed by visually inspecting the histograms and Q-Q plots. Based on these results, the analysis proceeded as planned.
The results from all four hierarchical regression models showed that participant's average drawing abilities, design context (developing, developed), course instructor, and design problem, and (ii) perspective-taking, fantasy, empathic concern, personal distress all did not significantly predict overall creativity, p > 0.05. These results indicated that trait empathy did not predict the overall creativity, usefulness, uniqueness, and elegance of the generated ideas.
The results from this research question refute our hypothesis that trait empathy would be related to creative idea generation. While the results from the first research question indicated that trait empathy predicted the number of ideas, it did not necessarily predict the creativity of those ideas. Specifically, all four empathic tendencies (perspective-taking, fantasy, empathic concern, and personal distress) failed to predict the overall creativity, usefulness, uniqueness, and elegance of the generated ideas. These results resonate with prior work that discussed varying points of views (Genco et al., Reference Genco, Johnson, Holtta-Otto and Seepersad2011; Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014; Breithaupt, Reference Breithaupt2019) on the role of empathy in concept generations, whereby we find evidence that supports the notion of the utility of empathy on the number of ideas generated, but the null impact it had in terms of the creativity of those generated ideas.
Hypothesis 3: participants with higher trait empathy would select more creative ideas
The third research hypothesis was that trait empathy would be positively related to the selection of overall creative, elegant, useful, and unique ideas. To address this research hypothesis, four hierarchical regression models were computed with the dependent variables being participants’ propensity for selecting (1) overall creative, elegant, (2) useful, and (3) unique ideas. In addition, since the design context, design problem, and course instructor have been shown to influence creativity (Alsager Alzayed et al., Reference Alsager Alzayed, Miller and McComb2020, Alzayed et al., Reference Alzayed, McComb, Menold, Huff and Miller2021), we controlled for these factors as they were not the focus of the current investigation. Additionally, we controlled for teams’ propensity for selecting ideas that are rated high in drawing abilities since prior research found that the drawing abilities portrayed in a design could have a potential impact on an individual's perception of the creativity of that design (Chan and Chan, Reference Chan and Chan2007). To account for this, the independent variables were entered in two blocks: (i) the propensity for selecting ideas rated high in drawing abilities, design context (developing, developed), course instructor, and design problem, and (ii) trait empathy (perspective-taking, fantasy, empathic concern, and personal distress). A visual schematic of the hierarchical regression analyses used for this RQ is shown in Figure 8.
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_fig8.png?pub-status=live)
Fig. 8. Schematic representation of the two-step hierarchical regression model for RQ3.
Prior to the analysis, statistical assumptions were checked. The results showed linearity of the independent variables as assessed by partial regression plots and a plot of studentized residuals against the predicted values. Visual inspection of a plot of studentized residuals revealed that the assumption of homoscedasticity was met. There was no multicollinearity in the independent variables, as assessed by tolerance values greater than 0.1 (Fox, Reference Fox1991). As assessed by the studentized deleted residuals greater than ±3 standard deviations, there were 4, 5, 3, and 3 outliers for the first, second, third, and fourth regression models, respectively. The outliers were found to have no significant impact on the significance of the results and, therefore, the full analysis is presented here. Additionally, there were no leverage values greater than 0.2 (Huber, Reference Huber1981), and no values for Cook's distance above 1 (Cook and Weisberg, Reference Cook and Weisberg1982). Finally, normality was confirmed by visually inspecting the histograms and Q-Q plots. Based on these results, the analysis proceeded as planned.
The results from the first hierarchical regression model showed that only drawing abilities, but not design context and problem, nor the course instructor, significantly predicted the propensity for selecting overall creative ideas, R 2 = 0.260, F(4, 89) = 8.34, p < 0.01, which is considered a medium effect. The addition of trait empathy (fantasy, perspective-taking, personal distress, and empathic concern) to this model also led to a statistically significant model F(8,89) = 4.51, p < 0.01, with an R 2 change of 0.022; however, all of the perspective-taking, fantasy, empathic concern, and personal distress, all did not contribute to the model, p > 0.05.
While the first regression model investigated the role of empathy on the propensity for selecting overall creative ideas, the second hierarchical regression model investigated the role of trait empathy on the propensity for selecting elegant ideas. The results from the second hierarchical regression model showed that only drawing abilities, but not design context and problem, nor the course instructor, significantly predicted the propensity for selecting elegant ideas, R 2 = 0.345, F(4, 89) = 11.18, p < 0.01, which is considered a medium effect. The addition of trait empathy (fantasy, perspective-taking, personal distress, and empathic concern) to this model also led to a statistically significant model F(8,89) = 6.420, p < 0.01, with an R 2 change of 0.043. Specifically, perspective-taking tendencies (p = 0.027) positively predicted participants’ propensity for selecting elegant ideas. All other empathic tendencies did not significantly contribute to the model, see Table 2 for a summary of the regression statistics.
Table 2. Summary statistics of the regression model on the relationship between participants’ propensity for selecting elegant ideas and trait empathy (* indicates significant results)
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_tab6.png?pub-status=live)
The results from the third hierarchical regression model showed that the design context and problem, and the course instructor, significantly predicted the usefulness of the generated ideas, R 2 = 0.599, F(4,89) = 31.77, p < 0.01, which is considered a large effect. However, the design context and problem, as well as the course instructor did not significantly contribute to the model, p > 0.05. The addition of trait empathy (fantasy, perspective-taking, personal distress, and empathic concern) to this model also led to a statistically significant model F(8,89) = 16.82, p < 0.01, with an R 2 change of 0.025; however, all of the four empathic tendencies, fantasy, perspective-taking, personal distress, and empathic concern, did not significantly contribute to the model, p > 0.05.
Finally, the fourth hierarchical regression model investigated the role of trait empathy on the propensity for selecting unique ideas. The results from the fourth hierarchical regression model showed that the design context and problem, and the course instructor, significantly predicted participants’ propensity for selecting unique ideas, R 2 = 0.179, F(4,89) = 4.62, p < 0.01, which is considered a small effect. However, the design context and problem, as well as the course instructor did not significantly contribute to the model, p > 0.05. The addition of trait empathy (fantasy, perspective-taking, personal distress, and empathic concern) to this model also led to a statistically significant model F(8,89) = 2.82, p < 0.01, with an R 2 change of 0.039; however, all of the four empathic tendencies, fantasy, perspective-taking, personal distress, and empathic concern, did not significantly contribute to the model, p > 0.05.
The findings partially support our hypotheses that trait empathy predicted creative concept selection. Specifically, perspective-taking tendencies positively predicted participants’ propensity for selecting elegant ideas. These results confirm previous work that highlighted the significance of perspective-taking tendencies in an engineering context (Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b; Alzayed et al., Reference Alzayed, McComb, Menold, Huff and Miller2021). However, perspective-taking tendencies were only impactful for selecting elegant ideas, and not ideas rated high in overall creativity, usefulness, and uniqueness. Additionally, the other empathic tendencies, fantasy, empathic concern, and personal distress, had no significant impact on creative concept selection.
Discussion
The main goal of this paper was to explore the role of trait empathy on creative concept generation and selection. The main findings from the study are as follows:
• Empathic concern tendencies positively impacted the generation of more ideas, while personal distress tendencies negatively impacted the generation of more ideas.
• Perspective-taking tendencies positively impacted participants’ propensity for selecting elegant ideas.
The implications of these findings are discussed below with respect to our research hypotheses.
The role of empathy in concept generation
The first finding from this study indicated that while empathy impacted the number of ideas generated by participants, it did not necessarily impact the creativity of those ideas. Specifically, empathic concern tendencies positively predicted the generation of more ideas. This finding partially corroborates a qualitative investigation with engineering students (Fila and Hess, Reference Fila and Hess2016) that found that empathic concern tendencies motivated students to work harder on an engineering task. Meanwhile, personal distress tendencies negatively predicted the generation of more ideas. This relates to findings in the literature that note how being empathic may restrict the designer from coming up with creative innovations to the existing problem (Mattelmäki et al., Reference Mattelmäki, Vaajakallio and Koskinen2014). While trait empathy had both a positive and negative impact on the number of ideas generated, the results found that empathy did not impact the creativity (overall creativity, usefulness, uniqueness, and elegance) of those ideas. These results resonate with the discussion in the literature that display varying points of views on the role of empathy in concept generation (Genco et al., Reference Genco, Johnson, Holtta-Otto and Seepersad2011; Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014; Breithaupt, Reference Breithaupt2019).
The role of empathy in concept selection
While the results from concept generation indicated that trait empathy did not impact the creativity of the ideas, the findings from concept selection indicated that trait empathy did impact the propensity for selecting elegant ideas. Specifically, perspective-taking tendencies positively predicted participants’ propensity for selecting elegant ideas. These results underline previous work that highlights the importance of perspective-taking tendencies in engineering contexts (Surma-aho et al., Reference Surma-aho, BjörklundKatja and Holtta-Otto2018b; Alzayed et al., Reference Alzayed, McComb, Menold, Huff and Miller2021). However, those results were only true for selecting elegant ideas, and not ideas rated high in overall creativity, usefulness, and uniqueness. Overall, the results from this study confirmed prior work that discussed varying points of views (Genco et al., Reference Genco, Johnson, Holtta-Otto and Seepersad2011; Johnson et al., Reference Johnson, Genco, Saunders, Williams, Seepersad and Hölttä-Otto2014; Breithaupt, Reference Breithaupt2019) on the role of empathy in design, whereby we find evidence that supports the notion of the utility of empathy and the negative impact of empathy in both the concept generation and selection stages of the design process. Since the design community has been invested in devising empathy invoking interventions (Raviselvam et al., Reference Raviselvam, Hölttä-Otto and Wood2016, Reference Raviselvam, Sanaei, Blessing, Hölttä-Otto and Wood2017), the results from this research call for the need to prepare specific interventions that trigger certain types of empathic tendencies (e.g., perspective-taking, fantasy, empathic concern, or personal distress) depending on the design stage (e.g., concept generation, concept selection) and the desired outcome (e.g., useful, unique, or elegant ideas).
Implications for AI in design
The main findings from this paper highlight that while empathy may be useful throughout design, the utility of specific types of empathy vary depending upon the design stage. The findings from this research can help guide the development of AI assistants that support designers during concept generation and selection. In this fuzzy front-end of the design process, designers are involved in numerous decisions that require cognitive effort (Toh and Miller, Reference Toh and Miller2019). For example, designers would have to decide whether an idea should be selected or not during concept selection based on a specific set of criteria. Thus, in the pursuit of building better AI systems, we also need to advance a fundamental understanding of empathy.
Conclusion, limitations, and future work
The main goal of this paper was to explore the role of trait empathy on concept generation and selection in an engineering design student project. The main findings from this research highlighted that empathic concern tendencies positively impacted the generation of more ideas while personal distress tendencies negatively impacted the generation of more ideas. During concept selection, perspective-taking tendencies positively impacted participants’ propensity for selecting elegant ideas. These results highlight that while empathy may be useful throughout design, the utility of specific types of empathy vary depending upon the design stage. In other words, the design community should be invested in preparing specific interventions to trigger specific types of empathic tendencies (e.g., perspective-taking, fantasy, empathic concern, or personal distress) depending on the design stage (e.g., concept generation, concept selection) and the desired outcome (e.g., useful, unique, or elegant ideas).
Despite the insights we found on the role of empathy during concept generation and selection, there are several limitations that need to be identified that could lead to interesting avenues for future research. While this work started exploring the relationship between trait empathy and creative concept generation and selection, future research should assess the relationship of trait empathy with other design outcomes, such as the quality of the final design. Moreover, while this research explored the utility of empathy in humanitarian engineering problems, future research is needed to extend these results with other engineering design tasks. Additionally, while this study investigated the role of empathy on creativity in an engineering design task, prior research has argued that the basis of empathic thinking is genetic and could impact the type of professions that individuals choose. Thus, future research is warranted to determine whether empathy has a role on the type of engineering professions they select. Finally, while prior research found that the ideation patterns of first year and senior-level students differ (Alsager Alzayed et al., Reference Alsager Alzayed, Mccomb, Hunter and Miller2019), this work only studied first-year students. Thus, future research is warranted to extend those findings beyond first-year students. Taken as a whole, this research took the first step in encouraging empirical investigations aimed at understanding the role of trait empathy across different stages of the design process.
Acknowledgments
The study presented in this paper is an extension of work published in the Proceedings of the 2020 Design Computing & Cognition Conference by the co-authors of this paper (Alzayed et al., Reference Alzayed, McComb, Menold, Huff and Miller2021). We thank Kuwait University for funding the doctoral fellowship of Mohammad Alsager Alzayed. The authors are also grateful to Daryl Cameron for his help on the project. We would also like to acknowledge the help of undergraduate research assistants Abby O'Connell and Lois Jung.
Conflict of interest
The authors declare that they have no conflict of interests.
Mohammad Alsager Alzayed is an Assistant Professor of Industrial and Management Systems Engineering at Kuwait University. He received his PhD in Industrial Engineering from the Pennsylvania State University in August 2020. He received his MS in Industrial Engineering in 2017 and a BS in Industrial Engineering in 2017, all from the Pennsylvania State University. His research interests are in the intersection of engineering design, human factors, and engineering education.
Scarlett Miller is an Associate Professor of Engineering Design and Industrial Engineering at the Pennsylvania State University. She received her PhD in Industrial Engineering from the University of Illinois in 2011, MS in Industrial Engineering from the University of Nebraska in 2007, and BS in Industrial Engineering from the University of Nebraska in 2006. Dr. Miller's research focuses on developing an in-depth understanding of human physical and cognitive abilities in order to develop next-generation products and technologies that support human capabilities. Her research to date has had three main thrusts: ergonomic product design, design cognition, and human–computer interaction.
Christopher McComb is an Associate Professor of Mechanical Engineering at Carnegie Mellon University. Previously, he was an assistant professor in the School of Engineering Design, Technology, and Professional Programs at Penn State. He served as a director of Penn State's Center for Research in Design and Innovation and led its Technology and Human Research in Engineering Design Group. He received dual BS degrees in civil and mechanical engineering from California State University-Fresno. He later attended Carnegie Mellon University as an NSF Graduate Research Fellow, where he obtained his MS and PhD in mechanical engineering. His research interests include human social systems in design and engineering; machine learning for engineering design; human-AI collaboration and teaming; and STEM education.
Appendix
A sample of the data used in these analyses is presented in Tables A1–A4 in the Appendix.
Table A1. Interrater reliability (α values) between raters for idea creativity assessment
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_tab1.png?pub-status=live)
Table A2. A sample of the data analyzed in RQ1
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_tab2.png?pub-status=live)
Table A3. A sample of the data analyzed in RQ2
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_tab3.png?pub-status=live)
Table A4. A sample of the data analyzed in RQ3
![](https://static.cambridge.org/binary/version/id/urn:cambridge.org:id:binary:20221012133454645-0630:S0890060421000196:S0890060421000196_tab4.png?pub-status=live)