1. INTRODUCTION
Most research on design rationale has focused on its potential utility in support of design reasoning, and as an approach to design knowledge capture and representation (Moran & Carroll, Reference Moran and Carroll1996; Dutoit et al., Reference Dutoit, McCall, Mistrik and Paech2005). Although the larger proportion of this work is concerned with improving the practice of design, some have suggested a role for design rationale as a tool for supporting design research and design science (Carroll & Rosson, Reference Carroll, Rosson, Moran and Carroll1996, Reference Carroll, Rosson and Carroll2003). The objective of the work reported here is to extend this latter thread, to provide empirical support for some of the claims made in this earlier work, and to highlight some new areas where the unique capabilities of design rationale can be applied as a scientific instrument by the design research community.
In this article, we investigate the potential utility of design rationale as a theoretical and empirical tool for research, in particular, for systems design research involving envisioning and creating technologies to support human activity. Our fundamental premise is that some of the very attributes that appear to make design rationale problematic in design practice (its rigor, structure, and relative lack of representational ambiguity, among others) translate into strengths in the research domain. Design rationale is about making explicit the assumptions of design (Brown, Reference Brown2006). This means codifying the knowledge that is created and that emerges from design activity. This knowledge is the currency of research involving the design, development, and use of interactive systems.
Design research has emerged as a bona fide field of scientific inquiry in the various disciplines related to information technology. Conferences, workshops, journal special issues, and other venues all provide evidence that design researchers working with technology are actively problematizing the ontological and epistemological bases of the research they do, the methods they employ, and the claims that they make based on their results. This is being driven partly by the National Science Foundation's Science of Design program, which is supporting research to explore the knowledge-generating role of design and the artifacts that result from design activity. This is, in our view, a positive state of affairs. That there are sharp distinctions between science and engineering, knowing and doing, and understanding and building has always been controversial (Mitcham, Reference Mitcham1994). More likely, the demarcation between design and science is, like other ideal dichotomies: gray and fuzzy rather than hard and fast.
We propose that concepts, representations, and tools developed within the field of design rationale provide a valuable resource for undertaking design research. We are focused on a specific, intentionally narrow view of both design and design rationale. Our concern is with design as a research undertaking and with design rationale as a set of concepts, representations, and tools uniquely qualified to support just this kind of research. We consider design research to be different from design in that the primary focus of design researchers is to contribute to some body of scientific or engineering knowledge. We consider design rationale uniquely qualified because it specifies approaches to capturing, representing, and communicating the knowledge surfaced through design activity.
It is within this context that we propose design rationale as a potential bridging mechanism to help translate design reasoning into tangible, reified knowledge that can be shared and reused. Establishing design rationale as a research approach will contribute to the work of researchers who envision and build technologies, especially software-intensive systems, which is where our own interests lie, by providing a touchstone method for naturalistic descriptions and generalizations. Because design rationale representations are descended from a form of argumentation logic (Toulmin, Reference Toulmin2003), the rigor and structure they provide guides designers to clearly specify how they use data, warrants, and backing; in other words, knowledge, to arrive at a particular design decision.
In this study, we interviewed researchers about the role that design plays in their research, how they use theory and prior empirical results as an input to the design process, and how results from their design research are translated back into theory. The role of design rationale as a theoretical or methodological tool was not directly interrogated in the interview protocol, but was a product of a secondary interview transcript analysis. The analysis revealed that design rationale possesses certain affordances that make it appropriate as an approach to design research. In the next section, we review the background literature on design rationale focusing in particular on its potential role as a medium for conducting scientific inquiry in design. We then describe the study method, procedures, and results before discussing how design rationale might be more effectively leveraged in design research.
2. DESIGN AND SCIENCE
According to Simon (Reference Simon1996), design is the study of how things ought to be, it is normative or deontic, whereas science is descriptive or predictive of how things are or will be. In many accounts of the relation of science to technology, scientific knowledge is characterized as an enabler of technological control over our natural environment (Habermas, Reference Habermas1971; Pitt, Reference Pitt2000). To Simon, a science of design, or science of the artificial, is concerned with understanding the relationship between the inner environment of the designed artifact, the structure and properties of the artifact itself, and the outer environment in which it is intended for use. The specific form of the inner environment is motivated by the explicit goals or purposes the artifact is meant to serve within this outer environment. Simon treats design activity as a kind of problem solving driven by bounded search through a design space. Wilson (Reference Wilson1952) echoes this view “In designing a bridge, an engineer naturally chooses the most economical design which satisfies all the specifications, including the aesthetic requirements.” Design is about using information and creative envisioning to synthesize a solution to a perceived need or goal within a problem domain. The design space consists of those elements of the problem space deemed salient to the design problem; the materials, tools, and techniques brought together to synthesize a solution; and the moves that occur in the design process including especially analysis and decisions.
Today's characterizations of science exist on a continuum from the classical conception, which defines science as a cycle of hypothesis generation, experiment, and observation leading to objective knowledge, to the postmodern view in which science is portrayed as a negotiated stance driven by personal and social interests with the scientist's goal being for his/her theories to dominate those of their rivals (Thagard, Reference Thagard1999). The “science wars” of the 1990s (Sokal & Bricmont, Reference Sokal and Bricmont1998) highlight how fractious our understanding of science really is. On one side postmodernists and other skeptics question whether knowledge can ever really attain the status of objective truth so central to conventional views of science. On the other side scientists in the realist tradition claim that skeptics base their claims on an only limited understanding of both the phenomena of study, and the methods used to study them. This lack of clarity and agreement confounds efforts to understand exactly how design and science relate to one another, because understanding of each varies independently.
For the purposes of this article, we define design research inquiry as being concerned with understanding artifacts and their materials and components, and the ways they are produced, and, importantly, how and why they have the effects that they do when they are used (or misused, or not used) in different contexts. Understanding the reasons and causes behind the relative success or failure of information technology and information systems is the goal of a science of design for software-intensive systems. Our objective here is to progress this understanding by investigating design rationale concepts, representations, techniques, and tools as a means for improving the state of the art in design research.
2.1. Design rationale
One of the underlying tenets of design rationale is that rationale is a kind of argument, where tradeoffs between identified solution alternatives are analytically evaluated on some set of criteria to choose between them within the decision space (MacLean et al., Reference MacLean, Young and Moran1989; Lee & Lai, Reference Lee, Lai, Moran and Carroll1996). Although design rationale is clearly about the reasons why a design is good, bad, or otherwise appropriate for the task goals, it is not always the case that there are identified alternatives for how to address the goal. There is evidence that designers do not develop sets of alternatives, but rather work opportunistically, depth first until they obtain a satisficed outcome, or fail, and then restart again (Thomas & Carroll, Reference Thomas and Carroll1979; Guindon, Reference Guindon1990). The distinction between the two kinds of design rationale are describing the design space of alternatives and describing the argument/reasons for the solution adopted. In our study, we put aside this focus on describing the design space while retaining an interest in design rationale as a means to capture, represent, and communicate the reasons behind a design decision, in whatever form they might ultimately take. Giving reasons and formulating answers to how and why questions are central to the goals of science.
There have so far been only a limited number of empirical studies of design rationale in use. Although at least one has identified some real benefits to the use of design rationale in an industrial setting (Conklin & Burgess-Yakemovic, Reference Conklin, Burgess-Yakemovic, Moran and Carroll1996), others have identified significant problems including especially the usability and technology fit of design rationale tools and representations to the design ecology (Buckingham Shum & Hammond, Reference Buckingham Shum and Hammond1994; Grudin, Reference Grudin, Moran and Carroll1996). Others have argued that lack of progress in design rationale research is a function of the difficulties that arise from the study of designers in action, an inherently complex and situated process requiring access to real designers for observation over extended periods of time (Burge, Reference Burge2006).
2.2. Design rationale as scientific method
One of the strengths of design rationale is that it makes explicit how design criteria are applied to influence a given design decision. Criteria used in this way are all potential candidate parameters for other designs in the same or analogous domains. Design criteria are a form of generalized knowledge that designers (and researchers) can draw on in what is otherwise considered a largely intuition-driven and creative process. Standardized design criteria are concerns that transcend specific design problems. For example, in interactive system design, usability is a key concern, as are reliability, performance, maintainability, and other nonfunctional requirements that reduce the range of possible solutions to a given design problem. Interactive systems design is relatively unconstrained, seemingly, but it may be that the constraints are much more subtle, but no less real in their casual effect. Design parameters in the form of standardized constraints essentially cause a particular design form to emerge in response to an identified problem. Identification of standard design criteria and design parameters is therefore at the heart of rigorous, scientific design (Vincenti, Reference Vincenti1990).
Design rationale has been proposed as a means for managing and evolving scientific knowledge as it is evoked or emerges from design research and practice (Carroll & Rosson, Reference Carroll, Rosson, Moran and Carroll1996, Reference Carroll, Rosson and Carroll2003). Carroll and Rosson argue that the explicitness of design rationale as a technique and as a representation helps to bring to the surface and to communicate the reasons why designers make the decision they do. In particular, they demonstrate how design rationale can form the basis of a design science along three fronts: as an ecological science identifying the causal relations between situated artifacts and for developing scientific generalizations out of the raw material of design practice; as an action science for transferring knowledge between designers, whether researchers or practitioners, and validating design knowledge through its effective application in design practice; and as a synthetic science, a tool for integrating knowledge and theories from diverse fields and sources (Carroll & Rosson, Reference Carroll, Rosson and Carroll2003).
As an ecological science, design rationale addresses the context in which artifacts are designed and used. The ecological perspective consists of three subcategories: taxonomic science, design science, and evolutionary science. In a taxonomic science, designers create an inventory of components and relationships in a software system. Designers also analyze potential consequences and tradeoffs related to designs and design features. As such, the inventory, consequences, and tradeoffs are preserved as design rationale. Design rationale as a design science can be used for describing generalizing categories across design and explaining consequences for users and their tasks. Evolutionary science is at the heart of the task–artifact cycle where the coevolution of technology, human activities, and the recognition of needs and opportunities via technology development exist.
The task–artifact cycle (Carroll, Reference Carroll1990; Carroll & Rosson, Reference Carroll and Rosson1992) has been proposed as one way to capture the flow of information between science and practice. In the task–artifact cycle, design rationale plays the role of a discovery representation, narrowing, and selecting successively smaller views of reality as an aid to scientific discovery. The task–artifact cycle and claims analysis support analyzing and communicating about domain tasks and supporting artifacts using a vocabulary derived from theoretical and empirical science, in particular, psychology, and thereby supports capture of design reasoning and design rationale as designers apply scientific knowledge to specific design problems (Carroll & Rosson, Reference Carroll and Rosson1992). Claims are propositions about the effects of a particular system feature on users in context. Claims analysis is a way of representing and reasoning about the positive and negative effects of a feature and its design alternatives.
Examples of the task–artifact cycle in action have shown how claims analysis and reuse can equate to the scientific application of design rationale (Sutcliffe & Carroll, Reference Sutcliffe and Carroll1999; Sutcliffe, Reference Sutcliffe2000b). Sutcliffe and Carroll developed a framework for classification and description of claims through creation of claims taxonomies. Likewise, Sutcliffe uses claims schema and claims taxonomies as a set of design principles for software architecture components, including generic models of tasks and domains. Reuse of design rationale has also been demonstrated within a theoretical framework of domain knowledge (Sutcliffe, Reference Sutcliffe2000a; Papamargaritas & Sutcliffe, Reference Papamargaritas and Sutcliffe2004). Within this framework, the domain theory, design rationale is applied as an enabler of software component comprehension and the mapping of components to analogous use scenarios both within and across domains. Design rationale is the central component in the domain theory, and forms the basis of solution templates used to address recurring problems in interactive task support.
As an action science, design rationale motivates and guides efficient application of knowledge, motivates the making and assessing of systems, which help with understanding, and motivates the social assimilation of research to practice. In action science, stakeholders are involved in decisions about the research endeavor itself. Furthermore, action science tends to be accurate, forged in practice, robust despite incomplete information, and dependent on public disconfirmability, which are aspects of design that are difficult to address with normal science. Design rationale contributes to action science by acting as a kind of boundary object or boundary representation (Star, Reference Star, Huhns and Gasser1989), for facilitating communication between the communities of science and practice.
The final contribution of design rationale to theory development is as a synthetic science. As a synthetic and integrative science, design rationale addresses the entire scope of system design. Design rationale as claims analysis helps designers see different levels of analysis and the tradeoffs within and across levels. Claims analysis is an analytical framework that makes clear design decisions and addresses the pros and cons of each decision. As such, design rationale built on claims analysis “can be grounded in existing scientific theory or it can instantiate predictions that would extend existing theory” (Carroll & Rosson, Reference Carroll, Rosson and Carroll2003).
Carroll and Campbell (Reference Carroll and Campbell1989) have argued that human–computer interaction is a design science that embodies theories in artifacts. They showed that applying psychological theories and methods to the design of artifacts, then in fashion, was not suited to the scale and complexity involved in a design science where design researchers both study and do design. Other work has suggested that design rationale is able to represent the entire design process, including the resulting artifact and decisions made during design activity, and that these representations provide a lens for design space analysis (MacLean et al., Reference MacLean, Young and Moran1989). Both of these perspectives (design as artifact and design as activity or process) describe ways in which design rationale can serve as an instrument or apparatus to enable research.
Design rationale and the task–artifact cycle have also been employed in the development of theoretically grounded design methods and tools. For example, the LINK system (McCrickard & Chewar, Reference McCrickard, Chewar, Forsythe, Bernard and Goldsmith2006) supports a design methodology that explicitly embeds claims analysis. The DUTCH usability framework (van der Veer & van Welie, Reference van der Veer, van Welie, Diaper and Stanton2004) is a theoretically grounded method that uses design rationale to illustrate how lessons from theory and practice can be synthesized into a usability method. Design rationale has also been used as the basis for a theory of quality driven validation and implemented in a model that provides reasoning support to explain why design objects exist and what assumptions and constraints underlie their existence and form (Tervonen, Reference Tervonen1992).
Much of this prior work is theoretical and analytic. In the rest of this article, we present results from a study of design researchers and use these results to provide empirical support for the idea of design rationale as a tool for research and inquiry. We develop a framework derived from some of the key concepts identified above, namely, ecological/action/synthetic science, and show where there appears to be empirical support in the qualitative data collected from interviews with design researchers. In the next section, we provide more details on the study approach and procedures before presenting the results from the study.
3. STUDY PROCEDURES
In this section, we describe development of the interview guide (the questionnaire), the participants in the study, and the analysis techniques we employed to understand the data collected.
3.1. Data collection
3.1.1. Interview guide
We developed an interview guide designed to elicit participants' perceptions and experiences using design as the basis for scientific research. Our interest was to elicit participants' characterizations of design and design research, their methodological approach to design research, use of theory, and how the research process and results might be improved. The interview guide begins with questions designed to gather basic information about the participant's background, their training, areas of current research, and how long they have been involved in design research. Subsequent questions explore the role of design in the participant's research program, the use of theory both as an input to and an output from design research, what they consider the core goals of their research, and how they believe that design research contributes to understanding, in particular, scientific understanding, of their phenomena of interest. Finally, we were interested in whether participants were familiar with theories of explanation from the philosophy of science. This last question was included to help understand whether participants' conceptions of design science and design research were their own or were derived from the work of others. The interview guide appears in Table 1.
Table 1. Study interview guide
3.1.2. Participants
We interviewed 16 researchers from the ACM conference on Computer-Supported Cooperative Work (CSCW). Participants were selected from the author list posted on the CSCW 2006 conference Web site. Prospective interviewees were selected based on their role as design researchers, by which we mean they conduct research often involving the design, creation, and use of technological artifacts. Approximately 40 CSCW design researchers were initially contacted by e-mail. Six were unable to respond and because of scheduling constraints only 8 of the remaining 34 were interviewed in person at the conference. Eight additional interviews were conduced on the telephone in the 6 weeks following the conference. Interviews lasted from 20 min to 1.5 h. Most participants were originally trained in computer science, although psychology, mathematics, geography, education, and anthropology were all represented. The areas of current research were also quite diverse, and included human–computer interaction, collaborative technology, ubiquitous computing, and health informatics, among others. All 16 of the participants had more than 10 years of experience doing research involving design; half had more than 20 years of experience.
3.1.3. Interviews
Participants were asked the questions from the interview guide in a semistructured interview, with the interviewer using probes or follow-on questions to reveal more detail in response to participants' answers. After the first eight interviews some wording of the questions was revised to help clarify intent and to remove what some participants considered jargon. For example, we replaced the term explanation with the more generic, ordinary language term understanding because it seemed to capture the broader concepts in which we are interested. Results from the data analysis, reported below, suggest that these changes increased the clarity of the questions and contributed to the overall flow of the interview without any obvious changes to the kinds of the responses we received. Similarly, we did not notice any substantial differences between results from the face-to-face and telephone interviews.
3.2. Data analysis
Responses collected in both the face-to-face and telephone interviews were audiorecorded using a digital recorder and then transcribed. The data analysis technique was derived from those commonly employed when developing grounded theory (Glaser & Strauss, Reference Glaser and Strauss1967), except that in this case we were specifically looking at opportunities, benefits, and barriers that would arise in use of design rationale as the basis for scientific design research. Our initial or open coding was guided by our review of related research, which clearly sensitized and guided development of a coding frame used to classify participant responses (Miles & Huberman, Reference Miles and Huberman1994). In the section following, we describe these themes and patterns and relate these to aspects and capabilities of design rationale as described in the review of prior work.
4. FINDINGS
Results of analysis are presented here as a set of envisioned capabilities or affordances of design rationale inspired by concepts and issues identified both in prior research and in the interviews. Affordances, first theorized by Gibson (Reference Gibson, Shaw and Bransford1977), are “action possibilities” of an object in relation to an actor and therefore reliant on the actor's capabilities. This envisioning of affordances is necessarily an interpretive act, a highlighting of the issues that participants identify as central to design research and a mapping to the particular capabilities of design rational concepts, techniques, tools, and representations. We make this envisioning transparent by providing extensive quotes from the interview transcripts and explaining how the concepts raised suggest a role for design rationale. The analysis uses participant data as the basis for identifying opportunities, posing questions, and highlighting opportunities for the use of design rationale as a vehicle for conducting scientific design research.
Five key affordances where design rationale can support design research were identified in the analysis. These were derived from prior work on design rationale, as discussed earlier, and especially from the perspective of design as ecological, action, and synthetic science. The affordances are presented roughly in the order in which we would expect them to engage over the course of a design research project, from marshaling theory as a design input to the descriptions, explanations, predictions, and discoveries that result from the completed research.
1. Marshaling theory: design rationale as a mechanism for identifying, mapping, and then tracing appropriate theories to design decisions and the artifact features that result. This marshaling of theory in the service of design is central to achieving a progressive design science.
2. Synthetic science: design rationale as a fulcrum for leveraging diverse knowledge and expertise in the creation of designs
3. Research apparatus: design rationale as an instrument for observation and reasoning, as a medium for thought, and as an intervention or independent variable in the design research process
4. Boundary representation: design rationale as a shared representation for reifying and communicating design research activities and their results across different communities
5. Description, explanation, prediction, discovery: design rationale as a format to describe, explain, predict, and discover phenomena in the research area of interest
These five affordances support and expand on prior work theorizing the role of design rationale in the scientific enterprise. In addition, direct quotations in the sections below provide empirical support for this theoretical work by linking research participant attitudes, opinions, and experiences to each of the different affordances.
Researchers who include design as a central component of their research practice are sometimes ambivalent toward the use of theory. They sometimes reflect an assumption that “design” and “science” are mutually exclusive. In many cases participant discussions of the role of theory in their design work point to a broad and somewhat mysterious gap between their knowledge and understanding of theories, and how these are actually translated into prescriptions for design. Nonetheless, participants responded to our questions with a diverse range of opinions about how theory is and can be used and these translate into a number of potential opportunities for design rationale.
4.1. Marshaling theory
Our participants frequently use theory as a means to focus their own design research, both as a guiding principle for the creation of a design and as a lens with which to interpret the results of an artifact in use. In this respect, theories appear to act as conceptual boundary objects, supporting transfer of a principle to the design space and then assuming (or losing) additional weight in response to evaluation and other research results. As one participant put it,
You do a little bit of theory you do a little bit of design and you go back to a little bit of theory and do a bit of design and evaluation fits in like that.
The level of specificity at which theory is applied in design research varies widely. In some cases it appears to act at the highest level, as a conceptual sensitizing device heightening the researcher's attention to what is known about phenomena and designs in a given domain. Other times the level of specificity is quite detailed, a particular phenomena is targeted with a specific design response, for example:
One of the main tenets of distributed cognition is that if you can get stuff in to the world you're lowering the cognitive load on short term memory.
The impact of theories on design research appears to be directed relatively more toward the front end of the research process. As the participant below notes, theories act as an orienting device to help designers understand the phenomena likely to manifest themselves in a particular domain and social–psychological setting.
Then there is a third type of theory which I call an orienting theory, which is how I describe the phenomenology which it needs you to ask certain kinds of questions and focus your attention on certain aspects of what you are doing without trying to give any kind of a quantitative measure or correlation or something like that. If you look at phenomenology you want to look at what is the nature of the breakdown in the coupling between the user and the system such that those breakdowns lead to reflection. So it tends to get your attention. Things like activity theory if you look at all the people talking uses the word theory all of the time and I don't think it has the characteristics of the first two kinds of theories I think it is much more a way of, and those I think get used in a much more implicit way.
Design rationale is intended to capture the reasons why designers make the design decisions that they do, how they move through a design space to identify questions and the answers or solutions to those questions, and the criteria they use to determine that a particular solution will work, or will work better than other possible alternatives. In the research context, these criteria often derive from theories or hypotheses about what will work best as a design response to a particular challenge.
Among the most prevalent theories used by the CSCW researchers we interviewed are theories about people and social groups. Again though, what is often less apparent is the way that these theories are used to drive design prescriptions. One of the most evident potential benefits of design rationale is making explicit how theories from the human and technical sciences are linked to specific design prescriptions. Design rationale helps to expose the underlying propositions and mechanics of a given theoretical position by exposing the otherwise invisible reasoning that unifies a theoretical construct with a constructed object.
Producing explicit knowledge is the central objective of science. Design activity and its relation to the artifacts that result are notoriously invisible, or tacit, meaning that much of the knowledge inscribed into a technology is unavailable to those who later hope to progress or assess the ideas central to a particular design approach. This invisibility means that much of the reasoning over what makes a design function well or poorly is predicated on suppositions about its intent and how this intent is realized in the artifact.
… a theory when marshaled properly can help make these big epistemological breaks and you know like if you need a language for talking about why the graphical user interface is better than the command line user interface there is a lot of cognition theory that is really helpful in explaining it and in absence of that body of theory, uh there is really there is no way to explain why those two are different but at the same time it can be dangerous to privilege what gets called theory too much and that it ruins the making practice which is the central organizing effort for design.
The quote above highlights both the promise of more theoretical (scientifically grounded) design and the ambivalence that attends researchers' attitudes toward the use of theory in building. Also evident though is the central role of theory in helping to explain why a design has the effects that it does in the use context. Linking these effects back to the rationale that led to a particular artifact structure or behavior helps complete the cycle of reflection through which we learn about design and technology.
4.2. Synthetic science
Participant discussions of the nature of design research frequently pointed to the synthetic and integrative nature of the work they do. Design, especially for interactive systems, involves bringing together a range of theoretical positions, prior empirical results, and researchers with many and varied expertise. Simon similarly characterized design science as integrative, in particular, integrative of formal models and theorems coupled with empirical results derived from experiments and simulations (Simon, Reference Simon1996). In this way, design rationale may play an important role in enabling design as a synthetic science with the capacity to traverse multiple levels of analysis in a systematic fashion.
… one of the ways I think about designing a system, is you design a system, or an artifact that you've essentially done is embodied, ah a number, often a very large number of hypotheses into an artifact. And then when you deploy that artifact out into the world and watch how people use it ah that gives you some understanding of how those various hypotheses are working or not working or right or wrong or playing out.
What often remains unsaid in quotes such as that above is how exactly we trace all of the conjectures built into a research design to the results obtained in evaluation. Of course, it may not be possible or necessary to map all of the results from a design study to a specific feature and its rationale, but extending our ability to understand the complex couplings of feature to effects increases the field of view tractable to causal analysis.
So for example if you want to do something that that uh that requires properties of both say a fisheye view from the George Furnas and Treemaps from the Ben Scheiderman sense they're now work together that that uh the tool that he is building would enable you to consider the properties of each of those and consider how they could be merged together such that you could have a fisheye tree map that would enable you to get at the benefits that you want from both of them ah and limit the downsides of each of them.
Design rationale may act as a mechanism for supporting multidisciplinary design. The use of bridging criteria, claims taxonomies, and other generalizations provide a portable package that can be used by design researchers. These could be provided with some of the example decisions the rationale has influenced help to communicate by analogy, and also to expose design reasoning to external assessments of validity. In this way, designs in science could be subjected to the kind of open scrutiny allowed by, say, an equation or ethnography, where as much as possible is made explicit about how findings are arrived at.
… a lot of design processes you have ah stakeholders from a lot of different disciplines right I mean we have technical people we have the designers we have the people we call user, but of course users are not sort of generic vanilla things, users are people who come typically with their own areas of expertise and so one of the big problems in carrying out the design process and this is where I started or one of the problems in sharing understanding from any kind of endeavor outside of the small group or discipline or group, ah one of the big problems is how do you communicate knowledge across disciplines and or across context.
Design rationale can act as a boundary representation for communicating between different disciplines. This is particularly important in cases where experts from areas come together on a development project and the goal is to leverage the unique strengths and capabilities of each.
… she's really first rate ethnographer, first rate psychologist and the prototypes that her groups build are solid but certainly less strong than one would expect from a computer science department.
One challenge for designers is understanding how diverse fields with their diverse theoretical and empirical knowledge, and sometime apparently irreconcilable technical discourses, can be leveraged together in the design process. This knowledge takes many years to acquire and in the increasingly complex and specialized world of information technology, researchers will remain anchored primarily to one or two areas of expertise where their skills and experience are most fruitfully applied.
In an ideal world I think all knowledge would be equally acceptable and any design that you come up with you would then consider all previous design knowledge equally and have the opportunity to combine or integrate or come up with new design knowledge that would somehow be better and in the end would borrow components that ah address the ah obviously or solved or new need that you have and to innovate at the gaps in which innovation is required. So that's focusing innovation on the problems that truly require it and not reinventing the wheel for all of the problems that have been addressed before.
The synthetic aspect of design rationale may be particularly valuable in the education and training contexts. Design rationale representations clarify and make explicit how design criteria are employed to drive the creative process and to assist with selection from among identified design alternatives. In the increasingly inter- and multidisciplinary domain of information technology research, we need better tools to represent and express how different fields of knowledge are brought to bear in the creation of new technological solutions.
… you can't do software engineering without HCI. Because you can't do software engineering, you can't practice it without addressing all the HCI issues including CSCW issues, but also whatever the software is you are building, if you don't know about HCI you can't do the user interface and that's such a large percentage of the code that is a lot of the job.
4.3. Research apparatus
Scientific instruments and other apparatus facilitate observation of phenomena through capabilities specially tuned to the attributes of the target phenomena, for example, a microscope for small objects, or a telescope for distant objects. An apparatus can act as a medium to foster conditions conducive to the emergence of an effect, as for example when agar is used to feed cultures growing in a Petri dish. They can also play a more active role in creation of the effect of interest, such as when a Bunsen burner is used to heat a chemical toward a reaction temperature. Software-intensive system design likewise employs a range of instruments, including programming languages and integrated development environments, media such as computing and communications infrastructure (e.g., networks), and the cameras, recorders, and software used to track human interaction with a design under study.
An obvious gap in the domain of these existing apparatus are tools for enabling observation and analysis of the design space. Although opinions differ as to whether software-intensive systems design is more like engineering design, more like art, or some craft in between, in the research context there seems a clear need for devices that allow researchers to examine design both as phenomena under study and as the cause of those designs we wish to study.
… there isn't any scientific understanding of the design space. Because most of the time coming to a new point in the design space is a substantial amount of work. You don't know its structure a priori. So you can't know if there is anything out there.
One of the complexities of design research is that a design can be either the independent or dependent variable depending on the phenomena of interest to the researcher. Another confounding aspect of design research is the issue of construct validity, whether the design as realized actually embodies the theoretical construct proposed for study.
So if we were trying to design something that was say playful, or whimsical or futuristic or retro, I think we have all sorts of methods, many adapted directly from social science to understand whether we have achieved those designs.
Sometimes the whole of a design simply acts as the medium, a support structure, for only one or a few features of interest to the researchers. As with colossal instruments such as particle accelerators and the Hubble telescope, significant design, engineering, and construction effort is required to serve experiments, observations, or other research activities forming the core of a given program.
… one can develop scientific understandings about the properties of a designed artifact by simulating its behavior with a computer program, which allows how a designed artifact will behave in a certain environment. Classical engineering science.
Among the qualities often attributed to scientific instruments is their accuracy and precision. Like a microscope, a well-formed rationale focuses attention on the salient aspects of a design research problem, linking theoretical constructs to working technology.
… you design an experiment it should you know test the claims that are most important.
Some of the challenges to design research instrumentation are fundamental, as highlighted in the following quote from a participant researching design support technology.
… what I'd like to be able to do is to say something about how designers are acting differently in the presence or by using such tools.
As in any other research field, our observations and the tools we use have the potential to change and confound the phenomena under study. In design research, this effect is particularly pronounced because the design is both the object under study and the medium in which effects occur.
… the things you do as a researcher can inevitably change the acts you are studying. Because if you are building a new medium to support design what designers do with that medium will be different from what they did without it. So you hear your interventions as a researcher are always changing the reality in which you are studying. But what you are trying to get at is theories for how to improve how people work.
One of the potentially most promising roles for design rationale representations and tools is to act as a repository for research design knowledge and cases. Although in design practice the Unified Modeling Language has become the de facto standard for representing designs, and design patterns have a significant foothold as a shared representation in both the research and practice domains, neither of these capture and support observation and analysis of design activity and reasoning. Although both can be augmented to include rationale, both are typically used as a representation of what has been decided, not why and how these decisions were reached or what alternative were considered and discarded along the way.
One of the things that really bothered me when I first started in software engineering research is because you know in order to understand principles you have to have a lot of, and this is one of the reasons I started doing qualitative research, because in order to have principles you have to have a lot of cases and you have to be able to see the generalities across the cases, right, when I started software engineering research there were even no empirical descriptions of what software engineers did. So it's kind of hard to have a principled basis to design when we don't even understand you know at a very basic level what they are doing on a day to day basis.
4.4. Boundary representations
Design rationale representations provide a theoretically grounded set of formalisms and semiformalisms for communicating both the ontology of a design space and the process of design space analysis that moves a design from intent through reasons to artifact. Design representations are boundary representations; indeed, some have claimed that design representations only emerged as a response to the need for multiple designers and craftspeople to collaborate on increasingly sophisticated technologies (Micham, Reference Micham1994).
I think we can argue pretty convincingly that any system that supports design space will privilege accessibility … it'll privilege accessibility over expressibility, meaning that how you get to something is an absolutely critical part of what such a system might support. And we build that argument both by appeal to prior research work and by analytical argument by analyzing the necessary structure of design space.
The design space of a modern interactive system is extraordinarily complex. As shown in the earlier section on synthetic science, modern design teams are frequently composed of designers, engineers, scientists, and artists from a wide variety of base disciplines. The modes of communication between these groups is, perhaps inevitably, ad hoc and driven by pragmatics. Nonetheless, participants viewed communication as one of the major challenges in modern design teams.
Uhm that's a challenge. A lot of people who we work with who are developers, uhm engineers in general want things that are instantiated in the world that they are used to. Like in the products they use and stuff like that. So as a researcher in IBM which is where I am. Ah more often than not I have to instantiate a design against that are IBM products because people I want to persuade are the people who work at IBM and those are the things they are used to.
One conception of design is as problem solving claims that “solving a problem means representing it so that the solution is apparent” (Simon, Reference Simon1996). Design rationale representations support not only communicating outside the design research team, but also within the team as a mode of operating on an evolving design.
The main thing I hope to get from a storyboard is uhm, a reasonably clear uhm, I hate to use the word vision but it is almost like a vision, of what this thing is supposed to achieve. so that all the various stakeholders have a chance to say you know my favorite thing is or isn't in this, uh this isn't going to work this way, or you know I don't believe it or whatever, they all have a chance to kind of express their own concerns about whatever it is we are trying to build through that storyboard and so it's used as a kind of coordination mechanism, it's used as a vehicle to make sure that people on the same page before they start in on what is, what is usually a much more costly enterprise which is building the system.
Especially promising is the potential of design rationale to help create shared, boundary representations of the design process, of how designers move from intent through envisioning to specifying and building an artifact appropriate to the use context.
So if you are building a representation it's not just of a thing, but it's there to support what designers do with that thing. So our design space representation is very very much about thinking about how designers can move around in design spaces.
The way you represent the design, the way you represent changes to a design. The way you represent what a designer does. but mostly in my work they've been used to represent designs, spaces of designs, and how one changes designs.
Some conceptions of science including even the “hard” sciences posit that scientific truth, in any meaningful sense of the word, is just that which is agreed upon by the largest number of stakeholders (Goldman, Reference Goldman1999). In the quote below, one interview participant points out that this version of truth or research quality extends even to mathematics.
… they argue from ah a lot of evidence from mathematicians that you cannot prove any real program is correct. Because it is too complicated. And they say that the people who advocate that don't know how mathematical proofs are done. The mathematical proofs are social processes.
The design patterns movement has shown the value of shared representations of design problems and solutions and provides an example of how a community can adopt a representational schema to enable more effective knowledge sharing. Although design patterns are an important means for sharing knowledge in the practical context, they fall short when called upon to expose and communicate the larger knowledge base, including theoretical and empirical knowledge, that are included in research designs. Rationale patterns, in contrast, have the potential to provide researchers and their students with examples of integrative, theory-driven design reasoning.
Just as ah the other fields, other fields have exemplary data sets for testing but design doesn't have them. Really useful to have a few dozen of those. in terms of social utility, the application of any design idea, most design researchers have been in the labs or have applied to toy problems as demonstrations. To actually get it out in practice and understand how it works in the wild requires some research knowledge and skill that typically isn't in much evidence in most design research labs and it's social science. Qualitative social science largely. And you've got to get out there in the field with indicating where new ideas are being applied.
By communicating the rationale for how a theory of other body of scientific knowledge impacts on design, social rationales may also help to bridge the gap between design research and design practice.
I think there are possibly two goals. One of them would be our design work having more influence on practitioners … and the other goal would be our design work having more influence on researchers.
Design rationales represent explicitly the identification of alternative approaches to a design problem and the criteria used to select from among them. Even in cases where design follows something less than this idealized model, still the representation of the reasoning makes clear the discourse leading to a design move, even when the rationale reveals a process that is less than rational.
I guess, for me my goal is to is to somehow demonstrate that a process is yielding worthwhile results, that the design process that I am undertaking is somehow ah making people do things better. Whether that be faster or of higher quality or whatever. That is what I seek to do when I do research on design.
Boundary representations of design rationale play a role in more practical contexts as well. Shared representations of the arguments surrounding a particular decision play a key role in participatory and other stakeholder-inclusive design approaches.
… something that seems like it would be useful during requirements analysis you know why aren't our participatory design sessions focused around you know we're going to talk about these five claims …
There are people in the world for whom a design has to be made reasonably specific to the way they experience things, otherwise they don't get it. And there is another group of people for whom it is important that design be fairly certain clear that it is general otherwise they believe it is too specific.
4.5. Description, explanation, prediction, and discovery
The boundary representations that are designs are created to serve a number of different purposes in the research context. Designs are embodiments of theory that help to describe and explain how the theoretical constructs have the effect they do when reified in a working system. In more traditional modes of science, designs are a prediction, often implicit, that the technology realizing the design will improve the situation of its users and other stakeholders. Sometimes these predictions are specified in greater detail, sometimes much greater, as when a feature of the design is expressed as having a particular effect on those who use it.
Other times designs and the results obtained from studying them in use are treated as post hoc descriptions or explanations of the effect a technology has in context. In their most basic form, these take the form of taxonomies, categories of technologies mapped to the intentions giving rise to them, and descriptions of the design process employed to realize the intention.
I actually wrote a paper on how people design and evaluate collaborative application infrastructures. So I was providing a taxonomy and saying that this is the whole process as far as I could see and I tried to fit various works I have seen various research efforts I have seen into this taxonomy of how the design process goes and how the evaluation process goes. So yes I have tried to describe. I have written many survey papers where I have tried to describe systems. So I try to describe applications, I have tried to describe architectures and I have tried to describe at a meta level the process used.
In some cases, descriptions are meant to be revelatory, to provide insights beyond the obvious that help outsiders to understand what happens in the design process. These descriptions are meant to uncover the causal forces that influence the relative success or failure of a design project.
There is a difference between how people developing systems think they work and how they actually work which goes back to Suchman's notion of plans [as] different from situated actions and it's often this goes between plans and situated actions that cause systems to fail. I often get called in after a system has shown some difficulties, not always, and I usually make an argument that by observing what people are doing in relation to the technology that sucked it should generate insights that will allow me to effectively collect information that can be sent back to designers as well as to implementation teams because I mean it's an iterative process.
Design rationale brings the advantage of an explicit structure to the form that design descriptions can take. Rationale representations force their creators to be more explicit in their descriptions of a design, to expose and clarify the justifications behind a design decision.
I would say describing, I mean it tends to be a dynamic process and claims and tools like claims are meant to move from a vague fuzzy understanding of it to a description of it, it's about describing it.
Observations are descriptions of some phenomena, of an object's existence or of an event's occurrence. Observations are the raw materials of science, we use them as the cornerstones for more speculative thinking, for theorizing about how and why we observe the things that we do.
Simulations are very much like observations of any sort. They give you some fact but that doesn't prove anything for sure and it doesn't even explain anything. It's an observation.
Some participants suggest that descriptions are just the first in a series of discursive moves, each dependent upon the one preceding for their expressive power. Explanations, for example, depend on a clear definition or description of the terms that make up their content. Some conceptions of explanation suggest that they are just the post hoc predictions, a good explanation of an event in the past is a good candidate prediction of what will happen in the future.
If you look at describe predict and explain they are hierarchies. They are a spectrum of explanation, you describe when you don't have concepts, you predict when you don't have model, and you explain when you've got them both. And so, and in terms of control, it's kind of a natural science and engineering argument. And ah I think a variant of it applies. perhaps in proof rather than in control because what we are trying to do is to build tools that people find useful in doing this very complex work we call design.
One of the chief claims of science is that it goes beyond folk understanding of the world to provide more in-depth, sometimes causal account for how and why the world is as it is. This depth is, arguably, also key differentiator between design in practice and design as science.
You have to be able to go deeper and see a deeper explanation. So you at least to some number of levels have to be able to get at something more than just the surface explanation. so it has to be there is a word for it. Uhm, I can't remember the word now. But you have to be able to poke at it and get something back. There is a word. you have to be able to investigate it. There is a particular word and I am just blanking on it but the explanation I guess one thing has to have enough parameters or hooks or something so that you can test it out.
The structure and rigor of scientific explanations allow for their terms to serve as parameters in a kind of qualitative equation. Substituting different values for terms in the explanation is one way to carry out the kind of thought experiment that leads to theorizing and hypothesis generation.
Scientific explanation needs to have those affordances. It may be a word for it but things that you can plug in so that you can say well what if it was a little different, then what would the theory tell me? And so that is part of it. But it's also got to tell you why. This kind of deeper explanation. Because if I just literally plug into a black box and get yesses and nos back in the first place I don't even know how to vary the parameters. I don't know the sensitivity.
Explanations are themselves boundary representations. They provide accounts of things in the world, often including a conjectured cause for how and why the thing obtains. Most importantly, explanations (excepting self-explanations) are public productions, researchers are often judged, especially in the classroom, by the quality of the explanations they provide.
but you can back off and say what is a good scientific explanation of the process of design at the meta level. And everything would apply there. How to be believable, how to be grounded. How to be inspectable. That is the word I was looking for. So it has to be an inspectable theory. Meaning that there has got to be something in that explanation that I can push on and find out more. I may have to work at it a bit, but if it is just, you know if it's opaque and I can't get inside it then it is not very useful.
A key attribute of a scientific explanation is that it holds the potential to generalize from a particular instance to a broader range of phenomena. Indeed, explanatory unification, the ability to account for more phenomena with a simpler explanation, is traditionally considered a defining characteristic of a good theory.
So, a good explanation to me will actually say what it is in this design that is that captures the essence of what it's about so that it lives beyond the design, so if I have an explanation that says why this works and why this failed, I can throw that design away and come up with a different one and it will also tell me something better far beyond that example.
Explanations are claims to understanding. When they are made public, they are subject to the scrutiny of peers and other knowledgeable individuals who can, through logical analysis or experiment, attempt to refute the validity of the account provided.
The key word that jumped out at me is reliable, that well we are looking for a reliable explanation of phenomena, and so that certainly involves measurement as well, but the reliability of the measurement really says we understand it enough and that's were you can get into issues of predictability and if we understand it enough to be able to say, the next time we go and measure this this is what we are going to get, uhm, that is an explanation that has a lot of value a lot of credibility.
… there is nothing as valuable as having the voice of somebody who really can, an individual can offer the explanation of why something works the way it does.
One of the goals of science, especially science with an engineering, design, or other applied dimensions, is to predict phenomena, and if feasible, predict how a technological intervention can result in a future state somehow better than the one we are experiencing now.
… certainly describe. I'd like to be able to predict. That's the goal.
I view myself as an inventor, so wanting to create some technology that serves some useful human need but in a way to try to understand. You know because we do it on the small scale so trying to predict what might be the affect of the impact of doing this on a larger scale.
Accurate predictions in fields with the complexity of software-intensive systems design are confounded by an array of mediating conditions. Our understanding of modern information technologies is still in its infancy. Research into individual psychology and social systems is more mature, but we are still a long way from understanding why people behave the way that they do alone or in groups. Youngest of all is the study of human–computer interaction, of how information technologies act to cause certain psychological or social responses.
Perhaps the greatest value in scientific predictions comes just from the attempt, and the incremental development of understanding what is gained from predictions even when they largely fail.
I mean what fits the place of prediction in the design process is iterative prototyping. So you create an artifact, an artifact could be anything from a drawing to almost a running model, prototyping covers a huge range, which when you give it to appropriate people, which could be the end users could be other designers, could be even yourself at other time will cause them to come up with new insights into what works and what doesn't. So the predictive activity is coming, you know what can you put together that will lead to the right questions. Now design is sort of that whole thing.
Design rationale may support prediction and discovery either qualitatively, by structuring information in such a way as to be tractable by a single design researcher, or computationally, by supporting pattern matching and gap identification across very large data sets.
I want then to actually get into the design habit of forming lots of ideas and you know reflect on which ones you want to pursue and then take one to develop for. So and the results have been really successful, it's ah, you know what students do in a ridiculous short amount of time with minimal training just blows me away. and like I really mean that. they are doing things that I never would have thought.
The process of scientific discovery is not well understood, but one conception prevalent in design is that it involves trying things out to see how they work in context.
Sometimes we just design because it's fun. You know we get cool ideas and in fact the reality is a lot of our things start because we think hey this is a neat idea let's actually run with it and we'll retrospectively go back and say well if we think about why we think it's good let's go back and actually see if there is any evidence for it, and see if it supports our beliefs and then we'll often modify the design. And then go on from there.
The claim that design rationale may help to support the creative aspects of information technology is itself in the spirit of discovery.
… what if you were just talking about the term creativity. or problem solving. Where does new inspiration come from. whether it be in a physical science, medical sciences, you know what we're looking for is innovation in computer science, that's what we do and we're looking for different methods that help us do that better.
The justification for design rationale as a vehicle for discovery hinges on its potential to structure and make more apparent the large and diverse range of factors that influence the decisions inscribed into a design.
The reason I sort of hedge when we talk about science of design is when I, in my approach to design, design is about noticing and paying attention to a lot of the details and not ignoring things and design in my view is not about any, maybe some prediction is involved sometimes, but that is not the point, and uh it is not about really prediction and control it is about creating rich environments and artifacts which people can use in ways that the designer doesn't necessarily anticipate or expect.
5. IMPLICATIONS FOR DESIGN RATIONALE AND DESIGN RESEARCH
Results from this study contribute to the literature of design rationale and design research generally by providing empirical support for affordances identified in previous work, and by integrating these into a framework for describing the use of design rationale in a research context. In particular, the framework suggests a role for design rationale as a way of marshaling theory and transforming theoretical statements and propositions into prescriptions for design. It also points to the need for tools to assist with the synthesis and integration of prior research from different disciplines and across levels of analysis into coherent design hypotheses represented as artifacts. Design research involves the reification of theories into tools to be used as a research apparatus to enable inquiry. Because it provides a vocabulary for describing design activities and artifacts, design rationale acts as an important boundary representation for the knowledge invested into and derived from design research. Finally, techniques such as claims analysis have been proposed as a means for creating descriptive taxonomies for design reasoning; for explaining and predicting causal relations between domains, behaviors, and design interventions; and for facilitating discovery of new theoretical propositions.
Among the most fundamental activities in science is development of theories and searching for ways to operationalize and assess the truth content of theoretical conjectures. Design rationale can help by providing a framework for understanding how theoretical conjectures are translated into artifacts, and by linking conjectures through bridging criteria to the specific criteria and design alternative chosen for a particular implementation. Such an approach to marshaling theory provides a traceable means for working back and forth from idea to artifact, for reflecting upon results of design evaluations and analyses, and for adjusting either the theoretical constructs or operationalized treatments in response to research results.
There is a role for design rationale as an enabler of the hard work that must occur in order for researchers to synthesize, integrate, and comprehend the increasingly vast array of theories, data, and disciplines that inform modern design of interactive systems. This opportunity may represent a point of leverage, of added value where design rationale can offset some of the acknowledged costs of its capture and management. It would facilitate this to have better representations and more usable design knowledge management tools to help researchers make more of their design reasoning explicit.
A key differentiator of design research from other forms of inquiry is its focus on the creation of artifacts as a central activity in the research enterprise. Design researchers create artifacts to reify their theories about how the situation in some domain can be improved through the introduction of a new technology. These artifacts are both the object of study, and the means by which they are observed. In essence, they are the instruments or apparatus used to enable observations of their own effects.
One of the potentially most significant opportunities for design rationale is to act as a common vocabulary, a lingua franca, or boundary representation for researchers whose work involves design of complex, software-intensive systems. As shown in the previous sections, this vocabulary may already exist as an implicit or tacit form of discourse between those whose research often involves specifying and building these systems. The opportunity for design rationale is to make the form of this discourse more explicit and sharable, and therefore more teachable, by specifying the syntax of moves through design space, opening it up to scrutiny by the community of interest, and allowing others to build on prior design work as a true progressive science.
As representations of how theories and prior empirical work are translated into designs, as well as results on how these constructs fair in evaluation, design rationales may serve as a sort of template or schema for how subsequent studies should be undertaken. The rationale itself plays a role as a reasoning pattern, a sharable justification for why certain design decisions are made in response to a specified goal, situated within a set of environmental constraints, and for a particular use context. One participant was explicit about the potential usefulness of such a device:
… a piece of software that would help us construct models that were consistent and you know that embodied the theory, that is by using this tool you would be constructing models that were good examples of what the theory said.
Observation is central to science as a well-defined activity. Hand in hand with observation is the activity of description, writing down and communicating observations made in the course of scientific inquiry (Wilson, Reference Wilson1952). Physical scientists and natural scientists, engineers, and ethnographers, to name just a few roles, all have well-defined methods for recording and communicating their observations. Traditionally these have involved the use of special-purpose tools, ranging in sophistication from notebooks to space-based telescopes. Design rationale and its attendant representations and tools provide structure for capturing these observations, for reasoning about them to develop explanations for their occurrence, for predicting how observed phenomena might interrelate, and for the discovery of as-yet unobserved phenomena and relations.
5.1. Clarifying the role of design in science and research
Leveraging opportunities for design rationale to support scientific research requires a clearer understanding of how design activities and design products fit into the scientific enterprise. Achieving this clarity may be problematic, however, given the diverse understandings of both design and science as evidenced in the opinions of interview participants. The relationship between design and science is, according to many of those we interviewed, quite complex and sometimes even mysterious. It appears that to at least some degree, researchers in this area would like very much to experience the best of both worlds in terms of how they practice design and how they do science. Design is certainly the more liberal discipline, admitting a host of perspectives and activities without necessarily prescribing specific ones as right or wrong or better or worse.
Study participants sometimes reacted sharply to the notion that their design work might somehow be a species of science, especially science of the more positivist or realist kind. This was a curious result because almost all of those interviewed are either in academic positions or are associated with industrial research laboratories. This perception of science as somehow constraining design may act to limit efforts such as the one represented here, which aim to increase the value, insights, and reusable knowledge obtained from design as research.
I've taught research methods and I've taught scientific methods and there is in some respects in my mind very little overlap between some of the scientific methods and the process of design.
5.2. Future research
Among our core interests is obtaining a better understanding of how design research can be facilitated by design rationale. Moving forward, we plan to clarify our understanding of different design research methods and concepts, and the opportunities and affordances that design rationale presents for supporting these. In particular, we intend to continue development of the integrated framework presented here through analysis of more diverse interview data from varied fields including human–computer interaction, software engineering, and information systems. We also intend to carry out evaluation and refinement of the framework through member checking with a sample of study participants. Our goal is to generalize and broaden these results across different design research disciplines.
6. CONCLUSION
We have reported on a study of design researchers and their perceptions and experience carrying out inquiries involving the construction of software-intensive systems and other information technologies. Analysis of the interview data reveals opportunities, and challenges, related to the scientific basis of design research and the potential for design rationale to act as a facilitator of design inquiry. These include marshaling theory as a first class input to the design process, and integrating diverse theoretical and empirical knowledge in the construction of modern systems, where development is informed by an increasingly broad range of technical, psychological, organizational, and other social theories.
Design research requires its own research apparatus for capturing, documenting, tracking, and viewing the evolution of theoretical concepts in a progressive science of design. The process of marshaling that occurs between theory and other prior knowledge from concept to concrete system feature needs to be made more explicit so that researchers can examine how a theoretical construct becomes a concrete intervention. Design rationale can contribute by providing a structured, formal or semiformal boundary representation shared by those who carry out this kind of research. In this way, design rationale can help by standardizing the form in which design research studies are carried out, and how their results are described, explained, and used for prediction and discovery of new phenomena in interactive, complex systems research.
This work builds on prior analytic research identifying the potential role of design rationale as central to design research and design science. In particular, it provides empirical evidence for the multifaceted ways that design rationale concepts, representations, techniques, and tools might contribute to the emerging science of software-intensive systems design. Results of the study identify a number of contexts in which design rationale might contribute to a more rigorous, progressive, and communicable science of design. Much additional research, in particular, empirical research, is needed to understand the specific affordances required in design rationale representations and tools to realize these potentials.
ACKNOWLEDGMENT
The authors thank the National Science Foundation (Grant IIS-0639939) for supporting this research.
Steven R. Haynes is Assistant Professor of Information Sciences and Technology at Pennsylvania State University. He holds a PhD from the London School of Economics. Prior to undertaking his doctoral studies, Haynes worked at Apple Computer, Adobe Systems, and several start-up software companies in the United States and Europe. He has been involved in the development of commercial and custom software solutions as a programmer, designer, analyst, and project manager. He currently teaches software design and design research methods. Dr. Haynes' research interests include design research, design rationale, and explanations derived from designing.
Paula M. Bach graduated from the University of British Columbia with a BA in English language and psychology and from Michigan Technological University with an MS in rhetoric and technical communication. She is a PhD candidate in the College of Information Sciences and Technology at Penn State University. She has industry experience working as a technical writer and information architect. Her research interests include the science of design and the usability of open source software. Specifically, she is investigating how Microsoft integrates usability activities into CodePlex, its open source project hosting Web site.
John M. Carroll is Edward Frymoyer Chair Professor of Information Sciences and Technology at Pennsylvania State University. His research interests include methods and theory in human–computer interaction, particularly as applied to networking tools for collaborative learning and problem solving and design of interactive information systems. Dr. Carroll's books include Making Use and Usability Engineering. He received the Rigo Award and CHI Lifetime Achievement Award from ACM, the Silver Core Award from IFIP, and the Goldsmith Award from IEEE. He is a Fellow of the ACM, IEEE, and Human Factors and Ergonomics Society.
