Introduction
Open innovation (OI), as an emerging innovation paradigm, has gained increasing attention from scholars and practitioners of various disciplines and fields (Huizingh, Reference Huizingh2011). OI is defined as ‘the use of purposive inflows and outflows of knowledge to accelerate internal innovation and expand markets for external use of innovation, respectively’ (Chesbrough, Reference Chesbrough, Chesbrough, Vanhaverbeke and West2006b: 1). The traditional, closed innovation mode was characterized by the solid boundary of an organization, the exclusive reliance on internal research and development (R&D), the vertical integration of internal functions, and the internalization of the entire innovation process from materials acquisition, R&D, manufacturing, to the new product commercialization (Chesbrough, Reference Chesbrough2003b). This approach has become increasingly incompatible with the changing business environment characterized by the growing technological complexity and shortened product life cycle, which makes it difficult for firms to do all innovations on their own (Chandra, McManus, & Neelankavil, Reference Chandra, McManus and Neelankavil2008; Howells, James, & Malik, Reference Howells, James and Malik2003).
OI, by contrast, is characterized by the porous boundary of an organization which allows ideas, resources, and knowledge to flow across organizations (Chesbrough, Reference Chesbrough2003a, Reference Chesbrough2003b). Instead of investing heavily in internal R&D, firms source ideas and obtain knowledge/technology from the outside to develop new products/services (Chesbrough, Reference Chesbrough2003a, Reference Chesbrough2003b). They also seek ways to bring their in-house knowledge to market for superior returns through selling intellectual property and licensing out technologies, etc. (Chesbrough, Reference Chesbrough2003a, Reference Chesbrough2003b). As such, OI is expected to help firms overcome limitations in the closed, isolated, and traditional way of innovation, and gain a competitive advantage over rivals (Chesbrough, Reference Chesbrough, Chesbrough, Vanhaverbeke and West2006b; West & Bogers, Reference West and Bogers2014). This argument is supported by the empirical evidence that OI contributes to a firm's performance (Brink, Reference Brink2014; Chen, Zhao, & Wang, Reference Chen, Zhao and Wang2015; Cheng & Huizingh, Reference Cheng and Huizingh2014; Du, Leten, & Vanhaverbeke, Reference Du, Leten and Vanhaverbeke2014; Ho, Ruan, Hang, & Wong, Reference Ho, Ruan, Hang and Wong2016; Huang, Chung, & Lin, Reference Huang, Chung and Lin2009; Hung & Chou, Reference Hung and Chou2013; Wang, Chang, & Shen, Reference Wang, Chang and Shen2015). However, the adoption of OI is not without concerns as sharing a firm's valuable knowledge might entail difficulties in knowledge search, costs in knowledge absorption, and risks in knowledge misappropriation (Brunswicker & Chesbrough, Reference Brunswicker and Chesbrough2018), which may somewhat weaken the potential benefits OI could generate. Thus, a key question is: To what extent can a firm actually seize the benefits of OI to enhance its performance? (Moretti & Biancardi, Reference Moretti and Biancardi2020; West, Salter, Vanhaverbeke, & Chesbrough, Reference West, Salter, Vanhaverbeke and Chesbrough2014).
First, the answer to this question depends on how to define performance – whether it refers to the direct outcomes of innovation or the overall firm financial performance (Lopes & de Carvalho, Reference Lopes and de Carvalho2018). OI may have different impacts on the firm as a whole or the innovation processes alone. Second, the variance in OI approach, be it inbound (i.e., in-sourcing knowledge from the external environment) or outbound (i.e., revealing or selling knowledge/technology to the outside), may make a difference (Dahlander & Gann, Reference Dahlander and Gann2010; Huizingh, Reference Huizingh2011). Furthermore, whether OI is investigated at the firm level or sub-firm level may lead to different findings in relation to its impact on performance (West et al., Reference West, Salter, Vanhaverbeke and Chesbrough2014). In a word, OI does not necessarily have a universally positive impact on performance. The effectiveness of OI is largely contingent on performance measurement, OI approaches, and unit of analysis (Moretti & Biancardi, Reference Moretti and Biancardi2020; West et al., Reference West, Salter, Vanhaverbeke and Chesbrough2014).
Although there has been some research in the form of systematic literature reviews that attempted to identify and analyze the factors that influence the effectiveness of OI (Hossain & Kauranen, Reference Hossain and Kauranen2016; Hossain, Islam, Sayeed, & Kauranen, Reference Hossain, Islam, Sayeed and Kauranen2016; Kovacs, Van Looy, & Cassiman, Reference Kovacs, Van Looy and Cassiman2015; Natalicchio, Ardito, Savino, & Albino, Reference Natalicchio, Ardito, Savino and Albino2017; Randhawa, Wilden, & Hohberger, Reference Randhawa, Wilden and Hohberger2016; Schroll & Mild, Reference Schroll and Mild2012; West & Bogers, Reference West and Bogers2014), most of the research failed to compare the magnitude of the effects of OI across studies (Lipsey & Wilson, Reference Lipsey and Wilson2001), or to provide a generalized conclusion about the effects of the moderating factors on the relationship between OI and performance. Meta-analysis, instead, can establish empirical generalization of a specific relationship because it is built upon the quantitative synthesis of relevant research findings on the relationship across a large number of studies in the field (Geyskens, Krishnan, Steenkamp, & Cunha, Reference Geyskens, Krishnan, Steenkamp and Cunha2008). In this way, meta-analysis can more systematically and convincingly identify, test, and verify the roles of the key factors in moderating the relationship between OI and performance, and helps better explain the sources accounting for the heterogeneity of research findings regarding the effectiveness of OI (Geyskens et al., Reference Geyskens, Krishnan, Steenkamp and Cunha2008). To the best of our knowledge, our study is among the first meta-analysis research that has been conducted in the OI field. As such, our study makes unique contribution to the current understanding of when OI is more effective and in which situations OI leads to better performance.
The paper is organized as follows. We propose our hypotheses in the next section. We then explain the methodology including sample, measures, and the analytic procedure of our meta-analysis. We report the results relating to homogeneity analysis and meta-regression in the ‘Results’ section. Finally, we discuss the contribution, implication, and limitation of our study, and point to directions for future research.
Theoretical background and hypotheses
OI performance
Innovation was traditionally viewed as the processes of effective internal R&D through tight control of knowledge inflows and outflows (Chesbrough, Reference Chesbrough2003b). This closed mode of innovation poses challenges to firms competing in the dynamic business environment today where they find it increasingly difficult to always keep sufficient resources in-house, constantly equip themselves with relevant capabilities, maintain adequate investment in R&D at all times, and cope with the risks and costs of innovation on their own (Chandra, McManus, & Neelankavil, Reference Chandra, McManus and Neelankavil2008; Howells, James, & Malik, Reference Howells, James and Malik2003). By adopting an open approach to innovation, firms are able to address these challenges by capturing value from external resources and capabilities and/or leveraging their internally generated ideas through external partners to maximize returns (Chesbrough, Reference Chesbrough2003a). Over the past few decades, ‘substantial evidence has accumulated showing that open innovation can improve business performance’ (Chesbrough, Reference Chesbrough2017: 35).
Hunter and Nielsen (Reference Hunter and Nielsen2013) defined performance as ‘an organization's ability to achieve its goals and objectives measurably, reliably, and sustainably through intentional actions’ (p. 10). Performance measurement is the ‘regular measurement of the results (outcomes) and efficiency of services or programs’ (Hatry, Reference Hatry2006: 3). In the context of OI, Lopes and de Carvalho (Reference Lopes and de Carvalho2018) summarized the most frequently used performance measures in OI and divided them into two categories: innovation performance and firm performance. Innovation performance is related to the direct output of innovation activities as indicated by, for instance, the number of new or significantly improved products/services, the share of new products/services in sales, the number of licenses and patents, the degree of product/service novelty, and the speed of new product development. Firm performance is related to the overall performance of an organization (Lopes & de Carvalho, Reference Lopes and de Carvalho2018). Measures of firm performance include, for instance, sales growth (Caputo, Lamberti, Cammarano, & Michelino, Reference Caputo, Lamberti, Cammarano and Michelino2016), market share (Cheng & Huizingh, Reference Cheng and Huizingh2014; Sisodiya, Johnson, & Gregoire, Reference Sisodiya, Johnson and Gregoire2013), profitability (Cheng & Huizingh, Reference Cheng and Huizingh2014; Faems, de Visser, Andries, & Van Looy, Reference Faems, de Visser, Andries and Van Looy2010), financial indicators (Cassiman & Valentini, Reference Cassiman and Valentini2016; Cheng & Huizingh, Reference Cheng and Huizingh2014), customer feedback (Cheng & Huizingh, Reference Cheng and Huizingh2014), and turnover (Caputo et al., Reference Caputo, Lamberti, Cammarano and Michelino2016; de Zubielqui, Fryges, & Jones, Reference de Zubielqui, Fryges and Jones2019). The linkage between OI practice and its outcomes depends on whether it is the innovation performance or firm performance that is evaluated as the outcome indicator of OI (Moretti & Biancardi, Reference Moretti and Biancardi2020).
Empirical studies found that OI influences both innovation performance and firm performance (Andries & Faems, Reference Andries and Faems2013; Brink, Reference Brink2014; Cassiman & Valentini, Reference Cassiman and Valentini2016; Cheng & Huizingh, Reference Cheng and Huizingh2014; Chiang & Hung, Reference Chiang and Hung2010; de Zubielqui, Fryges, & Jones, Reference de Zubielqui, Fryges and Jones2019; Faems et al., Reference Faems, de Visser, Andries and Van Looy2010; Ho et al., Reference Ho, Ruan, Hang and Wong2016; Huang, Chung, & Lin, Reference Huang, Chung and Lin2009; Popa, Soto-Acosta, & Martinez-Conesa, Reference Popa, Soto-Acosta and Martinez-Conesa2017; Stam, Reference Stam2009). We conjecture that OI has a greater positive effect on innovation performance compared to firm performance. This is mainly because despite the potential benefits it could bring about, OI incurs high direct and indirect costs. Direct costs involve transaction costs arising from the payment for acquiring intellectual property (IP), accessing external information, sources, and contracting partners (Christensen, Olesen, & Kjaer, Reference Christensen, Olesen and Kjaer2005). Indirect costs result from knowledge search, networks coordination, and knowledge transfer between various partners (Christensen, Olesen, & Kjaer, Reference Christensen, Olesen and Kjaer2005). The value and benefit an organization would obtain from OI are directly reflected in innovation output in the form of new products/services and patents. In contrast, firm performance is influenced by not only the benefits OI generates but also the costs OI incurs. Keeping a proper balance between the value-enhancing and cost-increasing effects of OI is a challenging task (Faems et al., Reference Faems, de Visser, Andries and Van Looy2010). Therefore, the extent to which OI improves firm performance may not be as large and evident as the extent to which it improves innovation performance.
Hypothesis 1: OI has a greater positive effect on innovation performance than firm performance.
OI approach
The complexity in the relationship between OI and performance also arises from the different approaches to openness, namely inbound (outside-in) and outbound (inside-out) OI (Chesbrough & Crowther, Reference Chesbrough and Crowther2006). Inbound OI is the process of in-sourcing knowledge from the external environment and integrating it into an organization's internal knowledge base to supplement its internal R&D (Chesbrough & Crowther, Reference Chesbrough and Crowther2006). With inbound OI, organizations do not need to rely solely on their internal R&D as what they used to do in the old, closed innovation mode (Lopes & de Carvalho, Reference Lopes and de Carvalho2018). According to Dahlander and Gann (Reference Dahlander and Gann2010), there are two main activities of inbound OI – ‘sourcing’ which refers to scanning, accessing, and utilizing important knowledge from external sources including customers, suppliers, competitors, universities, and research institutes; and ‘acquiring’ which refers to using various collaborative or contractual arrangements such as inter-firm collaboration, technology purchase, and licensing-in to acquire valuable knowledge from the outside (Dahlander & Gann, Reference Dahlander and Gann2010).
Outbound OI is the process of searching for mechanisms to transfer internally generated innovations to the outside stakeholders (Dahlander & Gann, Reference Dahlander and Gann2010). Such internally generated innovations include not only new product designs, but also unused technology, by-products of R&D, and redundant inventions (Chesbrough & Crowther, Reference Chesbrough and Crowther2006; Lichtenthaler, Reference Lichtenthaler2015). With outbound OI, organizations attempt to purposively exploit the value of internally generated knowledge through selling and revealing, which is also in stark contrast to the closed innovation mindset (Dahlander & Gann, Reference Dahlander and Gann2010). ‘Selling’ refers to selling intellectual property and licensing-out technology developed via in-house R&D while ‘revealing’ involves voluntarily revealing know-how and information to the external environment in exchange for useful feedback knowledge at limited transaction costs (West, Reference West, Chesbrough, Vanhaverbeke and West2006; West & Gallagher, Reference West, Gallagher, Chesbrough, Vanhaverbeke and West2006). In addition to the inbound and outbound OI approaches, Enkel, Gassmann, and Chesbrough (Reference Enkel, Gassmann and Chesbrough2009) further defined the ‘coupled’ OI approach which represents a combination of the inbound and outbound OI practices.
OI scholars have mainly focused on the impact of inbound OI, leaving outbound OI and coupled OI an underrepresented area in the literature (Chesbrough, Reference Chesbrough2017; Lichtenthaler, Reference Lichtenthaler2015). Chesbrough and Crowther (Reference Chesbrough and Crowther2006) pointed out that inbound and outbound OI are generally intertwined since each inbound activity in an organization is essentially generated by a reciprocal outbound activity in some other organizations. In theory, both of them are integral to the overall OI framework. The success of an OI system depends on the balance between ‘take’ (through inbound OI) and ‘give’ (through outbound OI). In practice, however, organizations are relatively more willing to ‘take’ and harness external ideas instead of ‘giving’ their own knowledge as the latter poses more challenges in knowledge protection to maintain core competitive advantages of the organizations (Huang, Rice, Galvin, & Martin, Reference Huang, Rice, Galvin and Martin2014). Empirical research has found a generally positive OI–performance relationship in both inbound and outbound approaches (Andries & Faems, Reference Andries and Faems2013; Arvanitis, Fuchs, & Woerter, Reference Arvanitis, Fuchs and Woerter2015; Cassiman & Valentini, Reference Cassiman and Valentini2016; Chen, Zhou, Probert, & Su, Reference Chen, Zhou, Probert and Su2017; Cheng & Huizingh, Reference Cheng and Huizingh2014; Gesing, Antons, Piening, Rese, & Salge, Reference Gesing, Antons, Piening, Rese and Salge2015; Roper, Vahter, & Love, Reference Roper, Vahter and Love2013).
We postulate that inbound OI is likely to generate a stronger performance impact than outbound OI, particularly to the whole organization (Huizingh, Reference Huizingh2011; West & Bogers, Reference West and Bogers2014). The use of external sources and knowledge inflows through inbound OI, such as collaboration and technology purchase, could strengthen and enhance the organization's knowledge base which in turn lead to increased organizational performance. However, the benefit of outbound OI to the organization may be more difficult to realize due to the high level of appropriation hazards. As noted earlier, outbound OI encourages licensing-out of internally developed technology and selling IP in order to commercialize innovation outcomes or revealing internal ideas in order to get some useful feedback knowledge in return (Dahlander & Gann, Reference Dahlander and Gann2010). All these activities require a largely free knowledge outflow to enable knowledge sharing with external parties, and involve knowledge spillovers. Knowledge spillovers were traditionally regarded as inadvertent outbound flows or unintended by-products of innovation in the closed innovation model (Chesbrough, Reference Chesbrough2006a). Contrary to conventional wisdom, outbound OI emphasizes voluntary knowledge spillovers to optimize the benefits from openness (Schmidt, Reference Schmidt2006). Unfortunately, this does increase the risks of losing distinctiveness of an organization's overall knowledge base, undermining its core competitive advantage (Huang et al., Reference Huang, Rice, Galvin and Martin2014; Mazzola, Bruccoleri, & Perrone, Reference Mazzola, Bruccoleri and Perrone2012). With outbound OI, it could be difficult to find the right balance between sharing useful knowledge to capture external opportunities and protecting the core knowledge to maintain competitive advantage (West & Gallagher, Reference West, Gallagher, Chesbrough, Vanhaverbeke and West2006). Accordingly, the non-inbound OI (outbound including coupled OI) approach may not as effectively contribute to performance of the organization as the inbound OI approach does.
Hypothesis 2: Inbound OI has a greater positive effect on performance than non-inbound OI.
Level of analysis
According to Bogers et al. (Reference Bogers, Zobel, Afuah, Almirall, Brunswicker, Dahlander and Ter Wal2017), OI is also a multi-level phenomenon, involving different levels of analysis. The two most frequently used levels of analysis in previous studies are the firm level and the lower project/team level (Bogers et al., Reference Bogers, Zobel, Afuah, Almirall, Brunswicker, Dahlander and Ter Wal2017; Lopes & de Carvalho, Reference Lopes and de Carvalho2018). That is, OI can be undertaken at the firm level but also at the level of projects, teams, and business units in an organization (West, Vanhaverbeke, & Chesbrough, Reference West, Vanhaverbeke, Chesbrough, Chesbrough, Vanhaverbeke and West2006). In this paper, we group all projects, teams, and business units into the lower, sub-firm level of analysis to differentiate it from the firm-level of analysis. Previous research reported a generally positive OI–performance relationship at both firm level and the sub-firm level (Bahemia, Sillince, & Vanhaverbeke, Reference Bahemia, Sillince and Vanhaverbeke2018; Brockman, Khurana, & Zhong, Reference Brockman, Khurana and Zhong2018; Caputo, Pizzi, Pellegrini, & Dabić, Reference Caputo, Pizzi, Pellegrini and Dabić2021; Kobarg, Stumpf-Wollersheim, & Welpe, Reference Kobarg, Stumpf-Wollersheim and Welpe2019; Lichtenthaler & Lichtenthaler, Reference Lichtenthaler and Lichtenthaler2009; Shi & Zhang, Reference Shi and Zhang2018; Vanhaverbeke, Du, Leten, & Aalders, Reference Vanhaverbeke, Du, Leten, Aalders, Chesbrough, Vanhaverbeke and West2014; Villasalero, Reference Villasalero2018; Walsh, Lee, & Nagaoka, Reference Walsh, Lee and Nagaoka2016).
We posit that OI is likely to bring about more evident benefits to the organization when undertaken at the firm level than at the sub-firm level. Although individual projects, if successful, might be associated with more direct innovation returns, OI implemented only at the sub-firm level may incur a higher failure rate. In a single project, a team or business unit usually has little capacity and room to integrate different innovation choices (Lichtenthaler, Reference Lichtenthaler2011), including the choice of ‘make’ or ‘buy’ for inbound OI which refer to developing the technology in-house or buying-in the technology respectively, and the choice of ‘keep’ or ‘sell’ for outbound OI which refer to keeping R&D outputs in-house or selling them in the market respectively (Chesbrough, Reference Chesbrough2003b). As a result, project teams/business units would either rely too much on the external knowledge which they cannot sufficiently absorb or remain too closed so that they are not able to benefit from the external knowledge at all. OI, once adopted, tends to become a substitute for internal R&D in this situation.
By contrast, OI at the firm level can be viewed as the aggregation of all lower-level (i.e., project, team, and business unit) OI activities, where the failure risks associated with a single OI activity could be effectively offset by the potential benefits from other OI projects, teams, and business units in the organization (Vanhaverbeke et al., Reference Vanhaverbeke, Du, Leten, Aalders, Chesbrough, Vanhaverbeke and West2014). Meanwhile, the overall organizational competitiveness can benefit from the synergies of OI activities in multiple projects, teams, and business units. OI at the firm level is more likely to be complementary to internal R&D activities, and can therefore generate better innovation outcomes (Chesbrough & Crowther, Reference Chesbrough and Crowther2006). This complementarity can be reflected in a simultaneous choice of both ‘make’ and ‘buy’ (namely, doing some R&D projects in-house while acquiring complementary technologies from the outside); and a simultaneous choice of both ‘keep’ and ‘sell’ (namely, keeping some core R&D outputs in-house and selling the by-products or redundant ones to the outside) (Lichtenthaler & Lichtenthaler, Reference Lichtenthaler and Lichtenthaler2009). The complementary relationship between internal and external R&D activities could be better achieved, thus more evidently observed, at the firm level wherein managers have more resources and capacities to coordinate and more room to leverage internal and external R&D investments than managers at the sub-firm level (Chesbrough & Crowther, Reference Chesbrough and Crowther2006).
Hypothesis 3: The positive relationship between OI and performance is greater at the firm level than at the sub-firm level.
Methodology
Our research is among the first OI-related studies that used meta-analysis. Stanley, Doucouliagos, and Steel (Reference Stanley, Doucouliagos and Steel2018: 709) noted that ‘meta-analysis systematically reviews all comparable research literature on a specific topic of interest and employs statistical methods to aggregate the information from independent studies.’ We chose meta-analysis methods for two reasons. First, conventional narrative reviews of the literature may fail to unveil the interconnections among individual studies (Dabić, Maley, Dana, Novak, Pellegrini, & Caputo, Reference Dabić, Maley, Dana, Novak, Pellegrini and Caputo2020; Palumbo, Manesh, Pellegrini, Caputo, & Flamini, Reference Palumbo, Manesh, Pellegrini, Caputo and Flamini2021). Meta-analysis represents a specialized subset of systematic reviews which could address this limitation by employing comprehensive search strategies to collect all available evidence and systematically analyzing the spectrum of the phenomenon (Dabić et al., Reference Dabić, Maley, Dana, Novak, Pellegrini and Caputo2020; Palumbo et al., Reference Palumbo, Manesh, Pellegrini, Caputo and Flamini2021). Furthermore, meta-analysis uses rigorous statistical approaches to further synthesize the data derived from the systematic review to produce generalizable conclusions on a specific topic (Geyskens et al., Reference Geyskens, Krishnan, Steenkamp and Cunha2008; Lipsey & Wilson, Reference Lipsey and Wilson2001).
Meta-analysis has been employed in various innovation-related domains such as the success factors of product innovation (Evanschitzky, Eisend, Calantone, & Jiang, Reference Evanschitzky, Eisend, Calantone and Jiang2012), and the success factors of service innovation (Storey, Cankurtaran, Papastathopoulou, & Hultink, Reference Storey, Cankurtaran, Papastathopoulou and Hultink2016). It has been also used to explain the moderators that influence the relationship between organizational size and innovation (Camisón-Zornoza, Lapiedra-Alcamí, Segarra-Ciprés, & Boronat-Navarro, Reference Camisón-Zornoza, Lapiedra-Alcamí, Segarra-Ciprés and Boronat-Navarro2004), between creativity and innovation (Sarooghi, Libaers, & Burkemper, Reference Sarooghi, Libaers and Burkemper2015), and between general innovation and performance in small and medium-sized enterprises (SMEs) (Rosenbusch, Brinckmann, & Bausch, Reference Rosenbusch, Brinckmann and Bausch2011). However, none of the previous studies have focused on OI – this specific emerging mode of innovation. Our research is thus believed to make valuable contribution by addressing this gap in the literature.
Meta-analysis sample
We conducted the literature search in the main databases EBSCOhost Business Source Complete, ProQuest, Econlit, Jstor, ScienceDirect, Scopus, and Web of Science. We searched with the keywords ‘open innovation’ AND ‘performance’ in either title or abstract. The initial search was limited to peer-reviewed academic journal articles written in English, published from year 2003 (when the term ‘open innovation’ was coined) until year 2018. We were aware that some books and book chapters are essential as well, particularly the seminal studies of Chesbrough and his co-authors. We thus included them in our search list.
The initial search with the above criteria produced around 1,600 publications. We screened all publication titles to exclude duplicates and then read through abstracts to further discard irrelevant ones. We then applied three key inclusion criteria to generate the final meta-analysis sample. First, studies should have reported the measurement of both OI and performance. Second, studies should have reported the regression results on the relationship between OI and performance at the firm or sub-firm level. Third, studies should have provided the correlations between OI and performance. When correlations were not reported in the published studies, we contacted the authors to request for their correlation matrices. We excluded articles for which we could not reach the authors or this information was no longer accessible. After applying the above selection criteria, our final sample consisted of 2,377,123 firms or sub-firm units from 171 studies. There are 782 correlations (k = 782) since some studies included multiple OI measures, performance measures, and samples or sub-samples. Readers can contact the corresponding author for availability of our meta-analysis dataset.
Moderators
The description of variables is presented in Table 1. As for performance measure, we constructed a dichotomous variable coded as ‘1’ if the effect size represented the impact of OI on innovation performance, and ‘0’ if the effect size represented the impact of OI on overall firm performance. As for OI approach, given the much less amount of sample studies on outbound OI and coupled OI (Lopes & de Carvalho, Reference Lopes and de Carvalho2018), these two were grouped together into non-inbound OI as a comparison to inbound OI. This classification is consistent with Mazzola, Bruccoleri, and Perrone (Reference Mazzola, Bruccoleri and Perrone2012) and Battistella, De Toni, and Pessot (Reference Battistella, De Toni and Pessot2017). We constructed a dichotomous variable with ‘1’ denoting inbound OI and ‘0’ denoting non-inbound OI. As regards level of analysis, we constructed a dichotomous variable coded as ‘1’ if OI was investigated at the firm level, and ‘0’ if OI was examined at the sub-firm level including project, team, and business unit level.
Table 1. Variable descriptions
Control variables
There were several variables which may influence the OI–performance relationship as found in previous empirical studies, so they should be controlled for in our meta-analysis. Year of sample was coded as ‘1’ if the sample in the study was collected before 2007 (including the year 2007) and ‘0’ if the sample in the study was collected after 2007. Year 2007 is the average value of the year of data collection in the studies under analysis. Country of origin was coded as ‘1’ if the sample used in the study was from developing economies (or transition economies) and ‘0’ if the sample was from developed economies according to United Nations' definitions (2018). OI dummy was coded as ‘1’ if OI was measured by a binary variable in the study and ‘0’ if it was measured by a continuous variable. Quality of study was coded as ‘1’ if the study was published in an A or A*-ranked journal on the ABDC (Australian Business Deans Council) journal list and ‘0’ otherwise. Data source was coded as ‘1’ if the study used primary data and ‘0’ if the study used secondary data. Industrial affiliation was represented in three categories – manufacturing, service, and mixed. Mixed also included samples which did not clearly specify industrial affiliation. We then constructed two dichotomous variables, manufacturing and service, to distinguish between the three categories of industries. The first dichotomous variable manufacturing was coded as ‘1’ if the sample in the study was in a manufacturing industry and ‘0’ otherwise. The second dichotomous variable service was coded as ‘1’ if the sample in the study was in a service industry and ‘0’ otherwise.
Meta-analytic procedure
Pearson's correlation coefficients (r) were collected from each study to represent the effect size. Some meta-analytic studies followed Peterson and Brown's (Reference Peterson and Brown2005) recommendation, to replace missing correlation coefficients with standardized regression coefficients (Beugelsdijk, Kostova, Kunst, Spadafora, & van Essen, Reference Beugelsdijk, Kostova, Kunst, Spadafora and van Essen2018). However, recent research argued that results from studies using the regression coefficients were linked to large biases when the mean population correlations were calculated and even larger biases when the standard deviations were calculated (Roth, Le, Oh, Van Iddekinge, & Bobko, Reference Roth, Le, Oh, Van Iddekinge and Bobko2018). Thus, it was suggested to use only correlation coefficients in meta-analysis in order to avoid biased results.
We employed the random-effect model in meta-analysis. The choice between fixed-effect and random-effect models is dependent on research assumptions and goals. According to Borenstein, Hedges, Higgins, and Rothstein (Reference Borenstein, Hedges, Higgins and Rothstein2009), the fixed-effect model is used when the primary studies are assumed to share a common effect and when the goal is to compute the common effect size for the identified population. In contrast, the random-effect model is chosen when the true effect sizes in the primary studies are assumed to be different and when the goal is to generate a range of scenarios. The random-effect model is an appropriate choice for our study.
To carry out meta-analysis, we created several pairs of subgroups for hypothesized moderators and control variables, and calculated the mean effect size for each of them. The descriptive statistics and correlation coefficients of the variables are presented in Table 2. In order to test heterogeneity, we transformed r into Fisher's z. We used Fisher's z to compute the within-group Q-statistic, the between-group Q-statistic, the estimated variance of the true effect sizes (T 2), and the scale-free index of heterogeneity (I 2). We conducted both homogeneity analysis and meta-regression analysis to test our hypotheses.
Table 2. Correlation matrix
**p < .01, *p < .05.
Results
Homogeneity analysis
Table 3 presents the results of the homogeneity analysis. As shown in row 1, the mean effect size was .17 with 95% confidence intervals of .16 and .18. The mean effect size and confidence interval statistics were all greater than 0, which indicated a generally positive relationship between OI and performance.
Table 3. Homogeneity analysis
Note: k = number of effect sizes; N = number of sample sizes; $\bar{r}$
= weighted mean effect size; Q (within sub-groups) = homogeneity test statistic within each subgroup; I 2 = scale-free index of heterogeneity; T 2 = estimated variance of the true effect sizes; Q (between sub-groups) = Q-statistic to compare the mean effect across subgroups.
***p < .01, **p < .05.
According to Borenstein et al. (Reference Borenstein, Hedges, Higgins and Rothstein2009), the estimated variance of the true effect sizes (T 2) reflects the amount of true heterogeneity while the scale-free index of heterogeneity (I 2) reflects the proportion of observed dispersion that is due to this heterogeneity. As shown in row 1, T 2 was .018 (T = .135), indicating that most of the effect sizes (95%) fell in the wide range of approximately −.09 to .43. Similarly, I 2 was 98.2%, indicating a high heterogeneity in the sample studies. The Q-statistic 43,366.46 (df = 781) was statistically significant (p < .01), indicating that the sample effect sizes were heterogeneous and that there was the presence of moderators in the sample population. The results supported our assumption that moderating variables influence the relationship between OI and performance.
Moreover, as shown in rows 2–21, the between-group Q-statistics were highly significant for all pairs of the subgroups, indicating that the effect sizes were heterogeneous between the subgroups in each pair. However, despite a very useful approach to identifying potential moderators, the homogeneity test alone is not adequate when several pairs of subgroups exist simultaneously. Therefore, we further supplemented it with meta-regression analysis in which the effects of each pair of subgroups are examined when the effects of other pairs of subgroups are controlled for.
Meta-regression analysis
Table 4 presents the meta-regression analysis results. We first included control variables (model 1) and the three moderators one by one in addition to control variables (model 2, 3, and 4). As shown in model 2, the coefficient of performance measure was negative and significant (β = −.055, p < .05), contradicting proposed hypothesis 1 about the greater effect of OI on innovation performance than firm performance. Thus, hypothesis 1 was rejected. The coefficient of OI approach shown in model 3 was positive and statistically significant (β = .065, p < .01). The results supported hypothesis 2, indicating that inbound OI had a greater positive effect on performance than non-inbound OI. As shown in model 4, the coefficient of level of analysis was positive and statistically significant (β = .234, p < .01). The results supported hypothesis 3, indicating that OI had a greater positive effect on performance at the firm level than at the sub-firm level.
Table 4. Meta-regression results
***p < .01, **p < .05, *p < .1.
We then pooled all the three moderators together in addition to control variables, and re-ran the meta-regression. As shown in model 5, the coefficient of performance measure remained negative and statistically significant (β = −.076, p < .01). The results further confirmed the rejection of hypothesis 1, indicating that OI actually had a greater positive effect on firm performance than on innovation performance. This finding contradicted our theoretical prediction, and a possible explanation is provided in the ‘Discussion’ section. In addition, the coefficient of OI approach remained positive and statistically significant (β = .083, p < .01), as did the coefficient of level of analysis (β = .261, p < .01). The results further supported hypotheses 2 and 3.
As for the effects of control variables, although the coefficients of OI dummy, quality of study, and data source were insignificant, the coefficients of year of sample, country of origin, and industrial affiliation (manufacturing and service) were significant at the level of 5%. The results indicated that OI had a greater positive effect in the sample data collected before 2007; OI had a greater positive effect in developing-country samples than in developed-country samples and OI had a greater effect in manufacturing industry samples and service industry samples than in mixed industry samples.
Discussion
Theoretical contribution
Based on a comprehensive dataset of 2,377,123 firms and sub-firm units in 171 studies from 2003 to 2018, our research demonstrates that although OI generally has a positive effect on performance as indicated by the average effect size of .17, the OI–performance relationship is significantly moderated by three key factors: performance measure, OI approach and level of analysis. To the best of our knowledge, our research is among the first to aggregate existing empirical evidence, comprehensively examine these moderating factors, and statistically test the significance of each of them in influencing OI–performance relationship using meta-analysis technique.
Our research differs from previous studies which mainly used literature review narratives to identify what were believed to be important factors accounting for the effectiveness of OI. As OI is a multifaceted phenomenon (Lopes & de Carvalho, Reference Lopes and de Carvalho2018), these studies only partially captured the importance of some factors under certain conditions, but failed to provide generalizable findings across various research contexts. Our research provides such generalization by applying the meta-analysis technique to existing empirical evidence available at the time of the study, and thereby makes a valuable contribution to the current understanding of when and how OI maximizes performance.
We found that OI generates better overall benefits to the whole organization in the inbound approach compared to in the non-inbound approach and at the firm level compared to at the sub-firm level. Importantly, contrary to conventional wisdom, OI contributes more to overall firm performance than innovation performance. We think that this is largely because most of the frequently used measures of innovation performance in previous studies were related to the output of product innovation, such as the number of new products, the share of new product sales and the number of patents. However, the role of OI in promoting other innovation types such as process innovation and marketing innovation was not fully captured by these measures, but was reflected in the overall firm-level outcomes (Huang & Rice, Reference Huang and Rice2012). It is imperative to develop more accurate measures of innovation performance that take into account the increasing application of OI in process innovation and service innovation (Chesbrough, Reference Chesbrough2011).
Our research thus sheds fresh light on the different, and sometimes even contradictory, findings on the effectiveness of OI from previous studies. The heterogeneous results were likely due to the variation of these studies in (1) whether they focused on the innovation output or the overall firm outcomes; (2) whether they focused on inbound or non-inbound OI; and (3) whether they focused on OI at the firm-level or at the sub-firm level. In this sense, therefore, our research opens the ‘black box’ of the OI–performance relationship that has been puzzling scholars and practitioners in this field.
Practical implication
Our research suggests that the benefits of OI are actually more evidently observed in a firm's overall performance than in its direct innovation output. Although the effectiveness of OI might be reflected in new or significantly improved products, patents, and sales of new products, the value of OI might be also embedded in the new/improved processes and commercialization channels which, although do not directly contribute to the product innovation, help improve the overall performance of the firm. The benefits of OI might also include improving know-how and organizational capabilities through firms' attempts to search for new knowledge from external sources, absorbing exogenous knowledge and applying it in business operations. In this sense, the effectiveness of OI should be evaluated from a holistic perspective. In the same vein, OI should be implemented as organizational level changes to drive the overall firm growth instead of ad-hoc R&D initiatives that may only boost innovation output in the short run.
Moreover, it is generally easier for organizations to garner benefits from inbound OI than from outbound or coupled OI. Although outbound OI through selling IP and licensing-out technology developed in-house might be associated with more direct economic returns, the balance between knowledge disclosure and knowledge protection is of great challenge when undertaking this OI approach (Belderbos, Cassiman, Faems, Leten, & Van Looy, Reference Belderbos, Cassiman, Faems, Leten and Van Looy2014; Foege, Lauritzen, Tietze, & Salge, Reference Foege, Lauritzen, Tietze and Salge2019; Laursen & Salter, Reference Laursen and Salter2014; Stefan & Bengtsson, Reference Stefan and Bengtsson2017; Zobel, Lokshin, & Hagedoorn, Reference Zobel, Lokshin and Hagedoorn2017). In taking the opportunities brought by the outbound or coupled OI, organizations should be aware of the potential risk of losing resource-stock rarity due to intended or unintended knowledge spillovers to the external environment, and should take effective measures to minimize appropriation hazards which might, in turn, undermine the organization's overall competitiveness (Huang et al., Reference Huang, Rice, Galvin and Martin2014).
Furthermore, consistent with the findings discussed earlier, OI is better considered as an organization-wide strategy rather than an ad-hoc solution to individual R&D projects. Although individual, sub-firm level projects might be more directly associated with innovation output, integrating lower-level OI projects into the organization's overarching business model helps mitigate the adverse consequences of failing in a single OI project, and enables the organization as a whole to benefit from the synergetic and complementary effects of multiple OI activities (Chesbrough, Reference Chesbrough2007; West & Bogers, Reference West and Bogers2014). In addition, integrating product innovation with process innovation, marketing innovation, and innovation in other parts of the value chain helps maximize gains from OI to enhance the overall organizational competitiveness.
Limitations and directions for future research
Meta-analysis is based on the information provided by the sample studies. Therefore, all errors, bias, and flaws embedded in the original studies in our sample pool were inevitably transmitted to our meta-analysis (Borenstein et al., Reference Borenstein, Hedges, Higgins and Rothstein2009). Therefore, the results of our meta-analysis should be interpreted with caution. Moreover, our data pool is limited to the primary studies which provided correlation coefficients. Studies that did not provide this information were excluded, which might influence the findings. It should be also noted that although we have controlled for the effects of most factors that may possibly influence the OI–performance relationship, some variables such as firm age and size were not included because a large number of studies in our sample pool did not report this information.
Our research suggests some interesting and promising directions for future research. To begin with, future research could look at whether and how other firm-level factors such as firm size and firm age may contribute to the heterogeneous empirical findings regarding the effectiveness of OI when data become available. It is interesting to examine, for example, how OI may affect firm performance differently in large firms which have more resources and SMEs which have fewer resources. It is also interesting to examine whether there are any differences in the effect of OI on performance between young firms which are more entrepreneurial and the established ones which are more experienced. The inclusion of firm size and firm age may help detect some hidden factors that influence the OI–performance relationship.
Moreover, although the firm and sub-firm levels of analysis are examined in our research, regional, and national levels of analysis are not included in the research due to the small number of relevant studies in our sample pool. It is interesting to investigate, for example, whether formal institutions such as political, economic, and legal systems at the national or regional level may affect the effectiveness of OI in improving business performance, or whether informal institutions such as values, culture, and norms in different societies may have an impact on the OI–performance relationship. The inclusion of regional and national levels of analysis may help deepen our understanding of the conditions under which OI works.
Conclusion
Using meta-analysis method, our research systematically examines three key moderators that affect the relationship between OI and performance. Our research finds an effect size of .17, indicating that OI generally enhances performance. Furthermore, the OI–performance relationship is found to be dependent on whether the performance is measured in terms of innovation output or overall firm outcomes, whether OI is inbound or outbound (including coupled), and whether OI is undertaken at the firm level or the sub-firm level. Our research helps explain the heterogeneous findings on the relationship between OI and performance in existing studies, and provides important implications to scholars and practitioners in this field.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/jmo.2021.38
Financial support
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Conflict of interest
The authors declared that there is no conflict of interest.
