Introduction

Productivity^{Footnote 1} growth is the key driver of longer-term improvements in living standards. Therefore, understanding what determines productivity growth is of key concern to economists and policymakers, and, ultimately, to the public at large. Complicating this understanding and the ability for policy to drive change are two things. First, the range of factors that influence productivity is very broad. At the economy-wide level productivity is influenced by firms’ decisions affecting the quality and quantity of inputs used (e.g. investments in tangible or intangible capital, innovation and workforce skills) and the efficiency with which production units are utilised (e.g. improvements in work organisation). It is influenced by the efficiency of resource allocation between firms, which in turn is determined by regulation and the competitive environment, as well as other factors that influence investment such as access to finance and economic uncertainty. It is influenced by learning and catch-up processes and the diffusion of best-practice across workers and businesses. Productivity is also dependent on wider institutional factors, such as the quality of transport and digital infrastructure, the quality of outputs from education and training systems, institutional supports for knowledge transfer, workers’ health and possibly even their happiness. Therefore, there is no single policy or simple set of policies that can drive productivity growth.

Second, and importantly, productivity analysis is often hampered by a lack of high quality data on production inputs and outputs. Labour productivity at a national level is simply measured as the ratio of GDP to the number of hours worked in the economy. There are of course many issues for concern in measuring GDP, recently brought to the fore in Reference BeanBean (2016), e.g. the difficulty of developing ‘real’ output measures when technological change is fast-moving and problems of accounting for national inputs and outputs in a globalised production system. But, understanding what drives average labour productivity also requires good data on production inputs and knowledge flows at different levels of aggregation (e.g. geographic, by economic activity, or by firm). Further muddying productivity measurement and the discourse at large is the separation in language used by economists and business, with the latter better recognising metrics such as growth in pre-tax profits, return on capital employed, sales growth versus planned growth, efficiency, defective parts per million and competitiveness. Thus, understanding of productivity performance and its determinants is intricately tied to economic measurement.

The study of productivity has a long history at the National institute of Economic and Social Research (NIESR), and has always been rooted in measurement and the development of new data. Particular areas of focus have been threefold: international comparisons of sector level productivity; analysis of the role of skills in determining productivity; and exploitation and development of business datasets for understanding productivity. In this paper we review the evidence on UK productivity performance and its determinants through NIESR's work in each of these areas. It is not our intention to provide a review of the productivity literature at large, but rather to review the lessons that have been learnt with contributions from NIESR colleagues past and present.

2. The UK's productivity gap: evidence from cross-country sector comparisons

At NIESR the measurement of internationally comparable productivity levels started with pioneering work by Reference RostasRostas (1948) in which he compared the UK and US, using quantities of outputs produced. This was followed by a number of studies in the 1980s (Reference Smith, Hitchens and DaviesSmith, Hitchens and Davies, 1982; Reference Davies and CavesDavies and Caves, 1987) and bilateral studies of manufacturing in the the 1990s (Reference Van ArkVan Ark, 1990a, Reference Van Ark1990b, Reference Van Ark1992; Reference O'MahonyO'Mahony, 1992) that used unit value ratios rather than quantities to compare productivity across industries. These studies highlighted long-standing problems of low productivity in the UK relative to major competitors. Most of the earlier work concentrated on manufacturing, for which unit value ratios could be constructed for well-defined products. A later set of studies focused on services sectors (Reference Smith and HitchensSmith and Hitchens, 1985; Reference O'Mahony, Oulton and VassO'Mahony, Oulton and Vass, 1998; Reference O'Mahony and OultonO'Mahony and Oulton, 2000) using a mix of unit values and OECD purchasing power parities (PPPs). Contrary to prior expectations, these studies also pointed to low labour productivity levels in services sectors in the UK relative to competitor nations such as France, Germany and the US.

Using data from bilateral studies, combined with time series from national accounts, Reference O'MahonyO'Mahony (1999) presented a complete picture of the UK's productivity gap for the total market economy^{Footnote 2} and by broad sector compared to France, Germany, Japan and the US, covering the period 1950–96. Table 1 shows the labour productivity levels for selected years from that study. It highlights that labour productivity levels in the UK did show some catch-up with those in the US but there remained a gap at the end of this period and the gap with both France and Germany increased throughout most of the postwar period. The UK had a labour productivity advantage only relative to Japan, and even there significant narrowing occurred over time.

Table 2 shows the labour productivity levels by broad sector in 1996. The UK's labour productivity gap was very large in manufacturing, in particular relative to the US, despite labour productivity growth in UK manufacturing having outstripped that in the US during 1979–94. Studying cross-country differences in manufacturing labour productivity over this period, Reference OultonOulton (1995) suggested that although supply-side reforms such as reductions in union power and a reduction in the share of low-skilled workers had boosted UK labour productivity growth, improvements were held back by the mismanagement of UK macroeconomic policy, with shorter boom periods and longer recessions than elsewhere. There are a few cases in table 2 where the UK had higher levels than Germany, notably in agriculture, mining, the utilities and construction, but only in agriculture did the UK have an advantage relative to France and in construction relative to the US. The striking message in table 2 is one of widespread weak UK labour productivity relative to these three countries. Japan outperformed the UK in manufacturing and the utilities.

In addition to labour productivity levels, Reference O'MahonyO'Mahony (1999) constructed internationally comparable measures of physical capital stocks and of labour force skills, although the latter were not available for Japan. This showed a large gap in the amount of physical capital used per worker in the UK relative to the US, France and Germany and a skills gap in the UK relative to the two European countries. After adjusting for capital and skills, the total factor productivity (TFP) gap was almost zero relative to France and Germany but there remained a large TFP gap relative to the US. Therefore the UK's lower levels of output per hour could be attributed in large part to poor investment in physical and human capital relative to France and Germany. However, other factors that affect TFP needed to be taken into account when seeking to explain the gap with the US. Reference Crafts and O'MahonyCrafts and O'Mahony (2001) suggested that greater shares of R&D expenditures in GDP explained a significant part of the US TFP advantage.

The UK's weak productivity attracted much attention in policy circles in the mid-1990s, but the following decade was one where the UK's fortunes improved markedly. This was the era when the productivity benefits from the large-scale adoption of information and communications technology (ICT) started to emerge in the aggregate statistics. Identifying this impact required internationally comparable data that allowed the separation of ICT from other forms of capital. Drawing on earlier work by Reference Jorgenson and StirohJorgenson and Stiroh (2000) among others, the European Commission-funded EU KLEMS project, in which NIESR was a major partner, developed an industry database that allowed comparisons of labour productivity and TFP growth for a much larger group of countries, including EU economies, Australia, Canada and Korea, as well as the US and Japan; see Reference O'Mahony and TimmerO'Mahony and Timmer (2009) for details of the construction of the database.

The focus of the EU KLEMS project was on contributions of knowledge inputs to output and labour productivity growth; see Reference Timmer, Inklaar, O'Mahony and van ArkTimmer et al. (2010) for an extensive discussion. Defining knowledge inputs as comprising the combined contributions of the use of highly skilled labour, ICT capital and TFP, two main findings emerge for productivity trends in the UK in the decade from 1995. The first was that the contributions to labour productivity growth from knowledge inputs was higher in the UK than in many European economies and close to that in the US, as illustrated in figure 1. Of the European countries studied, only Finland had higher contributions from knowledge inputs.

Figure 1. Labour productivity growth and contributions of knowledge inputs, annual average 1995–2005

The second important finding from EU KLEMS was to highlight the role of services activities in generating productivity growth. Traditionally, manufacturing activities have been regarded as the main source of productivity growth, benefiting from economies of scale and relatively high capital intensity and so seen as the locus of most innovations. Labour productivity improvements in services were seen as less likely than in goods-producing industries because most services are inherently labour-intensive, making it difficult to substitute capital for labour, and the long-term shift away from manufacturing towards services had reduced aggregate labour productivity growth. Following on from other papers that questioned if low productivity growth in services still held in the ICT era (e.g. Reference Jorgenson, Ho, Stiroh, Corrado, Haltiwanger and SichelJorgenson et al., 2005; Reference Triplett and BosworthTriplett and Bosworth, 2004), the EU KLEMS data allowed a more thorough analysis of productivity trends in services relative to goods production and how these compared across countries or regions. Comparing the EU15 with the US, the data showed much greater contributions of market services to aggregate labour productivity growth in the latter. A more in-depth analysis revealed that the labour productivity differences were particularly large in retail and wholesale trade and financial services.

Table 3 presents annual average growth rates of labour productivity over the period 1995–2005 for total market services and a division by broad sector. This shows performance in the UK was more akin to the US than either France or Germany. A similar picture emerges if TFP growth rates are compared. In general, the UK, being a services driven economy, benefited much more from the use of ICT in services than appeared to be the case for other EU countries. This enabled the UK to claw back some of the ground it had lost in previous periods relative to European economies, although the labour productivity gap widened slightly relative to the US.

Subsequent efforts to create harmonised cross-country data for productivity analysis further refined the concept of knowledge inputs. Building on the European Commission funded projects INNODRIVE and COINVEST, Reference Corrado, Haskel, Jona-Lasinio and IommiCorrado et al. (2012) developed estimates of the growth contributions of investments in intangible assets (as well as the contributions from tangible capital, labour force skills and TFP) for 14 EU countries and the US. Using the approach first developed by Reference Corrado, Hulten, Sichel, Corrado, Haltiwanger and SichelCorrado, Hulten and Sichel (2005), they estimated investments in computerised information (software and databases), innovative property (including R&D) and economic competencies (brand equity, training and organisational structures), making necessary adjustments to output. Comparing to the other countries shown in figure 1, these data suggested that UK labour productivity growth from 1995 to 2007 was supported by relatively strong contributions from labour force skills, TFP and intangibles.^{Footnote 3} Only in Austria, Finland and Sweden were the labour productivity contributions from TFP higher than in the UK. Labour productivity improvements from changes in labour force skills were better only in Spain. Only in Sweden and the US were the contributions of intangible capital deepening to labour productivity growth larger than in the UK, where contributions from intangibles were similar to Finland and France. These data also highlighted a key difference between the UK and other countries where growth was supported by significant investment in intangibles. In the UK, innovative property contributed relatively less to labour productivity growth, particularly in comparison to the US, thus confirming the suspicions of Reference Crafts and O'MahonyCrafts and O'Mahony (2001) from the decade before.

From a policy perspective it appeared by the mid-2000s that the UK was at least getting a grip on its productivity deficiency. However, all that was to change after the financial crisis, which saw reductions in productivity growth in most industrial economies, but especially so for the UK and other countries where productivity growth had been strong in the run up to the financial crisis. As shown in figure 2, the reduction in annual labour productivity growth during 2008–10 compared to 1999–2007 was greater than in the UK only in Sweden and Finland. With the exception of Finland, the 2011–15 labour productivity growth gap was largest in the UK, closely followed by the US. Most of the deterioration in labour productivity growth in these four countries was associated with a reduction in TFP growth, which remained subdued even as time moved on from the initial crisis years.

Figure 2. Annual labour productivity growth, 2008–10 and 2011–15: differences from 1999–2007

The sector breakdown of the collapse in productivity growth after the financial crisis revealed further commonalities and differences between the UK and other industrialised economies; see Reference Riley, Rincon-Aznar and SamekRiley, Rincon-Aznar and Samek (2018). In the UK, the US and the EU-15 the largest contributions to the slowdown in labour productivity growth came from the manufacturing, financial services and information and communication sectors. These were sectors that saw strong productivity gains before the financial crisis. Together these sectors accounted for around 70 per cent of the reduction in market sector labour productivity growth in the UK 2011–15 compared to 1999–2007, and close to all of the US and EU-15 gaps. The depth of the UK puzzle compared with other advanced economies was not easily explained by differences in industry structure across countries. During this period the larger fall in UK labour productivity growth compared to the EU-15 was due to widespread productivity weakness across sectors rather than UK-specific developments in particular sectors. Thus, UK productivity weakness after the Great Financial Crisis was exacerbated by UK specific macroeconomic developments. The labour productivity growth deficits in the UK and in the US during 2011–15 were similar in magnitude. Both countries exhibited strong labour productivity growth before the financial crisis and strong growth in hours worked during the period 2011 to 2015, mirrored by smaller contributions to labour productivity from capital deepening.

The slowdown in productivity growth in advanced economies since the financial crisis is thought by many to reflect a longer-term trend, pre-dating the crisis, and is a topic of significant international inquiry with explanations ranging from issues of measurement (Reference CoyleCoyle, 2018) to the dynamics of technology diffusion (Reference Brynjolfsson, Rock and SyversonBrynjolfsson et al., 2017).

3. The role of skills in explaining UK and cross-country differences in productivity

In the course of the international productivity comparisons carried out since the 1980s, cross-country differences in workforce skills were increasingly recognised as contributing to differences in national productivity performance. However, the extent and nature of this skills contribution remain hard to evaluate due to the many methodological issues that are still unresolved in this field. These concern in particular: (1) the measurement of skills, and (2) appropriate ways of modelling the potential channels of influence of skills on economic performance. Indeed, in the previous section, we saw a number of production inputs described as knowledge inputs, including workforce skills, ICTs, intangible assets and TFP. Amongst these the direct measured contribution of workforce skills to labour productivity growth and cross-country labour productivity differentials is relatively small, as is the direct measured contribution of R&D. This is a general result (Reference Timmer, Inklaar, O'Mahony and van ArkTimmer et al., 2010) that can be ascribed, at least partly, to the inability of growth accounting to represent correctly the contributions made by skills to productivity performance. This is because it cannot take account of complementarities between production inputs (such as the role of skills in supporting the effective use of new technologies) and other spillovers from skills (Reference Kirby and RileyKirby and Riley, 2008) and R&D.

NIESR efforts to address these issues over the years have their roots in research led by Professor Sig Prais during the 1970s, 80s and 90s which was summarised in two books: Productivity and Industrial Structure (1982) and Productivity, Education and Training (1995). This research programme combined comparative analysis of industry-level data in the UK, Germany and other countries with case-study establishment-level investigations in selected industries.

In terms of proxy measures of skill, this research parted company with the hitherto widespread reliance on measures based on years of completed schooling which captures educational attendance rather than attainment. Instead, extensive use was made of formal qualifications, measures which have the advantage of capturing something of what has actually been learned while undergoing education. At the same time great attention was paid to cross-country differences in the education and training institutions underlying differences in formal qualifications, for example, the higher incidence of apprenticeship training (combining employment-based training with part-time study in further education) in Germany compared to the UK.

In a series of comparisons of matched samples of establishments in the UK, Germany, France, and the Netherlands carried out between the early 1980s and mid-1990s, NIESR researchers and international colleagues highlighted several different channels of influence by which higher skill levels contributed to higher average levels of labour productivity. Examples included the greater ability of apprentice-trained German workers to suggest and implement ways in which machine utilisation and shopfloor efficiency in general could be improved and the extent to which senior managers and professional staff in British establishments were caught up in dealing with daily problems (‘fire-fighting’) because of the relative absence of intermediate-skilled (craft- and technician-level) workers to deal with those problems, or prevent them happening in the first place (Reference PraisPrais, 1995).

These studies covered a range of industries including precision engineering (Reference Daly, Hitchens and WagnerDaly, Hitchens and Wagner, 1985), furniture and clothing (Reference Steedman and WagnerSteedman and Wagner, 1987, Reference Steedman and Wagner1989), hotels (Reference Prais, Jarvis and WagnerPrais, Jarvis and Wagner, 1989) and food processing (Reference Mason, van Ark and WagnerMason, van Ark and Wagner, 1994). Their emphasis on intermediate skills contrasted distinctively at the time with a more common focus on high-level (university graduate) skills in cross-country skill comparisons.

At first sight the positive links between intermediate skills and average labour productivity levels in European countries seemed at odds with the fact that (then as now), the United States was notable for combining relatively high average labour productivity levels with relatively low levels of intermediate skills training. This issue was explored by extending intra-European comparisons in precision engineering and food processing to the US. These studies found that the classic ‘American model’ of semi-skilled workers supported by graduate engineers was well suited to the larger scale of production in US plants compared to their European counterparts. For example, in precision engineering plants, the larger batch sizes of components being produced in most US plants meant that less flexibility (between different machines and products) was required of shopfloor workers than was typically the case in Western Europe. In addition, graduate engineers in US plants substituted for scarce technician-level and supervisory skills in many ways (Reference Mason and FinegoldMason and Finegold, 1997).

Since the matched-sample comparisons were based on relatively small samples of plants in each country, they were open to criticism for being unable to shed any light on the relative importance of skills in explaining cross-country productivity differences as compared to other factors affecting productivity. In response to a critique of this kind by Reference DenisonDenison (1986), Reference PraisPrais (1988) argued that past attempts to evaluate the contribution of education and training to cross-country differences in real income levels through growth accounting methods had failed, partly due to the use of inadequate input measures of education (such as years of schooling) and partly due to the fact that such estimates typically left a large residual part of labour productivity differentials unaccounted for. The matched-plant comparisons were therefore justified, for all their acknowledged shortcomings, by their success in generating insights into the productivity effects of cross-country differences in educational attainments and the workings of different education and training institutions.

More recent work at NIESR has built on this legacy while making greater use of multivariate regression methods which are better suited than growth accounting to assessing the relative importance of skill differences compared to other influences on productivity performance. But in comparison with other researchers working in this area, many studies by NIESR and collaborating researchers remain distinctive for their focus on cross-country variation in the mix of intermediate- and high-level skills.

For example, Reference O'Mahony and van ArkO'Mahony and van Ark (2003) found that the ready availability of university graduates in the US over several decades helped the US in the late 1990s and early 2000s to outperform most European countries in terms of both productive applications of Information and Communication Technologies (ICTs) and in the estimated contribution of ICTs to growth in labour productivity. But a later study by Reference O'Mahony, Robinson and VecchiO'Mahony, Robinson and Vecchi (2008) reported some evidence that, as ICTs have become more established on both sides of the Atlantic, the ICT bias against lower-skilled workers has decreased over time in the US and UK while in France ICT-related demand for intermediate-skilled workers has increased relative to demand for high-skilled workers.

Subsequently, Reference Mason, Holland, Liadze, O'Mahony, Riley and Rincon-AznarMason et al. (2014) developed cross-country estimates of higher, intermediate vocational and intermediate general skills to study the impacts of the mix of skills on productivity at cross-country and -industry level. As shown in table 4, the graduate share of employment in the UK increased sharply between 2002–12 to reach near-parity with the US. But, at the end of this period, the UK still had a relatively small share of workers with intermediate vocational qualifications compared to European countries such as France and Germany. A number of policy initiatives have been introduced in the UK in an attempt to address this shortfall, most recently the introduction of the Apprentice Levy in 2017.

Incorporating intermediate vocational skills within an augmented production function framework alongside high-level and intermediate general skills, Reference Mason, Holland, Liadze, O'Mahony, Riley and Rincon-AznarMason et al. (2014) found that intermediate vocational skills contributed positively to cross-country and industry differences in productivity levels but only when vocational skills were broadly defined to include uncertified skills acquired through employer-provided training as well as certified vocational skills. The main mechanisms by which intermediate vocational skills contributed to productivity performance included facilitating use of ICTs and complementing the use of high-skilled workers.

In addition to the use of skill measures based on formal qualifications, efforts have been made to develop new skill measures which take account of skills acquired in the workplace that are not certified in any way (for example, skills deriving from employer-specific on-the-job training and informal learning through experience). For example, Reference O'MahonyO'Mahony (2012) derived cross-country estimates of training capital stocks using a perpetual inventory methodology borrowed from the intangible assets literature. This literature treats continuing training as an activity largely undertaken by firms who pay the direct costs of training programmes and also incur indirect costs in terms of production output foregone. Reference Kirby and RileyKirby and Riley (2007) considered the returns to experience accumulated on the job with a specific employer. Using the United Kingdom Labour Force Survey 1994–2001 they suggested that the rising ICT intensity of capital was associated with a rise in the return to schooling, and a reduction in the return to job-specific experience, which is likely to be less transferable across different jobs or technologies.

With regard to the role of uncertified skills, Reference Mason, O'Leary and VecchiMason, O'Leary and Vecchi (2012) developed a quality-adjusted measure of human capital in the UK, France, Germany, Netherlands and the US which combined data on formal qualifications at industry level with relative earnings data in order to capture differences in relative productivity between different skill groups. The underlying assumption is that these wage differentials reflect productivity differences arising from the possession of both certified and uncertified skills. They found a stronger role for human capital in explaining productivity differences at country-industry level when taking uncertified skills into account than when human capital was proxied by formal qualifications alone.

In spite of such findings, when examining the recent weakening of UK productivity performance relative to many other industrial nations, it is difficult to argue that this decline is primarily due to skill deficiencies, or that skill improvements on their own will ensure more rapid growth in productivity in the future. Indeed, growth accounting-based estimates suggest that labour productivity growth could have been even weaker in the UK in recent years had it not been for significant up-skilling of the workforce. Using skill measures based on formal qualifications, Reference Rincon-Aznar, Forth, Mason, O'Mahony and BerniniRincon-Aznar et al. (2015) estimated that, in the run-up to the 2008–9 financial crisis, growth in skills accounted for around 20 per cent of TFP growth in the UK. Between 2008–13 (that is, during and after the financial crisis), overall growth in labour productivity was negative on average – largely because of declining TFP growth – in spite of a continuing positive contribution from skills. However, later releases of data suggest that growth in labour quality slowed in more recent years during 2014–16.^{Footnote 4}

In this context, it is not surprising that much recent productivity research has focussed on the role of investment in a wider range of intangible assets, including skills but not confined to skills. In this Review, Reference Goodridge, Haskel and WallisGoodridge et al. (2013) argued that part of the slowdown in TFP growth following the financial crisis was accounted for by the slowdown in intangible and telecoms investment in the early 2000s. They also suggested that the omission of intangibles in National Accounts measures of real value-added growth meant that the fall in real value added growth during the recession was overstated, because intangible investment proved less cyclical than output more generally. Taking these two factors together they suggested that, during its first few years, around 30 per cent of the post-crisis productivity puzzle was accounted for by intangible investment.

4. Understanding UK productivity: lessons from business micro-data

In the past twenty years productivity research made much headway due to the increased availability and accessibility of business level micro data and linked business datasets. These allowed researchers in the UK and elsewhere to study and understand the drivers of productivity at the level of the firm. One of the key findings to emerge from analysis of these business datasets was that firms operating in similar industries and with similar measured inputs exhibited very different levels of productivity. This finding instigated the development of new models of heterogeneous productivity firms (Reference MelitzMelitz, 2003) and a barrage of research on the drivers of firm-level productivity (Reference SyversonSyverson, 2011). The availability of these datasets also facilitated study of the connections between productivity at the level of the firm and at more aggregate levels.

Early NIESR work on productivity using large-scale business microdata included a business survey undertaken in the early 1990s to study how firms were coping with the UK recession of 1991 (Reference Geroski and GreggGeroski and Gregg, 1997). Responses to a questionnaire concerning firms’ organisational, innovation and hiring strategies in the wake of recession, from more than 600 leading UK companies, were combined with long time-series data from Companies House on financial performance so that it was possible to trace effects left over from the UK recession in the early 1980s. These data suggested that firms that had performed particularly well, in terms of turnover growth, in the run-up to the 1990s recession, were most affected by the recession, performing particularly worse thereafter. Echoes of these patterns were seen in the aftermath of the financial crisis of 2007–8. Reference Riley, Rincon-Aznar and SamekRiley et al. (2018) suggested that after the financial crisis, growth collapsed most in many of those sectors that experienced particularly high productivity and turnover growth during the early 2000s.

Initially, the exploitation of representative business microdata to study the links between the dynamics of productivity at the level of the firm and more aggregate productivity developments focused on manufacturing. Reference OultonOulton (1987) investigated whether increases in UK manufacturing productivity during the early 1980s might be due to the closure of low-productivity plants. He concluded that the increase in large plant closures sparked by the recession in 1980 was not necessarily productivity enhancing. Many of the plants that closed had above average productivity, although below average profitability. Extending this type of analysis to consider business restructuring more broadly, Reference Disney, Haskel and HedenDisney et al. (2003) found that around half of sector level labour productivity growth between 1980 and 1992 occurred because of the external restructuring of firms rather than through productivity improvements within firms.

After the financial crisis of 2007–8 there was much concern in UK policy circles that the processes of resource reallocation between firms were no longer as productivity enhancing as they had been in the past, stilted by inefficiencies in the banking system, low interest rates and generalised uncertainty. Examining these issues using business surveys conducted by the UK Office for National Statistics (ONS), NIESR researchers and Bank of England colleagues decomposed market sector productivity growth into the effects arising from external restructuring (the entry of new firms, the exit of firms and shifts in market share between incumbents) and changes in productivity within firms (see figure 3). Comparing the post crisis period with the early 2000s and the 2008 recession with that of the early 1990s, these studies gauged the extent to which resource reallocation provided a drag on UK productivity growth. A key emphasis in these studies was to illustrate the importance of appropriate weighting systems for analysing the longitudinal elements of these business surveys and to highlight appropriate decomposition methods for analysing the aggregate productivity effects of business restructuring over the cycle. The results of these analyses suggested that the most striking feature of the collapse in UK market sector productivity growth after 2007 was the widespread collapse in TFP growth within firms, pointing to the effects of a common macro-economic shock (Reference Riley, Rosazza Bondibene and YoungRiley et al., 2015). However, whilst widespread changes within firms may have been the most striking feature in the data, the data examined in these studies also exhibited patterns that were consistent with inefficiencies in resource allocation across firms arising from problems in the banking sector.

Figure 3. Decomposition of 3-year labour productivity growth – UK market sector

NIESR was also very much involved in the development and analysis of successive waves of the Workplace Employment Relations Survey (WERS). The WERS was a series of surveys which aimed to provide a nationally representative account of the state of employment relations and working life inside British workplaces. Surveys were conducted in 1980, 1984, 1990, 1998, 2004 and 2011 and involved an establishment level survey alongside a survey of employees within these same establishments. NIESR researchers were involved in the last three of the WERS waves. The bulk of research supported by the WERS did not necessarily concern productivity, but inclusion of subjective performance questions in the establishment part of the survey and the feasibility of linking financial performance data to the survey from other sources meant that the WERS also supported productivity research; in particular, the links between productivity and employment relations. For example, using WERS 1998, Reference Forth and MillwardForth and Millward (2004) found a positive association between subjective measures of business performance (labour productivity, financial performance and product/service quality) and high-involvement management practices amongst UK establishments. Analysis of WERS2004 in Reference Riley, Metcalf and ForthRiley, Metcalf and Forth (2013) showed that the productivity and proitability improvements that could be achieved by establishments by implementing Equal Opportunities policies were not large and widespread, suggesting that one could not assume Equal Opportunities policies would be widely implemented without public intervention. Using WERS2011 and linked information on financial performance from the ONS business register, Reference Bryson and ForthBryson and Forth (2018) explored the impact of management practices on firm performance among UK small and medium size enterprises (SMEs) over the period 2011–14. They found that SMEs were less likely to use formal management practices than larger firms, but that such practices helped firms to increase their productivity. The returns were most apparent for SMEs that invested in human resource management practices and that set formal performance targets. Other NIESR work using Companies House data showed that increased labour costs from minimum wages, which might ‘shock’ managers into organisational improvements, could lead to improved productivity amongst firms where low pay was prevalent, without reducing employment (Reference Riley and Rosazza BondibeneRiley and Rosazza Bondibene, 2017).

The development of comparable cross-country business micro-data is still in its infancy, complicated in part by data access issues, but the rewards are potentially very significant. The NIESR was involved in a number of projects with international researchers that sought to build and exploit cross-country patterns in business data. The European Commission funded INNODRIVE project developed firm-level data on own account investments in intangible assets using linked employer-employee data for six countries; see e.g. Reference Riley and RobinsonRiley and Robinson (2011) for the UK. These measures were derived using information on the occupations of workers in the firm. Cross-country evidence based on these data suggested that organisational capital (e.g. management and brand equity) was important for productivity in all countries examined and that discrepancies between performance- and cost-based approaches to measurement further indicated the importance of organisational capital for productivity (Reference PiekkolaPiekkola et al., 2011; Reference PiekkolaPiekkola, 2016). Comparable analysis of the Community Innovation Surveys in the UK, Ireland and Germany, also revealed the importance of organisational innovations, including marketing, for productivity performance in service firms (Peters et al., 2018). In addition, this study emphasised the importance of foreign ownership for productivity in service firms, particularly for firms based in the UK and Ireland.

Over the past fifteen years a positive association between structured management practices and firms’ productivity was established in a series of studies by Bloom, Van Reenen and co-authors. Comparable cross-country firm-level data suggested that differences in management practices may help to explain the UK's lower productivity performance compared to some other countries (Reference Bloom, Sadun and Van ReenenBloom et al., 2017). The new large-scale UK Management and Expectations Survey, developed in collaboration between the Economic Statistics Centre of Excellence (ESCoE) at NIESR and the ONS, has already led to new insights into the relationship between productivity and management in the UK. This new survey is all the more powerful for being comparable to similar surveys in other countries. Seminal work by the OECD (see Reference Andrews, Criscuolo and GalAndrews et al., 2016) to develop cross-country business micro-data from company accounts drew attention to the increasing divergence between the performance of high- and low-productivity firms in many industrialised countries since the early 2000s, raising questions about the nature of technology diffusion and the distribution of the benefits of growth. In the UK this led to a focus on measures to improve the productivity of businesses that lag behind leading firms in the recent Business Productivity Review led by HM Treasury and the Department for Business, Energy and Industrial Strategy. Reference Riley and Rosazza BondibeneRiley and Rosazza Bondibene (2018) consider this divergence in the UK using linked employer-employee data.

5. Conclusions

The evidence reviewed in this paper has highlighted the importance of successive and coordinated initiatives to improve the measurement of production inputs and outputs at the firm, industry and national levels for generating understanding of UK productivity. While not an end in itself, it is clear that developments in measurement have been fundamental to insights into the ways that the interaction of firm choices with the institutional environment and the aggregate economy combine to determine productivity and living standards.

The development of internationally comparable sector level data on production outputs and basic inputs has shown that for many years productivity levels in the UK have been below that of its peers. This has partly been due to a lack of investment in different forms of capital (tangible and intangible) and deficiencies in the UK skills base. The UK was catching up with its peers during the late 1990s and early 2000s, but that process derailed after the financial crisis when productivity growth stagnated. More broadly, recent evidence suggests that any attempts to kick-start UK productivity growth should draw on a better understanding of the causes of the productivity slump that is affecting many other advanced economies, not just the UK. In part, this will require addressing key concerns around the measurement of real versus nominal output and the national location of value added in a global economy.

The evidence suggests that skill deficiencies and poor management are long-standing issues for UK productivity. Careful comparison and investigation of the links between skills and productivity in the UK and abroad have illustrated the complexities of these relationships, how they depend on the characteristics of education and training provision and interact with organisational structures and business models. These relationships are fundamental to the connection between labour markets, productivity and the distribution of productivity gains, and illustrate the importance of a skills agenda that focusses on vocational education and training as well as higher education and also supports continuing training for adult workers. Much can be learned through the analysis of linked administrative datasets on education participation and attainment and labour market progression, such as that being carried out by the Centre for Vocational Education Research, in which NIESR is a main partner.

The potential for gaining further insights into the drivers of productivity growth from business micro-data and the development of comparable micro-data across countries is very significant. Many lessons have already been learnt, for example, about business dynamics and aggregate productivity and about key correlates of productivity across firms. The potential for what we might learn from these data will be greatly enhanced through data linking efforts, e.g. as in Reference Wales, Black, Dolby and AwanoWales et al. (2018), and the continued development of longitudinal data, so that the knowledge base can extend to the causal drivers of productivity. Coordinated efforts to disentangle international ownership structures in cross-country business micro-datasets could yield important insights for national output and productivity measures. Finally, analysis of productivity at the level of the firm typically focuses on revenue productivity, conflating pricing behaviour and developments in efficiency. The potential of business micro-datasets to improve our understanding of the drivers of productivity would be much improved with the development of firm-level price measures.