We decide not on the basis of what we know but on the basis of what we do not know.
Bayless Manning to Albert Hirschman, August 29, 1967Footnote 1
I. INTRODUCTION
For an organization such as the World Bank, with its commitment to development finance, project evaluation is an essential activity. Yet, this function was not conceptualized by the Bank until the 1960s, and it was put into practice only in the early 1970s, more than twenty-five years after the Bank was created. In 1964, after twenty years of operations, the Bank carried out the first in-depth evaluation of its own projects; up to that point, it had already made 480 loans to eighty-five countries, totaling almost $10 billion, in a broad spectrum of sectors including industry, agriculture and fishing, electric power, and education.Footnote 2 The late emergence of appraisal techniques was not limited to the Bank alone, but was characteristic also of other multilateral organizations, such as the United Nations Development Programme (UNDP), the United Nations Industrial Development Organization (UNIDO), and the Organization for Economic Cooperation and Development (OECD). If we consider the eminently “applied” nature of a field like development economics, this late emergence is remarkable. Once the need for developing an evaluation culture within the development field was established, however, the research on the subject thrived, involving some among the brightest minds in the field, such as Partha Dasgupta, Albert Hirschman, Ian Little, Stephen Marglin, James Mirrlees, and Amartya Sen. Soon, different approaches to evaluation techniques and different views on what the evaluation function of an organization should accomplish became visible.
As these few hints suggest, the birth of evaluation between the late 1960s and the early 1970s implied more than a discussion on different techniques of project appraisal. In fact, it is a perfect perspective with which to look at the institutional changes that affected the development field after its first, foundational phase, and it sheds light on the ways in which global development institutions participated in the intellectual debate as part of their effort to foster effective development policies in less developed countries. The case of the World Bank is particularly interesting, as it was the stage of perhaps the first attempt at developing a systematic reflection on how to embed an evaluation function in the policy-making processes of a development organization. In addition, the debates and contrasts kindled by this attempt show the intellectual cleavages that divided scholars on the ultimate purposes of project appraisal. This article focuses on this episode and the debates that it originated. Another article of mine, forthcoming, will start from here and focus on the later evolution of the evaluation function at the Bank (Alacevich forthcoming). Far from being a matter only internal to the World Bank, however, the discussion on project evaluation that took place in the 1960s is important for the broader history of development theory and institutions: it marked a shift in the policy strategies of the World Bank, the most important multilateral player in the development field, as well as in the theoretical landscape of development economics at large. This article, thus, is a reflection on the theory and practice of development economics in the 1960s and 1970s, following my focus on the 1950s in a previous article in this journal (Alacevich 2011).
Up until the 1960s, the theoretical landscape had been characterized by debates about macro-approaches to development strategies, such as “balanced growth” vs. “unbalanced growth,” and project appraisal and evaluation had remained a somewhat marginal topic in the larger field of development economics. The World Bank was accustomed to making ex ante evaluations of the “soundness” of projects—‘soundness’ being the buzzword of those times—and was satisfied as long as its loans were repaid by the recipient countries. Of course, the usually prolonged relations between the Bank and client countries made it natural for the Bank’s officers to consider and discuss previous experiences, and a few evaluational studies of specific Bank projects had been promoted by the Bank in the early 1960s. Yet, no systematic evaluation of Bank-financed projects or assessment of their success (or lack thereof) had been put into place. The Bank had no independent evaluation function in its organizational chart, and its first project evaluations were implemented by operational staff.
When Albert Hirschman (1915-2012)—a prominent economist at Columbia University and a founding father of development economics in the 1950s—proposed joining forces with the Bank to study some general elements of project appraisal, he was venturing into almost completely unexplored territory. Hirschman showed the centrality of project appraisal for the design and implementation of effective development projects, thereby becoming a major contributor, as we will see, to the shift in development economics from broad development theories to more focused analyses of dynamics at the project level. His groundbreaking volume, Development Projects Observed (1967a), emerged from his study of a series of World Bank projects in the mid-1960s, and served as the first attempt to establish project evaluation as a standard practice in the development field (Picciotto Reference Picciotto, Rodwin and Schön1994; Willoughby Reference Willoughby, Grasso, Wasty and Weaving2003). Yet, his emphasis on the intrinsic uncertainty of knowledge derived from project evaluation, as well as his reluctance to make his reflections more “operational,” complicated his relationship with the Bank. The Hirschman study, for which Bank staff and management had such high expectations, soon disappointed most of them. Furthermore, the Bank all but ignored the final outcome of his project, the 1967 book, and especially disliked its first chapter, “The Principle of the Hiding Hand,” which was widely circulated after appearing as a separate article in Daniel Bell’s and Irving Kristol’s journal The Public Interest (Hirschman Reference Hirschman1967a and Reference Hirschman1967b). In particular, Hirschman’s insistence on uncertainty as a structural element in the decision-making process did not fit in well with the operational drive of Bank economists and engineers.
This article is based on archival material from the World Bank Archives and the Albert Hirschman papers held at Princeton University. It will tell the story of the genesis, development, and conclusion of Hirschman’s collaboration with the World Bank, and discuss its relevance for the post-war history of development.Footnote 3 The study of these dynamics allows for a better understanding of how an institution thinks, how it produces and uses knowledge, and how it positions itself in the international market of development knowledge. As Judith Tendler has masterfully explained, strong tensions arise among the intrinsically uncertain nature of the knowledge acquired on development issues, the limits of knowledge transmission between donors and domestic recipients, and the status of “development experts” with which individuals and whole institutions present themselves on the development stage. Similar considerations to those discussed in this article could be applied to the bilateral US Agency for International Development and the multilateral United Nations Development Programme (Tendler Reference Tendler1975; Murphy Reference Murphy2006). This discussion becomes particularly relevant due to the increasing importance that knowledge production, accumulation, and transmission have acquired in the development world. Since 1996, the World Bank has defined itself as a “knowledge bank”; that is, a bank transferring to developing countries not just financial resources, but also “knowledge assets” (for a recent reaffirmation of this, see World Bank 2011). There is, thus, a direct link between today’s reflections on how to shape efficient multilateral development institutions and those early discussions on what kind of knowledge is needed to design and implement more effective development projects.
II. ALBERT HIRSCHMAN AND THE 1960s TURN IN DEVELOPMENT ECONOMICS
Development economics as an autonomous field of study took shape at the end of World War II as a result of the demise of colonial empires, the birth of many new, less-developed countries in Asia and Africa, and the industrialization efforts of Latin American countries. Early development theories were characterized by a high degree of abstraction; they discussed general sets of typical “conditions” of underdevelopment and “obstacles” to development, and were translated into broad approaches to policy and often comprehensive economic planning. Paul Krugman defined the first fifteen years of the new discipline as the “glory days of high development theory” (1994, p. 40; see also Krugman Reference Krugman, Summers and Shah1992). Those years were opened by a 1943 article by Paul Rosenstein-Rodan making the case for big-push industrialization and balanced growth, and ended with Albert Hirschman’s The Strategy of Economic Development (1958), which radically criticized the balanced approach, proposing instead an “unbalanced” strategy for economic growth, in the same vein as Paul Streeten (Reference Streeten1959). In Krugman’s view, the publication of Albert Hirschman’s book in 1958 marked the end of that early period of confrontation between grand development theories, and coincided with a methodological crisis in the discipline that would effectively make development economics a marginal field of economic research. While Krugman’s thesis—which highlights the lack of formal modeling on the part of development economists as the main reason for their marginalization—is questionable (see, for example, Stiglitz Reference Stiglitz, Summers and Shah1992), it is nonetheless true that, between the late 1950s and the mid-1960s, the nature of analysis and debate within development economics was changing in profound ways. The years of broad, overarching development theories were giving way to more detailed, country- and time-specific analyses.
Development economics had by then moved beyond its foundational phase, which was characterized by heavy, systemic discussions and mutually delegitimizing general theories vying for hegemony within the discipline. Global and all-embracing theories, though still alive and well after this period, came no longer to represent the most advanced reflections in development studies.Footnote 4 A new wave in development economics brought increased complexity to the analytical landscape. Broad generalizations were considered increasingly inadequate to address country-specific bottlenecks; therefore, much more specific and targeted analyses were needed to tackle underdevelopment successfully. Furthermore, the centrality of per capita income as an indicator of development began to wane as additional factors were taken into consideration, such as nutrition, public health, education, and housing (Hirschman Reference Hirschman and Hirschman1981).
Albert Hirschman was among the inspirational leading figures for this new season of development economics. In keeping with his role as a dissenter in the “old generation,” Hirschman was among the first to form new hypotheses to frame and test the process of development and the causes of economic and social backwardness. Following in the footsteps of Alexander Gerschenkron’s studies in economic history, which had persuasively shown that the process of economic growth had not followed the same pattern in different countries, but had unfolded for each country in a different way (Gerschenkron Reference Gerschenkron1962, though his most important papers collected there date back to 1952 and 1957), Hirschman became increasingly aware that thorough historical reconstruction was the only way to unravel the mechanisms of development. He explicitly addressed this issue in his foreword to Judith Tendler’s book on the economic and political consequences of technological choices in Brazil’s electricity sector:
Underdevelopment having been diagnosed as something so multifaceted, tangled, and deep-rooted, it was often concluded that the situation called for revolution, massive redistribution of wealth and power from the rich to the poor countries, or at least coordinated attack on pervasive backwardness through highly competent central planning.
But what if none of these dei ex machina are available to take matters properly in hand? What if the fortress of underdevelopment, just because it is so formidable, can not be conquered by frontal assault? In that unfortunately quite common case, we need to know much more about ways in which the fortress can be surrounded, weakened by infiltration or subversion, and eventually taken by similar indirect tactics and processes. And I suggest that the major contribution to our knowledge of economic development must now come from detailed studies of such processes. (Hirschman Reference Hirschman1968a, pp. vii–viii)
Judith Tendler’s book, the final output of PhD research conducted under Hirschman’s supervision at Columbia University, was among several to pioneer this new approach.Footnote 5 The book discussed how hydro vs. thermal power generation and distribution affected the Brazilian economic development between the 1950s and the 1960s. She asked:
should the work therefore be properly identified as a narrow case study, with no pretension of relevance to more general questions of economic development? The answer … is in the negative, for this study is meant to illustrate the general thesis that technologies vary as to their political vulnerability, their ability to draw out and train competent talent, and their capacity to brook the coexistence of politically antagonistic institutions. The lesson of this particular case is not that hydro should be favored in developing countries, but that the technological configuration of a project, program, or economic activity is a valuable source of information in the study of opportunities for economic development. (Tendler Reference Tendler1968, p. 6; for an earlier discussion of this question, see Tendler Reference Tendler1965)
Following in Hirschman’s footsteps, Tendler developed an historical analysis of a sector’s development, taking into consideration technological choices and opportunities, economic incentives, and political behaviors.
Meaningful examples can be also drawn from Hirschman’s work. Two-thirds of his 1963 volume, Journeys Toward Progress, is occupied by a detailed historical analysis of how three specific economic policy problems in three different countries had evolved over a long period of time. Hirschman examined the questions of drought in northeast Brazil, land reform in Colombia, and inflation in Chile. As he put it, “the essence of this volume is in the flow of the three stories” (Hirschman Reference Hirschman1963, p. 1). To understand the problems faced by economic policy makers in Latin America, Hirschman felt that “the best method of looking for answers was to scrutinize the record of a few specific, documented, protracted, significant policy problems” (Hirschman Reference Hirschman1963, p. 2). This propensity for historically grounded analyses would remain an important feature of Hirschman’s approach. As he suggested at a 1980 conference:
Following in detail the process of a revolution gives us a strong feeling, as the structuralist approach does not, for the many might-have-beens of history.… As a result, the event-minded historian is less likely than the sociologist to declare that, given such a structural condition, the outcome was preordained. [This] emphasis on the revolutionary process … in effect promises to restore a few degrees of freedom we were in danger of losing to the structuralists. (Hirschman Reference Hirschman and Hirschman1980, pp. 171–172)Footnote 6
The research that resulted in Hirschman’s Reference Hirschman1967 book, Development Projects Observed, followed the same approach. Although the book does not contain the stories of the individual projects analyzed, the research had an “intensive concern with ‘cases’” and the projects considered had “an extended history.” “Immersion in the particular,” Hirschman concluded, “proved … essential for the catching of anything general” (Hirschman Reference Hirschman1967a, p. 3).
This tension between the particular and the general was an answer to the widespread disorientation, among development practitioners and scholars, regarding the state of knowledge about development. After almost twenty years of development assistance, the results were inconsistent, and early post-war development economics was increasingly considered wanting as an effective analytical tool and a basis for aid policies. Perhaps even more fundamentally, the record of what had been done, and with what results, remained unclear. In an attempt to gain a closer understanding of the mechanisms of development processes, general and unifying theories had given way to a much more fragmented body of knowledge, but the promise of better insights had been coupled with the feeling of a lack of vision. A 1964 Brookings Institution report on foreign aid summarized this unresolved tension:
Aid is being extended, bilaterally and multilaterally, to 100 countries, democratic and authoritarian, allied and neutral, progressing and retrogressing, without satisfactory standards for evaluating competing claims, promoting particular strategies, harmonizing aid with other available instruments of policy, or appraising the results achieved. There is need for a better political, moral and economic framework within which to review our foreign aid programs and to prepare recommendations regarding their future.Footnote 7
The collaboration between Albert Hirschman and the World Bank in the mid-1960s grew out of a similar sense of frustration at the lack of a framework for the evaluation of the work done and the establishment of more coherent and effective policies.
III. ALBERT HIRSCHMAN AND THE WORLD BANK
In the spring of 1963, Albert Hirschman made a proposal to Burke Knapp, vice-president of the World Bank, for an analysis of specific projects financed by the World Bank in developing countries, which would be conducted through an in-depth, worldwide field investigation.Footnote 8 His original idea was to look at Bank-financed projects in order to test the hypotheses elaborated in his two previous studies on development issues, The Strategy of Economic Development (1958) and Journeys Toward Progress (1963). He planned to conduct extensive on-the-ground investigations, paying attention “to the upkeep and performance of the project itself, to the economic activities that it has stimulated (or destroyed?), and very much also to the wider economic, social and political ramifications of the project such as its educational effects and its contribution to the formation of new local or national elites.”Footnote 9 An additional purpose of Hirschman’s, after two books heavily based on Latin American cases, was to expand his expertise in development issues by becoming acquainted with Asia and Africa.Footnote 10 From the early months of 1964, Hirschman’s study was framed within a larger collaboration between the Brookings Institution and the World Bank; more precisely, it became a Brookings Institution initiative endorsed by the Bank.Footnote 11 In addition to the involvement of Brookings, the Carnegie Corporation conceded a grant to the Brookings Institution to finance Hirschman’s travel expenses, and the Ford Foundation awarded Hirschman a Faculty Research Fellowship for 1964–65.Footnote 12
At that time, the Brookings Institution was setting up a wide-ranging program of studies on American foreign assistance; the quotation above comes from that program. The research program was ambitious: it aimed at building, in a period of five years, a general evaluation of post-war American foreign assistance, preparing supplementary studies on a variety of aspects of foreign aid, and reinforcing the role of the Brookings Institution as the leading think-tank on the relations between foreign aid and the economic, social, and political conditions of developing countries.Footnote 13 The role of the World Bank was, of course, fundamental in this research program: the first history of the World Bank was to be published in 1973 by Brookings (Mason and Asher 1973).Footnote 14 Another equally fundamental part of the research was a detailed study of the impact of selected World Bank projects on the social and economic structure of client countries; that is, the study originally proposed by Albert Hirschman.
The principal purpose of Hirschman’s study was, in his words, “to explore in detail the direct effects as well as the broad repercussions of a project on economy and society,” as well as to reach “some improvements in the process of project evaluation and selection.”Footnote 15 The aim of the study was to gain a deep understanding of the salient features of a project, both at the planning stage and after its completion. The fact that Hirschman chose to analyze projects financed by the World Bank was not unexpected; the Bank’s twenty-year-long experience constituted “the most ample, varied, and detailed source of information and documentation in this area” (Hirschman Reference Hirschman1967a, p. 1). Also, none of the selected projects had developed without encountering serious problems. Therefore, this sample—albeit limited in number—could prove useful to the goal of building generalizations about the problems arising in project lending. More fundamentally, as Hirschman later put it, a comparison of the different projects would highlight similarities and differences in project experience, so as to inquire whether they could “be traced to what … may be called their ‘structural characteristics’” (Hirschman Reference Hirschman1967a, p. 4). These structural characteristics included economic, technological, administrative, and organizational features, and were closely connected to the broader socio-political environment.
An early draft of Hirschman’s proposal was criticized by those at the Bank who thought it was oriented “too much towards ‘side’ effects and too little towards the economic analysis of the Bank-financed projects.”Footnote 16 In a revised version, Hirschman identified three areas of focus for questions and analysis of project effectiveness. The first was the decision process in the recipient country. Who took the initiative for the project? What political and interest groups were for or against it? Did any of these groups shift their initial position about the advisability of the project? A second focus area was the decision process within the Bank. Here, too, Hirschman underscored the importance of comparing the Bank’s initial positions with ex post considerations. Thus, one question would be whether the original arguments for or against the loan, including qualitative arguments, would stand up even after the project’s implementation; or how wise the Bank’s choice would seem in the light of the consequences of implementing the project. Hirschman was particularly interested in understanding whether the Bank, in its decision-making processes, could reach correct conclusions despite analytical flaws; more generally, as will be discussed in section VI below, his goal was to shape a method by which project planners could navigate the intrinsic uncertainties of a project’s performance and predict probable lines of behavior. A third focus area was the project in retrospect. Relevant questions included a review of the effects of the construction phase (from a description of unexpected events, to the direct and indirect capacity-building effect on local engineering, managerial, and planning capabilities) and the project’s ultimate impact on the society and economy of the recipient country.Footnote 17 Of course, Hirschman’s curiosity dwelt on the linkage effects that could originate from the construction or the operation of the projects. The concept of “backward” and “forward” linkage had been introduced by Hirschman in 1958, and was at the center of his view of growth as a fundamentally unbalanced process. According to Hirschman, certain economic activities might create the conditions for establishing new activities for which there were previously no incentives. Thus, through these linkage effects, these new activities would, in fact, function as substitutes for entrepreneurial skills and as more efficient and pragmatic undertakings than central planning efforts, thereby fostering economic growth. Hirschman, thus, wondered whether “construction (as distinct from operation) had given rise to economic activities which have since then continued and expanded,” and “what linkage effects to other kinds of economic activity have resulted from the operation of the project.”Footnote 18 Finally, Hirschman recommended analyzing even the most far-reaching consequences, such as the effects of projects on the distribution of wealth, income, and power in the recipient country, the development of new entrepreneurial and administrative capabilities, in both the public and private sectors, and the introduction of “fundamental structural changes.”Footnote 19
Candidate projects for inclusion in Hirschman’s study had to meet three basic criteria: they had to be diverse in sector and geographical area; they had to have been operational for several years, in order to enable a reconstruction of their historical evolution—detailed historical analysis, as we have seen, was central in Hirschman’s approach; and, finally, they had to be identifiable activities that required continued maintenance and brought about identifiable linkage effects. Hydroelectric projects, specific highways, or industries (preferably basic ones) were the natural candidates, as opposed to loans for general highway reconstruction and maintenance activities, for the purchase of agricultural machinery, or for balance of payments purposes (Hirschman Reference Hirschman1967a, p. 3). With the help of Bank officers in the Area Departments, the Technical Operations Department, and the Economic Department, Hirschman selected a list of projects.Footnote 20
Feedback from Bank officers was positive. Dragoslav Avramovic—a senior economist, soon to become Director of Special Economic Studies—remarked that “probably for the first time, the contemporary theory and practice of project appraisal in infra-structure will be subjected to a systematic ex-post methodological scrutiny on a wide basis.”Footnote 21 Because of these high expectations, the Bank became remarkably involved in the research, especially considering the fact that it was conducted by an external researcher. Robert Asher, a major officer at the Brookings Institution who worked with the Bank on the Hirschman study, noted that “it is an extraordinary thing for the Bank to open its project files to an outside researcher, to provide an assistant for the outside researcher, to request the collaboration of its member governments, and to do the other things the Bank has agreed in this case to do.”Footnote 22 Beyond the good reception of Hirschman’s proposal, however, one might particularly note those comments that, while acknowledging the proposal’s usefulness, attempted to expand the borders of inquiry. Some Bank officers argued that undertakings more complex than specific projects should be taken into consideration. The case was made for including the activities of local development banks and regional development agencies among the projects under investigation. These institutions were new in most developing countries, and their impact on the development of a country was not easy to weigh; still, according to many at the Bank, they should not be disregarded. As Einar Sekse put it, “the economic, financial and social effects of the activity of the development banks concerned are, mainly, a composite of the effects of the projects these banks have financed. This may make a study as proposed of a development bank much more complex than that of some other projects.... However, … such a study … will be most interesting and valuable.”Footnote 23 Eventually, the Industrial Credit & Investment Corporation of India (ICICI), the Damodar Valley Corporation, and the Cassa per il Mezzogiorno for the development of southern Italy were included in Hirschman’s travel schedule.
Other voices in the Bank questioned the representativeness of the small and heterogeneous sample that Hirschman intended to examine: “one wonders,” wrote an officer, “whether in relation to the wide variety of projects and the widely differing economic and political backgrounds of the Bank’s and IFC’s member countries, such a small sample as a dozen projects out of about 300 will be sufficient to bring out valid general conclusions.”Footnote 24 Along similar lines, the economist Robert F. Skillings, at the Bank since 1947, proposed focusing on a specific sector (e.g., power or roads), and implementing a thorough analysis of all the projects financed by the Bank in that particular sector.Footnote 25 This second suggestion was not followed at that time, but it was not forgotten by the Bank: among the first studies sponsored in the early 1970s by the newborn Bank’s Operation Evaluation Division was a thorough analysis of the Bank’s accomplishments in the power sector (World Bank 1972).
IV. THE GENESIS OF THE BANK’S EVALUATION DEPARTMENT
The coming-together of the Bank and Hirschman, under the umbrella of the Brookings Institution’s research, was a timely joint venture. Like several other institutions, the World Bank was increasingly realizing that it was necessary to establish and refine methods for evaluating the impact of its policies on less-developed countries.Footnote 26 Although, as we will see, Hirschman’s attempt did not produce the expected results, the questions that prompted his work remained very much alive. Those early reflections were progressively institutionalized within the Bank in an evaluation function, embodied first in a Programming & Budgeting Department (from 1970) and subsequently in an Operations Evaluation Department (from 1973).Footnote 27 The most ambitious outcome of the initial phase of evaluation were two reports that, in line with Skillings’ recommendations, presented a full analysis of the Bank’s activity in one specific sector: electricity, and in one specific country: Colombia (World Bank 1972a and 1972b).Footnote 28 In subsequent years, the operations evaluation function struggled, not always successfully, to define and strengthen its role within the Bank.
A history of the evaluation function at the World Bank has yet to be written. The few available recollections usually focus on organizational or governance matters, in particular on establishing an arm’s-length relationship with operational staff and on how to strengthen the independence of evaluators from the Bank’s top management, so as to report in an unbiased way to the executive directors.Footnote 29 As for the contents and methods, in the late 1960s, a Bank publication described the Bank’s approach to project appraisal as all-embracing: a project would be examined “in terms of what it can be expected to contribute to the over-all development of the country” (King, Jr. 1967, p. 5). Yet, the six different aspects that were considered fundamental in project appraisal (economic, technical, managerial, organizational, commercial, and financial) belonged to the economic or business spheres. This was understandable in light of the fact that, up until the mid-1960s, more than two-thirds of the money lent by the Bank went to electricity and transportation projects. Less than a decade later, the scope of project appraisal—in all international agencies, but especially within the World Bank—had significantly expanded to encompass “not only financial and technical viability [of the project], but also broader and less easily quantifiable ‘developmental impacts’: sectoral, spatial, and environmental effects.… All projects are generally subjected to some form of cost-benefit analysis incorporating economic and social—qualitative as well as quantitative—variables” (Rondinelli Reference Rondinelli1976, p. 585).
Early exercises in project evaluation were based on a thick description of the cases under examination, often written in a narrative style. “In the old days,” remembered Robert Picciotto, who later became the head of the Operations Evaluation Department, “OED would write a book about the history of the Bank’s relationships with [a] country. It was a beautiful history, actually, a very solid piece of work” (Picciotto Reference Picciotto2000, p. 16). King’s and Tendler’s reports testify to a tradition of consistently accurate and detailed reconstructions of facts and dynamics (King, Jr. 1967; Tendler Reference Tendler1982 and Reference Tendler1993).
Thus, in the mid-1960s, the World Bank and Albert Hirschman, along with several other major players in the development field, were in agreement that it was essential to focus on specific case studies in order to derive insights into the mechanisms that governed development projects. It was a shift away from broad, comprehensive strategies as the main methodological focus of development economics, and the World Bank was actively involved in this shift. Dani Rodrik recently emphasized, among the major features of the so-called new development economics, its being diagnostic rather than presumptive, experimental (in the broad sense of “a predisposition to find out what works through policy innovation”), based on monitoring and evaluation, aimed at selective and targeted reforms, suspicious of universal recipes and best practices, and, instead, focused on policy reforms (Rodrik Reference Rodrik2008, pp. 27–28). These were also characteristics of Hirschman’s analysis of development projects and policies. During the 1980s and early 1990s, and until the disavowal of the mid-1990s, the World Bank often applied a universalistic, one-size-fits-all approach, especially with its emphasis on macro policies (World Bank 2005). It is, thus, interesting to note that well before, in the early 1960s, the Bank was agreeing with Hirschman on positions that are in tune with today’s most articulate analyses of development economics. At that time, although it was perhaps a rather experimental stage, nobody in the business of development doubted the importance of building development policies based on thorough case diagnostics.
V. DIVERGING VIEWS ON PROJECT APPRAISAL
Albert Hirschman and his wife Sarah spent one year between July 1964 and August 1965 traveling to four continents (see Table 1). Sarah was not simply an accompanying spouse but, as often in Albert Hirschman’s work, a partner in field research and an important intellectual interlocutor. Her contribution was fundamental in the late 1950s, enabling Albert Hirschman to expand his 1958 book into the sort of anthropological literature relevant to development studies. Likewise, she participated in field research during the summer of 1960 in Mexico, Colombia, Chile, Argentina, and Brazil that would serve as the basis for Hirschman’s Reference Hirschman1963 book, Journeys Toward Progress.Footnote 30 In 1964–65 Sarah was again “a member of the ‘team’,” and this explains why several field notes we will be using in the following pages are authored by her.Footnote 31
Source: Albert O. Hirschman, “A Study of Selected World Bank Projects—Some Interim Observations,” August 1965, World Bank Hirschman Folders, Vol. 1.
Note: Italy was added to the list only as a second thought, after the advice of several World Bank officials who maintained that the Bank’s loans to the Italian Cassa per il Mezzogiorno—the Italian agency for the development of southern Italy—would be very relevant to Hirschman’s study; see Douglas J. Fontein to Prof. Gabriele Pescatore, April 9, 1965, World Bank Hirschman Folders, Vol. 1.
Once returned from his travels, Albert Hirschman circulated a memo with some preliminary observations.Footnote 32 These focused on what Hirschman called “Behavioral Characteristics of Development Projects in Different Sectors.”Footnote 33 As he put it: “having learnt in fairly rapid succession about a wide variety of projects, I became alerted to the characteristic advantages or handicaps under which power projects, say, proceed as compared to irrigation projects.”Footnote 34
The principal aim of Hirschman’s interim observations was clearly methodological. Far from addressing questions such as the economic return of the Bank’s loans, or the traditional distinctions of, say, infrastructure vs. agricultural and industrial projects, or human vs. physical capital, Hirschman emphasized the need for a change in perspective. What really affects projects is their degree of uncertainty: “the element of the unknown, the uncertain and the unexpected which deflects projects from the originally chartered course is considerable in all projects. But it is far more important in some projects than in others and it may be of interest to the Bank to gain an approximate idea about the principal determinants of this uncertainty.”Footnote 35 Among these determinants, Hirschman listed the ability to completely map out the project upon its launch (for example, electricity projects can more easily be mapped out than agricultural improvement projects), the direct link between the new supply produced by the project and the actual demand to absorb it (a power station will present different degrees of uncertainty depending on the level of economic development of the region where it is established), and the degree to which economic, social, and political change can interfere with a project’s implementation (e.g., to what degree the rise of labor costs affects irrigation projects, or to what extent it is possible to insulate projects from political interference).Footnote 36 Notes taken during his trips into the field also elaborate on the difficulty of computing and quantifying benefits in several types of projects. Hirschman, referring to Ethiopia, underscored the case of telecommunications: “how do you compute [the] benefits of easier communications, [the] ability of scarce managerial talent to spread itself over wider areas, better information of market information, etc.”Footnote 37 Hence, these kinds of projects are often ignored, even if they may be the most productive.
This change in perspective called for a corresponding change in the Bank’s behavior. The Bank, Hirschman wrote, should avoid the “air of pat certainty” that emanated from project prospects and, instead, expose the uncertainties underlying them, exploring the whole range of possible outcomes. Moreover, the Bank should take into account the distributional and, more generally, the social and political effects of its lending. As a matter of fact, an excessively uneven distribution of the benefits of a project, besides jeopardizing political and social stability, was likely to create threats to the project itself. A new road project, for example, would open up new land for agricultural exploitation. According to Hirschman, the Bank should have explicitly addressed the issue of ownership of that land, with the aim of spreading the benefits of the project to the largest possible population. Of course, Hirschman was aware that spreading the benefits might require time: frequently, a project initially contributes to the development of the already more advanced segments of a country, and only in a second phase does it reach larger segments of the population and territory. This trickle-down mechanism was even clearer, for example, in the establishment of a power station, usually conceived to serve the largest city, and only subsequently capable of supplying smaller cities or the countryside. Still, Hirschman thought it important that “such a second benefit-spreading phase of projects … either be explicitly foreseen or at least its possible emergence be carefully watched and seized upon when it occurs.”Footnote 38 Hirschman also criticized the inadequate consideration of social and political factors in project analysis. In his opinion, many projects had encountered huge difficulties during their implementation because of too cursory an analysis of their political and social context. Regional, tribal, or center–periphery antagonisms, and the political power of specific interest groups are important elements that affect the success or failure of a specific project.
In addition to the influence that social and political factors might have on a project, Hirschman was interested in studying the reverse as well: were there projects that would help the development of political skills and ease intergroup agreements? In their travels, the Hirschmans noted that irrigation projects often brought in new resources that initially were believed to be available in strictly limited quantities: irrigation almost immediately creates disputes over the distribution of water, and related problems of administration and coordination between engineers and farmers, as well as between various organizations such as farmers’ organizations, extension services, etc. At the same time, these problems may have an unexpected positive spillover, as they become “a veritable school for bargaining, adjustment, etc.”Footnote 39 The case of an electrification project in Uganda provided another example of possible community building to come out of project management. In the town of Masaka, Indians and Africans lived completely separate lives, carrying on distinct economic activities and following different patterns of urban migration and settlement. The Indians all lived in town and could connect to the electric power network for a nominal fee; the Africans, on the other hand, lived scattered in the countryside, and, in order to connect to the power network, they were required to make a capital contribution that could be more than five times the fee paid by the Indians. The Ugandan project of power distribution had, thus, highly exacerbated the conflicts between ethnic groups in the country. At the same time, Hirschman noted, there was room to restructure the project, bringing the Indians and the Africans together to cooperate in managing the project, or to coordinate in having the two communities manage the project in different areas: “the technical nature of the common task,” the Hirschmans ventured to hope, “helps in making both [ethnic groups] ‘forget’ about the other’s skin.”Footnote 40 If properly reorganized, the Ugandan electric project would yield strong social benefits.
At the opposite side of the spectrum of projects with institution-building potential, one might consider highways. As a matter of fact, highway construction always left ample margins for improvisation, sloppiness, cutting corners, and sacrificing quality for quantity without any specific drive toward the implementation of more structured and effective institutions.Footnote 41 Moreover, no natural or technical constraint limited road constructions, thereby making them an activity particularly prone to political pressures and the disrupting effects of political struggle on national cohesiveness. This characteristic was shared by thermoelectric power as opposed to hydro. While hydroelectric power generation is limited by the actual availability of natural resources, a thermoelectric plant can be built virtually anywhere. Investments in thermoelectric plants, therefore, appeared to be much less protected from political pressures. In Hirschman’s terminology, this was an opposition between “foot loose” (roads and thermoelectric power generation) and “resource tied” public utilities.Footnote 42 If we consider projects from the point of view of institution building, irrigation would produce the maximum effect, highways the minimum, and railroads and electric power would fall somewhere in between.Footnote 43
Highways were nonetheless an interesting case study for the links between social phenomena and economic development. Often, Hirschman argued, road maintenance is prompted by a combination of physical deterioration and public protest: there is no doubt that the truck drivers’ refusing to pay road tolls because of the roads’ poor condition will negatively affect the financial resources needed to repair the roads, but, at the same time, their actions signal that the roads have reached an unacceptable level of deterioration, and this will probably induce a reaction (i.e., road maintenance).Footnote 44 In his The Strategy of Economic Development (1958), Hirschman had discussed at length the fact that in less-developed countries, often the resource that was actually lacking was not capital or any other physical asset but, rather, the ability to make decisions; he urged the individuation of “inducement mechanisms” that function as substitutes for this missing ability. The circular mechanism of (i) road use, (ii) lack of maintenance, (iii) road disruption, (iv) public protest, and (v) road maintenance was one of these “inducement mechanisms.” This observation was the seed for a particularly productive yield in Hirschman’s intellectual journey: as a matter of fact, it would become the starting point for the discussion of “voice” in Hirschman’s analysis of social interactions in his 1970 book, Exit, Voice and Loyalty.
In his “Interim Observations,” Hirschman argued that the Bank had thus far ignored those questions: “the projects appear to be judged wholly on their technical merits.”Footnote 45 In the notes taken during their field trips—but not in the “Interim Observations”—Albert and Sarah Hirschman fully elaborated on the limits of judging projects by only their technical merits, directly addressing the fundamental question of social equality and redistributive issues. “Profitability alone,” they maintained, “is no yardstick for social desirability of investment.”Footnote 46
Hirschman also focused on some specific Bank policies usually set forth as preconditions of lending: the Bank’s preference for dealing with development agencies independent from the national government and with contractors selected through international biddings; the Bank’s insistence that governments should profit from public utilities established through its loans; and the Bank’s insistence on exerting strict control over the internal administrative processes of the borrowing countries. Hirschman underscored instead the need to adapt to specific circumstances.Footnote 47 During his travels, he had noticed that certain agencies, which in principle were supposed to be autonomous from the national government, were actually impeded in effective decision making by disunited boards, whereas more streamlined chains of command in governmental ministries could prove to be more effective. In other cases, autonomous authorities were turning into semi-feudal centers of power, effectively making coordination among different branches of the national government impossible and destroying the principle of the unity of the budget.Footnote 48 As for the Bank’s attempts to control the internal administrative processes of the borrowing countries, Hirschman noted that very often this was plainly impossible: internal difficulties within recipient governments and intra-governmental conflicts were a common characteristic of borrowing countries, and the first casualty of these difficulties was reliable and open communication with external subjects.Footnote 49
Most of Hirschman’s observations were dismissed by Bank officers as either fairly obvious or definitely wrong. Some critics underscored that on several occasions, Hirschman had based his conclusions on only partial analyses of specific matters. “As for the tendency ‘to clothe the prospects of all projects with an air of pat certainty’,” said one comment, “the Doctor may have a point, but it must be remembered that our project reports are not economic dissertations. They must be brief and [the] many points considered are not always included in the reports.”Footnote 50 Considering that Hirschman, in his studies, had traditionally been interested in processes of decision making and “reform-mongering,” it is noteworthy that reactions from Bank officers underscored the need to calibrate messages to the audience and the situation, and insisted that at different stages of the decision-making process, different levels of complexity and thoroughness were advisable. As explained by Duncan S. Ballantine, a Bank officer and former professor of history at MIT and Harvard University, “in presenting a positive recommendation for action to the [Executive] Directors, a report must be positive … at some expense to the uncertainties. Nevertheless, during the process of appraisal leading up to the inevitable simplification of the issue, the staff … does ‘make a sustained effort at visualizing’ the uncertainties.”Footnote 51 In addition, the recently appointed head of the Transportation Division in the Projects Department, Warren Baum, listed several Bank projects “explicitly described as ‘experimental.’”Footnote 52
Had Hirschman taken more elements into account, the critics pointed out, he would have probably discovered that his proposals were actually very much in line with the Bank’s thinking and practices, or, as another officer put it, that “there is relatively little in [Hirschman’s] observations which has not normally been taken into account by Bank staff during project work.”Footnote 53 For example, when addressing the conflict that might arise between the Bank and a borrowing country over the borrower’s obligation to consult the Bank prior to key personnel changes, Hirschman had emphasized the need for a systematic look at the Bank’s previous relevant experience, adding that “efforts should be made to limit requirements of prior consultations to a few matters deemed essential.”Footnote 54 It was easy for a critic to note that the Bank was already doing what Hirschman proposed: “Dr. Hirschman is not well aware of the Bank’s practice concerning consultation. In the 253 loans and credits made in the last five years … only 40 loan and credit agreements required consultation…. These were ‘deemed essential’.”Footnote 55 Campbell Percy McMeekan, a senior agriculture specialist in the Division of Technical Operations who was responsible for the appraisal of agricultural development loans, remarked that “the Bank does do just what is suggested.”Footnote 56 Hans Adler, an authority on transportation projects and, like Hirschman, a refugee from Nazi Germany, concluded: “some of his suggestions are now a regular part of our appraisal. It might therefore be useful for Hirschman to read some of our recent reports.”Footnote 57
In other cases, Hirschman’s assumptions were considered plainly wrong. In his “Interim Observations,” he had stated that the Bank’s task was “to undertake projects that are unattractive to private capital because uncertainty surrounding their success is too high.”Footnote 58 Spottswood sharply noted that “it has been the policy of the Bank not to make loans for projects where uncertainties surrounding their success were high, on the ground that the borrower’s welfare and development would be best served if his investments were made, and Bank money used, only for sound projects.”Footnote 59 As for Hirschman’s claim that the Bank insisted on dealing with agencies that were autonomous from government, it was remarked that the point was not, as Hirschman seemed to see it, the theoretical opposition of politically autonomous agencies vs. politically dependent agencies, but rather the practical separation of political and managerial functions.Footnote 60 In this respect, Warren Baum argued that the Bank was far from doctrinaire, while those familiar with Hirschman’s chosen examples considered them “singularly inappropriate.”Footnote 61 Some Bank officials also doubted that it would be useful for the Bank to explicitly address political and social factors in project appraisal. The Bank was perceived by its staff as relatively free from political pressures and driven by a technocratic approach. An explicit inclusion of political and social considerations in project appraisal, therefore, meant that the Bank would be “vulnerable to every variety of reaction from member countries and would lose its fortunate position.”Footnote 62
An important cause for criticism from Bank officers was that Hirschman showed a complete lack of interest in quantitative evaluations, focusing instead on a qualitative analysis or, as he wrote, on “comparing ‘personal profiles’ of projects in different sectors.”Footnote 63 To a bureaucracy that expected Hirschman’s study to shed some light on issues such as the measurement of indirect economic benefits of projects and the feasibility and the effects of applying shadow prices for products and factors of production, this was most likely disappointing.Footnote 64 Furthermore, criticism may have been intensified by Hirschman’s claim that his reflections were “strictly policy-oriented,”Footnote 65 and, thus, had an immediate consequence for the Bank’s loan policies, while many Bank economists considered that they did not have any application. A Bank engineer commented that Hirschman was “not well aware of the choices facing the Bank.”Footnote 66
As a matter of fact, when the Bank’s economists had the opportunity to read the final output of Hirschman’s research, published in 1967 by the Brookings Institution as Development Projects Observed, they were not excessively critical of it; rather, they highlighted the fact that “his conclusions have only very limited applicability.”Footnote 67 Robert Asher reported that Dick Demuth, the director of the Bank’s Development Services Department, considered the book’s first chapter, “The Principle of the Hiding Hand,” “a bit thin, particularly with respect to relevant guidance for those who must decide whether to undertake, continue, or complete a proposed project.”Footnote 68 This was not unexpected since Asher himself had previously called Hirschman’s theses “disconcerting … to those in quest of clearer criteria to govern eligibility for foreign aid” (Asher Reference Asher and Asher1962, p. 217). In “The Principle of the Hiding Hand,” Hirschman stated that the underestimation of problems is a powerful mechanism that allows projects to be taken up that would never otherwise be initiated. According to Hirschman, the problems that arise when a project is under implementation usually trigger a creative effort that leads to their solution. Differently from Toynbee’s theory of challenge and response, Hirschman’s principle posited that “people undertake some new task not because of a challenge, but because of the assumed absence of a challenge.”Footnote 69 Toynbee’s thesis, according to Hirschman, was an ex post rationalization, “per fare bella figura.”Footnote 70
Though rhetorically forceful and “Hirschmanesque” (Walter Salant’s term), this principle left many unconvinced. Even the members of the Brookings Advisory Committee who were more favorable to Hirschman’s approach, such as Walter Salant or Edward Mason, considered the principle one-sided.Footnote 71 Salant, for example, noted that this principle, far from being a general principle as Hirschman seemed to imply, was only one of the many possible associations between the estimation of problems and the estimation of one’s ability to solve them: of course, problems might be underestimated, but they might also be either correctly estimated or overestimated, and the same would be true for the ability to solve them. Nine outcomes were actually possible, from the successful Hiding Hand to sheer disaster (when an underestimated problem is coupled with an overestimated ability to solve it).Footnote 72
Years later, Hirschman admitted that his opening chapter “was close to a provocation. Nothing could be less ‘operationally useful’ than to be told that underestimating the costs or difficulties of a project has on occasion been helpful in eliciting creative energies that otherwise might never have been forthcoming” (Hirschman Reference Hirschman and Hirschman1994, p. ix). But this was the point: the principle of the hiding hand, based as it was on the actor’s ignorance, was not intended to be a policy tool. It was a way for Hirschman to elaborate on the need to include uncertainty and limited rationality in the Bank’s epistemology. While this principle received its final name in juxtaposition to Adam Smith’s “invisible hand,” its original name was perhaps more revelatory: in January 1965, from India, Hirschman wrote a long explanation of what he then used to call his “theory of providential ignorance” to his two daughters in New York:
The Pakistan project, the Karnaphuli paper mill, is perhaps most interesting because it shows that it isn’t so easy to ‘transfer’ an industry from one country to another. It looks easy—why shouldn’t the same machines perform just as well in East Pakistan as in Sweden? The fact is, however, that there are many differences, from the raw materials (bamboo instead of pine) to the demand in the market which require far more ‘creative’ adaptation than the country had probably expected it would to be. One almost feels that had they known all the troubles they were headed for, they would never have founded this industry, but having founded it they managed to solve their problems one by one. The secret of creativity is then to place yourself in situations where you’ve got to be creative, but this is done only when one doesn’t know in advance that one will have to be creative. This, in turn, is so because we underestimate our creative resources; quite properly, we cannot believe in our creativity until we experience it; and since we thus necessarily underestimate our creative resources we do not consciously engage upon tasks which we know require such resources; hence the only way in which we can bring our creative resources into play is by similarly underestimating the difficulty of a task.Footnote 73
More generally, Hirschman underscored the centrality of side effects, and, in his 1967 book, he described project appraisal as the art of visualizing them. In his definition, side effects are not just “secondary effects,” mere spillovers or repercussions of a project. More often than not, they turn out to be “inputs essential to the realization of the project’s principal effect and purpose” (Hirschman Reference Hirschman1967a, p. 161). They are called “effects” instead of, for example, “conditions” or “prerequisites” because they are “eventual requirements,” essential for the project to survive after the start-up and mature into a long-lived endeavor, although they are not needed from inception. A project for the development of highways (as opposed to railways) would serve as an example. It is commonly held that investment in highways develops the trucking industry and, therefore, enhances entrepreneurship. But this secondary effect may have much wider consequences: “entrepreneurship means political power, which in turn means the ability to change the rules of the transportation game decisively in favor of the highways” (Hirschman Reference Hirschman1967a, p. 162), and what might seem a mere secondary effect (enhanced entrepreneurship) becomes a decisive element, making the decision to develop motorways instead of railways irreversible. For this reason, side effects are at the same time necessary and unpredictable, and therefore central in Hirschman’s approach: “a search for the indirect effects is to be recommended if only as a heuristic device, as a means of identifying some of the basic conditions for the project’s success” (Hirschman Reference Hirschman1967a, p. 169).
Cost-benefit analysis and its attempt to make precise calculations of the secondary effects of investments, according to Hirschman, turned this search for indirect effects into an excessively rigid process, hampered by too many arbitrary assumptions, and made the quest for a unique ranking a futile exercise. “How could it be expected,” wondered Hirschman, “that it is possible to rank development projects along a single scale by amalgamating all their varied dimensions into a single index when far simpler, everyday choices require the use of individual or collective judgment in the weighing of alternative objectives and in the trade-off between them,” especially when the aim is not to facilitate the decision maker’s exercise of informed judgment, but to make him dispense with such judgment altogether (Hirschman Reference Hirschman1967a, p. 179). The following passage captures Hirschman’s unsystematic approach, his idea that it is impossible to detect a set of criteria uniformly applicable to all projects:
Upon inspection, each project turns out to represent a unique constellation of experiences and consequences, of direct and indirect effects. This uniqueness in turn results from the varied interplay between the structural characteristics of projects, on the one hand, and the social and political environment, on the other. To facilitate the understanding of this interplay I focused … on various properties of projects—primarily uncertainties and latitudes—that condition their total behavior and career…. There was no intention to erect these manifold aspects of project behavior into full-fledged criteria that should be applied to all projects; rather I was seeking to provide project planners and operators with a large set of glasses with which to discern probable lines of project behavior, in the expectation that the analysis of each individual project would require different and rather limited subsets of the full set of glasses which has been exhibited. (Hirschman Reference Hirschman1967a, p. 186, emphasis in the original)
By the time Hirschman was drafting the manuscript of his book, it had become clear that his research agenda was at variance with the Bank’s. The comment of an early supporter of Hirschman’s project summarizes the discouragement within the Bank at the end of their collaboration:
I was one of the original supporters of World Bank cooperation with Professor Hirschman because of my conviction that the Bank still has a good deal to learn about project preparation and evaluation and because I thought that a fresh look by a perceptive and objective observer could add significantly to our knowledge of this important subject. However, I don’t believe the manuscript does this. It is well written and contains a number of interesting observations. But by and large it does not contain any operationally useful analysis of the merits and priority of the particular projects observed by Professor Hirschman or of the kind of reshaping or rethinking of the projects which might have made them better. In short, I for one gained no significant new insights into the process of project preparation and evaluation.Footnote 74
VI. THE ROOTS OF THE DISAGREEMENT
Hirschman’s attempt to establish a qualitative approach to project appraisal had as its cornerstone the detailed historical reconstruction of the “personal profiles” of projects as well as their larger political and social context. It aimed at underscoring “the element of the unknown, the uncertain and the unexpected,” as Hirschman put it, in order to understand what deflected the projects from their originally charted course. Finally, its goal was to assess the broader impact of a project, such as its effects on the distribution of wealth, income, and power in the affected population. Hirschman’s approach to project appraisal was a natural evolution of his previous work, which had marked an increasing distance from the early debates in development theories. The “failure of several of the earlier ideas as practical policy solutions,” as Tony Killick put it (Killick Reference Killick1978, p. 27, italics in the original), prompted Hirschman’s detailed inquiries into the mechanisms of economic policy making, as well as the early evaluation studies that the Bank undertook in the 1960s and the convergence of the Bank and Hirschman on the need to develop a more systematic method for appraisal.Footnote 75 But the shared perspective on what was needed did not become a shared perspective on how to meet this need. Hirschman tried to transform the Weltanschauung of the Bank and the Bank’s approach to project design, management, and appraisal. In his effort, rather than focusing exclusively on building evaluation procedures, he added questions that were more ample and strategic in nature. Moreover, while some of his insights were path breaking and innovative, such as the need to carefully consider the social and political factors that could affect a project, other reflections appeared excessively academic—for example, his discussion of the degree of independence of regional development agencies from the political sphere. Ultimately, Hirschman neglected, de facto, the eminently practical needs that were at the basis of the Bank’s effort to discuss evaluation procedures. Yet, the Bank expected Hirschman to analyze the Bank’s projects without any revolution in perspective, and to work on making the Bank’s project design and management somehow more measurable, predictable, and possibly replicable. Referring to the contemporary literature on cost-benefit techniques of project appraisal, Bank economists expected that Hirschman would shed some light on questions such as “the causes of cost run-ups and of delays in execution, [and] the feasibility and the effects of applying ‘shadow’ prices for products and factors of production.”Footnote 76 Herman van der Tak suggested that Hirschman ought to have “summarized the positive and negative aspects of various types of projects” into “a more operational ‘summary and conclusions,’ … which would provide practitioners with guidelines.”Footnote 77 An attempt to write this operational guide was eventually made by the Bank’s Economics Department, resulting in a hybrid between a “summary and conclusions” for practitioners as proposed by van der Tak, and a series of “annotated questions” about projects as proposed by Hirschman. However, a proper operational version of Hirschman’s book never saw the light.Footnote 78
While the Bank was struggling with the seeming inapplicability of Hirschman’s reflections, cost-benefit analysis received a tremendous boost from research pursued by other multilateral organizations. Cost-benefit analysis had been taking shape in the 1920s as a technique in water-resources development, cultivated mostly by engineers, which subjected public investment decisions to economic analysis and evaluated alternative projects in terms of the maximization of public “utility.” It received a further stimulus during the 1930s expansion of public investment activity in the United States, especially thanks to the Flood Control Act of 1936, which dictated that only projects whose benefits were projected to exceed their costs could receive federal funds. However, the intrinsic difficulties of quantifying the projects’ effects, compounded with competing views among the different bureaucracies that had authority on water management in the United States, made it impossible to agree on a standardized method of cost-benefit analysis until the 1960s (Porter Reference Porter1995, pp. 148–189). Moreover, as Stephen Marglin observed, “benefit-cost analysis was introduced as a means of project ‘justification’ alone …, not as a tool for project planning; in American practice (as distinct from theory) it often has served as window dressing for projects whose plans have already been formulated with little if any reference to economic criteria” (Marglin Reference Marglin1967, p. 18).Footnote 79 A push toward standardization and uniformity came only in the late 1940s and especially the 1950s, as a way to overcome conflict among agencies and because of the increasing interest in cost-benefit analysis among economists: the development of the new welfare economics after World War II gave intellectual legitimacy to the attempt to turn, as a critic put it, “a useful way of roughly assessing the promise of a particular project, or comparing various ways of carrying out a project,” into “a precision tool for attaining general economic efficiency” (Hammond Reference Hammond1966, p. 222).
Hirschman had already shown doubts about the accuracy of cost-benefit analysis in his preliminary work for the Bank. In that specific case, however, Hirschman seemed more concerned about the possible inhibitory role of cost-benefit analysis than about its role as ex post justification: “were costs underestimated as is frequently the case? If so, was this error matched by an underestimate of benefits (also frequent) so that the Bank’s judgment of the actually experienced cost-benefit ratio would have been poorer if just its cost estimate had been more nearly correct?”Footnote 80 Between the late 1960s and early 1970s, a new wave of studies in cost-benefit analysis appeared. In 1968 the Development Centre of the Organisation for Economic Co-operation and Development (OECD) published the second volume of its Manual of Industrial Project Analysis in Developing Countries. While the first volume focused on the profitability of industrial investments from the firm’s point of view, the second volume, authored by Ian M. D. Little and James A. Mirrlees, centered on social cost-benefit analysis. The Little–Mirrlees volume, republished in a hugely modified version in 1974, was immediately regarded as a groundbreaking contribution in project appraisal in developing countries, especially for their use of shadow-prices; i.e., prices that were meant to reflect the social effects of projects, as opposed to their private profitability (Little and Mirrlees 1968, 1974). In the turn of a few years, this approach had become “a school” that directly influenced “the thinking of economists engaged in planning in developing countries” (Kornai Reference Kornai and Boskin1979, p. 76). In 1972 the United Nations Industrial Development Organization (UNIDO) published another milestone volume in project appraisal, authored by Partha Dasgupta, Stephen Marglin, and Amartya Sen (Dasgupta, Marglin, and Sen 1972). Although the differences from Little–Mirrlees’ approach were important (see, for example, the comparison of OECD and UNIDO approaches by Dasgupta Reference Dasgupta1972, and a critique of the OECD volume by Frances Stewart and Paul Streeten 1972), the two approaches still had enough in common to be viewed “in some immediate sense [as] similar” (Dasgupta Reference Dasgupta1972, p. 41) or having “a similar spirit” (Kornai Reference Kornai and Boskin1979, p. 76). To be sure, both these approaches discussed the role of uncertainty in project design and appraisal. But, in practical matters, they tended to conflate this term with what is usually meant by “risk”; that is, something subject to measurement.Footnote 81 Hirschman, instead, following the Knightian dichotomy between “risk” and “uncertainty,” considered uncertainty a quality that was impossible to measure. In an article from the early 1960s, co-authored with political scientist Charles Lindblom, Hirschman elaborated on the impossibility of quantification at a more general level: “it is clearly impossible to specify in advance the optimal doses of … various policies under different circumstances. The art of promoting economic development … consists, then, in acquiring a feeling for these doses” (Hirschman and Lindblom Reference Hirschman, Lindblom and Hirschman1971, pp. 83–84).
By the early 1970s, in any case, Hirschman’s book had been completely forgotten. The 1974 Little–Mirrlees book dismissed it as “a stimulating essay on the theme that any assessment leaves things out” (Little and Mirrlees 1974, p. 379), and the rest of the literature completely ignored it. When, in 1970, the World Bank published a small volume on Techniques for Project Appraisal under Uncertainty (Reutlinger Reference Reutlinger1970)—a topic that seemed naturally connected with Hirschman’s inquiry—it did not include a single reference to Hirschman’s 1967 book. The same happened in 1975, when the World Bank published its own reference text in project appraisal—very much in the tradition of Littlle and Mirrlees (Squire and van der Tak 1975).
Another issue may have accounted, if not for the oblivion into which Hirschman’s book fell, at least for the reaction it prompted in the Bank, a reason that has more to do with the sociology of organizations than the history of development economics. The dismissal of Hirschman’s work by several Bank staff may have been due to the fact that he was an outsider to the institution, but commented on (and criticized) its core activity, and the ability of its staff to read and interpret development patterns. This distrust had first appeared in the “Interim Observations” that Hirschman had circulated. On that occasion, a staff member dismissed Hirschman’s observations on Bank policies as “the echo of some of the most frequently recurring beefs we heard from the other side of the fence, but only from some people on the other side of the fence, not necessarily representative people,” and accused Hirschman of having been heavily one-sided and uninterested in the Bank’s viewpoint.Footnote 82 Another staff commented: “what [Hirschman] is trying to do is to call our attention to some of the pitfalls inherent in our operations.… It is useful to have an outsider’s reminders and the advantage of his perspective. But we should be sure that Mr. Hirschman has access to all the facts, and all the points of view, before completing his book. In the academic and literary professions, criticism seems to draw higher marks than does praise, and while Mr. Hirschman is very knowledgeable and capable, he may not be above succumbing to this temptation.”Footnote 83 Even when there was no suspicion from the Bank’s commentator that Hirschman might be fishing for praise through indiscriminate criticism, there remained concern that his academic perspective would render his effort irrelevant to the operations of a development institution like the World Bank. Duncan Ballantine, a former academic himself, noted that Hirschman was attempting a “marriage between two inherently incompatible points of view—that of the decision maker and that of the academician or seeker of truth.”Footnote 84 While he recognized the value of such an exercise, the risk of irrelevance due to the prevalence of the academic perspective was high. The common note was that an outsider was not equipped to understand the Bank’s operations.
Much of the criticism by World Bank officials seems plausible and sensible: there is no question that, no matter how insightful Hirschman was, he nonetheless failed to provide any concrete tools for operations evaluation. At the same time, and beyond the Hirschman case, the legitimacy of comments, criticisms, and advice from external subjects is a sensitive issue at the heart of the evaluation function: the reactions of Bank officials highlight that feedback mechanisms are a soft spot in most organizations. Even for the Bank’s internal (but independent) Operations Evaluation Department, balancing independence and objectivity with collaboration and communication between operational and evaluation staff has always been a major challenge. This equilibrium is highly unstable and subject to recurrent swings. At times, the evaluation function has seemed insufficiently independent and credible. In the mid-1990s, a (now defunct) Quality Assurance Group was established to strengthen the Bank’s project evaluation and accountability as an answer to a steep deterioration of the Bank’s operations in the previous decade.Footnote 85 However, unlike the main Operations Evaluations, which remained highly independent, the Quality Assurance Group reported directly to management.Footnote 86 As a Bank officer underscored in 2003, this undermined the credibility of the evaluation function in principle and in practice: “the latest review by the Bank’s Quality Assurance Group shows that among the Bank projects entering the portfolio last year, 93 percent were satisfactory. If you put out a report showing that 93 percent of your projects are satisfactory, do you have a credibility problem?”Footnote 87 At other times, independence had come at the price of apparent irrelevance: Robert Picciotto, the OED Director-General from 1992 to 2002, reported that “under Preston, basically Management had a benign neglect of evaluation. That was also Ernie Stern’s view: ‘You are independent. You do your thing. Don’t talk to me….’ ... Basically [evaluation] was viewed as [a] sideline kind of occupation” (Picciotto Reference Picciotto2000, p. 16). Perhaps even more discouraging is the testimony of a former OED director who maintained that, when he worked in Operations, he was not influenced, either directly or indirectly, by OED’s output: “the typical reaction of operational staff to [OED’s] findings was still to dispute the factual details and to question the usefulness of the findings, and to attack OED’s methodology” (Köpp 2003, p. 55). No wonder that Albert Hirschman experienced similar criticisms.
VII. CONCLUDING REMARKS
This article has shown a chiasmic movement in the relationship between Albert Hirschman and the World Bank in the 1960s. Initially, Hirschman and the Bank agreed on a common objective: a thorough and systematic analysis of development projects, in order to draw generalizations for the Bank’s decision-making process. Subsequently, Hirschman and the Bank diverged: most of the Bank’s officers considered Hirschman’s observations and his final book to be either useless or wrong, and no further discussions took place between Hirschman and the Bank.Footnote 88 As we have seen, this story shows how unstable the equilibrium between evaluation and operations within this type of organization can be, a very complex issue of which this article has only scratched the surface. Instability is a structural feature of the relation between evaluation and operations, and historical analysis can provide useful elements for understanding this tension. Furthermore, this story discusses how, during the 1960s, the Bank participated in, and encouraged, a series of studies that linked theoretical and practical analyses for the purpose of gaining a better understanding of the outcomes of its development efforts. It is commonly believed that, prior to the advent of McNamara at the end of the 1960s and his hiring of a new cohort of economists, the Bank had little relation with the field of development economics. History shows that, on the contrary, the Bank was an active participant in the development economics debates and the shift in focus that took place in the 1960s. As often in the case of the World Bank and other organizations active in the field of development assistance, the Bank’s contribution to the debate was the result of specific operational needs. In other words, policies influenced theoretical reflections, and the latter, in turn, contributed to shape policies.Footnote 89
Something remains to be said about why the need for an evaluation function emerged only in the 1960s, after more than two decades of activity in the field. The answer to this question can be only a tentative one. The publication of guidelines and manuals for project appraisal by other multilateral organizations, which happened in the late 1960s and early 1970s, demonstrates that this delay involved the entire galaxy of development agencies and not the World Bank alone. The United Nations Development Programme (UNDP) did not establish an evaluation system until the 1970s. “For many years,” Craig Murphy writes in his history of the UNDP, “project evaluation was just a matter of asking the expert assigned, the executing agency, and the developing country government whether they were ‘satisfied’ with the project” (Murphy Reference Murphy2006, p. 112).
Murphy also reports the refrain of the UNDP’s first administrator, and former administrator of the Marshall Plan, Paul Hoffman: “if we do our jobs well, we will be out of business in twenty-five years.”Footnote 90 Project evaluation is perhaps a function that comes with age, when early optimism has yielded its ground to a more sober assessment of the difficulties in fostering development, and an organization recognizes a need to reflect on itself, learning from past experiences: both successes and failures. Not by chance, after all, did Hirschman highlight the need for project evaluation when he acknowledged the inadequacy of previous approaches and grand development theories. But the enthusiasm of the “glory days of high development theory” was possible because the solution seemed close at hand. Perhaps, if the enormous difficulty of fostering development had been evident from the very beginning, the initial wealth of intellectual energy devoted to development would have been much smaller. Hirschman’s principle of the hiding hand failed to become an analytical instrument in the toolbox of project appraisal, but it may prove useful today to explore the very question of the birth of project appraisal.