What explains the shift in the effectiveness of counterinsurgency operations over time? Why did nineteenth-century incumbents have an easier time defeating insurgent opponents than their twentieth-century counterparts? These questions drive much of the recent work on counterinsurgency and asymmetric conflict. Lyall and Wilson, for example, find that incumbents defeated insurgent foes “in nearly 81 percent of pre–World War I cases but in only 40 percent of the post–World War I cases.”Footnote 1 Arreguín-Toft likewise finds that powerful states were victorious in 88 percent of their asymmetric conflicts in the first fifty years of the nineteenth century, compared to just 49 percent in the last fifty years of the twentieth century.Footnote 2
Political scientists propose various hypotheses to explain this sudden and dramatic shift. Some point to changes in the organization of counterinsurgent forces. Lyall and Wilson, for example, argue that the mechanization of modern armies deprives them of the ability to extract information from local populations, limiting their ability to identify and separate insurgents from their bases of support.Footnote 3 Others emphasize shifts in counterinsurgent strategy. Arreguín-Toft, for example, argues that states that adopt conventional strategies are unable to defeat adversaries who respond with guerilla tactics of evasion and harassment.Footnote 4 Still others have focused on the increased capacity of insurgents themselves. Mann, for example, points to the spread of nationalism and small arms, which has transformed the capacity of relatively weak societies to resist.Footnote 5
These theories are attractive, especially in light of the U.S. failure in Vietnam and recent struggles in Iraq. In Vietnam, for example, the American military remained wedded to a conventional war-fighting strategy for most of the conflict, adapting slowly to the unique challenge posed by the communist insurgency.Footnote 6 All the while, the Vietcong took advantage of Vietnamese nationalist sentiment, as well as assistance from external powers like China, and fought a skillful guerilla campaign. In Iraq, the United States likewise emphasized conventional operations against suspected insurgent strongholds at the expense of protecting the local population.Footnote 7 The Sunni Arab insurgency, meanwhile, combined easy access to explosives and used propaganda opportunities provided by new media to capture the sympathies of the Iraqi population.
These theories do not simply claim to explain contemporary failures, however, but historical cases as well. If twentieth-century incumbents relied on mechanized forces that alienated them from local populations, the assumption is that their nineteenth-century counterparts fielded agile forces that culled information. If twentieth-century incumbents struggled against well-armed and ideologically driven adversaries, the assumption is that their nineteenth-century counterparts easily triumphed over ill-equipped and unmotivated opponents. These theories are united in their view that counterinsurgency wars—and the militaries that fight them—have undergone a fundamental transformation over the past 200 years.
I challenge the argument that changes in the character of counterinsurgency warfare can explain the decline in incumbent victory. States in the nineteenth century frequently embarked upon counterinsurgency operations with poorly thought-out strategies and with troops who were inadequately trained for guerilla warfare. Conversely, nineteenth-century insurgents were not nearly as unmotivated or inept as is often assumed. Indeed, a detailed examination of the historical evidence reveals much more continuity than discontinuity in the nature of counterinsurgency warfare.Footnote 8 Like their twentieth-century counterparts, counterinsurgency forces in the nineteenth century achieved success when they managed to progressively deprive insurgents of sanctuaries from which to operate. Much as today, the recruitment of local security forces and the cultivation of indigenous intermediaries played a vital role in this effort. While interstate warfare has arguably undergone a revolution over the past two centuries, strategies of insurgency and counterinsurgency have stayed remarkably constant.
Instead, the decline in incumbent victory can be explained by a broader shift in the normative and material structure of the international system from one that favored colonial governance to one that opposed it. In the nineteenth century, colonial powers operated in a relatively favorable political context when conducting counterinsurgency operations. They encountered minimal resistance from other states, who viewed the acquisition of colonial empires as a legitimate activity. They possessed the support of home populations, who valued colonial possessions. They exploited ties with local elites, as well as divisions within targeted societies, to recruit local collaborators and extract local assets. They enjoyed the flexibility to pick and choose when to expand the colonial frontier, which allowed them to modulate their demands and co-opt local opposition. With these advantages, nineteenth-century colonial powers systematically overwhelmed local resistance.
In the twentieth century, in contrast, colonial incumbents struggled in an unfavorable political environment. Home populations viewed colonial possessions as expensive burdens. Norms of self-determination challenged the legitimacy of colonial domination, while new superpowers turned a skeptical eye toward imperial holdings. Indigenous elites bristled under the rigid confines of the colonial state and sought new external patrons. Fearful of setting counterproductive precedents, colonial officials struggled to find policies that would balance demands for local autonomy with the need to maintain imperial control. In these circumstances, incumbent militaries were rarely afforded the time, resources, or support necessary to implement effective counterinsurgency strategies. These combined trends resulted in a dramatic decline in incumbent success in the twentieth century.
This argument builds on a growing literature that illustrates how shifts in the character and composition of the international system can shape patterns of insurgent violence.Footnote 9 In particular, the decline of imperial rule's legitimacy in the core alongside the colonial state's failure in the periphery transformed the political context of counterinsurgency operations. These developments not only eroded the capacity of colonial militaries to meet insurgent challenges, but also provided opportunities for anticolonial movements to use violence in politically efficacious ways. While counterinsurgency operations are primary shaped by local factors, this article demonstrates how changes in the normative and material foundations of international politics can interact to shape the political choices of incumbents and rebels alike. Although rooted in an analysis of historical trends, this argument also has broad implications for the study and practice of counterinsurgency operations. In particular, it calls into question the vast literature that argues that success in counterinsurgency operations is primarily the result of choosing the proper strategy.Footnote 10 The findings suggest that political conditions may make it difficult for incumbents to defeat guerilla opponents, no matter how sophisticated their particular strategies.
Theories of Counterinsurgent Success and Failure
There are many theories of how actors can defeat insurgent opponents, but they are all based on one simple principle: to be successful, a counterinsurgent must identify guerilla forces and direct violence against them. Insurgents succeed when they are hidden, either in sanctuaries or among the population. Clausewitz recognized this point when he noted, “the most characteristic feature of insurgency” is that “the element of resistance will exist everywhere and nowhere.”Footnote 11 Callwell, the author of the definitive 1906 book on small wars, likewise argued, “the most unpleasant characteristic of these wars” is “the difficulty of bringing the foe to action.”Footnote 12 A basic goal of counterinsurgency warfare, therefore, is to force insurgents into the open by limiting their ability to operate from sanctuaries or to hide among the population.Footnote 13 When insurgents are exposed, they are vulnerable to the superior numbers, organization, and weaponry of counterinsurgency forces. A corollary to this observation is that undirected violence, designed to punish a rebellious society into submission, rarely succeeds. At a minimum, such random violence fails to erode the fighting capacity of insurgents, who remain secure in their hidden sanctuaries. At a maximum, this violence can fuel popular resentment and swell the ranks of insurgent groups.
Although most theories of counterinsurgency warfare accept this basic premise, they differ on which strategies an incumbent should employ to identify insurgents, how best to use force to erode insurgent capacity, and what types of insurgents are best able to evade detection and destruction. They also differ in their accounts of why incumbents in the nineteenth century proved more effective than their contemporary counterparts in identifying and overwhelming insurgent opponents.
Changing Structure of Counterinsurgency Forces
Some political scientists argue that the structure of incumbent militaries can influence their capacity to effectively identify and destroy insurgent enemies. Lyall and Wilson, for example, argue that while modern mechanized armies are “isolated from local populations,” the foraging armies of the nineteenth century were “forced to interact extensively with local populations to acquire their provisions.”Footnote 14 The extraction of supplies from the local population not only provided nineteenth-century armies with “excellent awareness of local-level power relations, cleavages and languages,” it also freed up forces for “maintaining a direct and sizable presence in contested areas.”Footnote 15 Cohen likewise emphasizes how small wars require different doctrines and force structures than large-scale conventional wars. American forces equipped and trained for rapid armored offensives in Europe, for example, are ill-suited for small wars where units must fight defensively against insurgent opponents “under a host of political constraints.”Footnote 16 In contrast, the British regimental system of the nineteenth century created units who were experienced in operating in “remote and alien locations” and could retain cohesion despite lengthy deployments overseas.Footnote 17
The argument that nineteenth-century armies were better suited for the rigors of counterinsurgency warfare appears plausible. At the end of the nineteenth century, for example, more than 54 percent of the soldiers in the British regular army were deployed overseas.Footnote 18 By way of comparison, an average of 23 percent of the U.S. military was deployed overseas during the Cold War, mostly in Western Europe.Footnote 19 Nineteenth-century troops' tendency to serve together overseas for long periods undoubtedly prepared them well.
At the same time, nineteenth-century militaries exhibited a number of limitations. First, most nineteenth-century European militaries were conservative bureaucracies that adapted slowly to insurgent challenges.Footnote 20 In terms of doctrine, nineteenth-century militaries largely planned to fight European opponents along conventional lines.Footnote 21 Near the end of the century, accounts of “desultory warfare” appeared more frequently in military journals and specialized volumes such as Callwell's Small Wars, yet these lessons were not systematically integrated into training or doctrine.Footnote 22 As a result, commanders facing insurgent enemies either remained wedded to inappropriate tactics or were forced to improvise.
Second, most nineteenth-century European militaries were equipped to fight conventional forces rather than guerilla opponents. Heavy cavalry and artillery batteries had little utility for combating insurgents in dense bush and jungle. European militaries did make some use of light infantry forces, which proved effective in skirmishes with guerilla opponents. By the early 1820s, for example, the British army had raised eight regiments of specialist light infantry. Yet as Strachan notes, “the scattered nature of the army's imperial obligations” meant that “the nearest true light infantry regiment would be hundreds of miles away.”Footnote 23 Moreover, many officers believed light infantry made ineffective soldiers. Others worried that training men to fight in more open orders would wreck discipline.Footnote 24
Third, nineteenth-century European militaries suffered from considerable logistical difficulties when operating against guerilla armies. Foraging may have been a useful means to support troops on the cultivated plains of central Europe, but elsewhere European armies were forced to bring forward supplies via wagon or human portage.Footnote 25 Dense vegetation, heavy rainfall, the absence of roads, and presence of disease also conspired to frustrate efforts to move forward supplies.Footnote 26 Even when European militaries were able to rely on foraging for supplies, few viewed foraging as an effective way to obtain intelligence from local populations. Contra Lyall and Wilson, military theorists described foraging as a liability: the dispatch of foraging parties sapped power from advancing columns, foraging parties were vulnerable to ambush and desertion, and extracting food from local populations often provoked resentment.Footnote 27 In short, nineteenth-century militaries were not trained or equipped to fight small wars and encountered significant logistical challenges when operating against illusive enemies in distant theaters.
Changing Counterinsurgent Strategies
A second set of arguments emphasizes the particular strategies employed by incumbents and their insurgent opponents. Arreguín-Toft, for example, argues that counterinsurgents in the nineteenth century were successful because they could engage in barbarism. By systematically targeting civilians, nineteenth-century militaries could deprive insurgents of “sanctuary and social support”Footnote 28 and thus reduce their capacity to resist. In contrast, twentieth-century militaries found barbarism “a difficult strategy to prosecute effectively.”Footnote 29 In particular, democratic countries find it “risky” to employ barbarism because it “carries the possibility of domestic political discovery (and opposition).”Footnote 30 The argument that a strategy of barbarism allowed nineteenth-century militaries to defeat insurgent opponents is echoed by Merom's work on democratic states and small wars. Like Arreguín-Toft, Merom argues that the main reason democracies lose small wars is that “they find it extremely difficult to escalate the level of violence and brutality to that which can secure victory.”Footnote 31
While nineteenth-century counterinsurgents routinely employed barbarism, there are a number of problems with the argument that brutality paved the way to victory. First, it is not clear that barbarism is a tool employed by only nineteenth-century nondemocracies. Valentino, Huth, and Balch-Lindsay, for example, find that 32 percent of states facing large-scale guerilla insurgencies in the post-1945 period engaged in mass killing of civilians in an effort to defeat their opponents.Footnote 32 Nor are democracies necessarily less likely than authoritarian states to engage in barbarism. In his study of interstate wars between 1815 and 2003, for example, Downes finds that 32 percent of democracies engaged in civilian victimization compared to 30 percent of autocracies.Footnote 33
Second, barbarism often proved a clumsy tool. Between 1894 and 1908, for example, the British conducted eighty-nine expeditions to punish the restive population of western Kenya. Of these, 25 percent failed to achieve their objectives because of a lack of manpower or enemy harassment.Footnote 34 The Dutch likewise labored for more than forty years to defeat armed bands of guerillas in the mountainous jungles of Aceh.Footnote 35 Not only were military operations designed to punish or kill local populations time consuming, they were also expensive. British Colonial Secretary Lord John Russell, for example, warned that such brutality could only lead “to flagrant injustice, cruel wars, and protracted misery.”Footnote 36
Given the limits of outright brutality, nineteenth-century militaries often resorted to more collaborative strategies for securing victory. In his case study of the Murid uprising, for example, Arreguín-Toft finds that Russia's policy of “kill everyone and destroy everything … backfired.”Footnote 37 It was not until Russian generals began to offer clemency to Murid fighters that they were able to affect the surrender of the resourceful rebel commander Imam Shamil.Footnote 38 Other states adopted a policy of recruiting indigenous manpower to serve in local army and police forces.Footnote 39 Indian sepoys, for example, consistently outnumbered European officers, comprising between 80 and 90 percent of the Indian Army under the East India Company.Footnote 40 A similar percentage of French expeditionary forces in the Sudan were comprised of African tirailleurs.Footnote 41 In other cases, Europeans used proxies to fight on their behalf. During their conquest of Uganda, to name one example, the British relied heavily on military assistance provided by local Ganda chiefs to subjugate and administer neighboring peoples.Footnote 42 Much like their contemporary counterparts, therefore, nineteenth-century militaries used a variety of strategies to defeat insurgents, sometimes including barbarism, but also the recruitment of local forces and collaborators who could help detach insurgents from the population.
Changing Lethality of Insurgent Forces
A third explanation emphasizes shifts in the capacity of insurgents themselves, arguing that various technological and social changes have increased the lethality of insurgents over the course of the twentieth century. Mann, for example, argues that a “second revolution in military affairs” has “turned the tide of pacification technologies against imperialism.”Footnote 43 Kaldor likewise contends that the spread of light weaponry and the availability of external funding sources have contributed to a rise in the frequency and lethality of rebel groups.Footnote 44 Along the same lines, Desai and Eckstein emphasize the ways modern guerilla movements can exploit ideologies such as communism and nationalism to “harness the grievances and fantasies of the oppressed.”Footnote 45 Kilcullen likewise argues that contemporary insurgents can exploit advanced communications to mobilize followers, attract outside resources, and share best practices.Footnote 46
Taken as a whole, these arguments paint a dismal picture of nineteenth-century insurgents, whose defeat at the hands of technologically superior and better-organized Western armies was seemingly inevitable.Footnote 47 Yet indigenous forces during the nineteenth century were not as disadvantaged as is often assumed.Footnote 48 Africa during the precolonial period, for example, was awash in firearms. The Zulu kingdom possessed nearly 8,000 firearms prior to its 1879 clash with the British.Footnote 49 Similarly, nearly half of the 18,000-man army of the Merina Kingdom of Madagascar was armed with rifles, most of which were “modern Sniders or Remingtons.”Footnote 50 European proconsuls and frontier officials repeatedly protested to officials back home about the danger posed by the firearm trade. The British acting consul for the Oil Rivers protectorate, to take a typical example, described the prevalence of small arms as a “standing menace” and a “cause of frequent wars.”Footnote 51
While nineteenth-century insurgents were well-armed, they had difficulties putting their weapons to effective use. Indigenous forces, for example, had rarely been trained “to use guns in a disciplined way” and often lacked reliable powder or ammunition.Footnote 52 But there is little reason to think that the fundamental skill imbalance between insurgents and conventionally trained armies has narrowed over the course of the last century. As Mack has noted, in all of the major asymmetric conflicts of the twentieth century, conventional militaries were able to impose decisive defeats upon insurgents in direct confrontations on the battlefield.Footnote 53 Despite the proliferation of small arms and light explosives, well-trained forces remain quite lethal when facing disorganized insurgent enemies, as evidenced by the lopsided exchange ratios in recent battles between U.S. troops and irregulars in Mogadishu and Fallujah.Footnote 54
Second, technology is much less important in explaining the emergence or ferocity of insurgencies than is often assumed. Small independent bands of fighters are frequently able to resist despite their lack of advanced weaponry. Indeed, insurgents have a number of distinct advantages over those trying to suppress them.Footnote 55 They possess better knowledge of the terrain than their opponents, and they have shorter lines of communication than do counterinsurgent forces. Nor are additional resources necessarily an advantage for insurgent forces. As Weinstein has argued, well-equipped and financed insurgents may have a more difficult time mobilizing effective fighters because of their inability to separate motivated recruits from opportunists.Footnote 56
Third, the importance of ideology in modern guerilla mobilization is also overstated. As Kalyvas has argued, the leaders of guerilla movements may possess ideological commitments, but for most guerillas and insurgents “local considerations tended to trump ideological ones.”Footnote 57 Fearon and Laitin likewise argue “insurgencies can thrive on the basis of small numbers of rebels without strong, widespread popular support rooted in grievances.”Footnote 58 Loyalty to the group, the esteem of a leader, a desire to plunder and profit, as well as a fear of punishment are among the myriad motivations that can animate insurgents.Footnote 59 During the nineteenth century, simple yet powerful attachments to one's family, clan, or chief formed the basis for much of the observed insurgent activity. In short, nineteenth-century guerillas were not nearly as poorly armed or motivated as is often portrayed. We are thus left with two questions: Why is it that counterinsurgents met with so much success in the nineteenth century, and what can explain the dramatic decrease in their success in the twentieth century?
The Colonial Origins of Counterinsurgent Failure
Reasons for the success of nineteenth-century counterinsurgents, and the struggles of their twentieth-century counterparts, can be found not in strategic choices or technological dominance, but in changes in political context. In particular, during the nineteenth century, the international system privileged a certain type of counterinsurgent—colonial powers. Because of a relatively permissive international context and broad domestic support for imperialism, nineteenth-century wars of colonial conquest and pacification tended to result in incumbent victory. In contrast, normative and material shifts in the international system during the twentieth century undercut colonial counterinsurgents' capacity while providing new opportunities for anticolonial insurgents to challenge them. As a result, colonial incumbents suffered a string of defeats at the hands of their guerilla opponents.
This focus on colonial powers might seem intuitive. After all, they not only dominated international politics in the nineteenth century, but insurgency warfare as well. The nineteenth century featured a rapid expansion of colonial empires as numerous powers scrambled to acquire distant possessions, and by 1913, colonial powers had conquered an estimated 40 percent of the world's territory and 30 percent of its people.Footnote 60 Numerous non-European societies took up arms against their conquerors, and many adopted guerilla strategies. Indeed, prior to World War I, colonial or third-party interveners accounted for 61 percent of incumbents in counterinsurgency conflicts (see Figure 1). In contrast, after World War I, colonial or third-party actors dropped to just 31 percent of incumbents. Central governments became the primary actor engaging in counterinsurgency warfare, as most modern insurgents sought to capture or secede from a recognized state.
FIGURE 1. Frequency by incumbent type
It is odd then that most analyses of counterinsurgency outcomes fail to distinguish between types of incumbents—whether counterinsurgents are nation-states, colonial powers, or third-party interveners—or note the significant shift in who fights counterinsurgencies between the nineteenth and twentieth centuries.Footnote 61 This distinction is critical because different types of incumbents vary considerably in their capacity to defeat insurgent opponents across different historical periods. Specifically, the loss rate for colonial and third-party incumbents skyrocketed from 16 percent before World War I to 57 percent afterward (see Figure 2). In comparison, the loss rate for domestic incumbents exhibited a much more modest increase from 11 percent to 27 percent over the same period. While all incumbents experienced a decline in effectiveness against insurgents in the twentieth century, colonial powers were particularly disadvantaged.
FIGURE 2. Loss rate by incumbent type
These trends suggest that the decline in counterinsurgent success may be driven, not by strategic or technological change, but by changes in the type of incumbent. To test this claim, I reexamined the data found in the Correlates of Insurgency data set, developed by Lyall and Wilson.Footnote 62 Following Simpson, I classified cases of insurgency into three categories: (1) domestic insurgencies (164 cases), where the incumbent was the central government of a recognized state and the insurgent group was located within the state's existing political boundaries; (2) third-party insurgencies (6 cases), where the incumbent was an external power that had intervened to support the government of a recognized state against a domestic insurgency; and (3) colonial insurgencies (116 cases), where the incumbent was a state that was establishing or defending a possession outside its recognized territorial boundaries against local insurgents.Footnote 63 I then coded a dichotomous variable colonial, which was coded 1 if the incumbent was a colonial government or third-party intervener, and 0 if the incumbent was a central government.Footnote 64
To test the hypothesis that twentieth-century colonial insurgencies are particularly difficult to defeat, I generated a dichotomous variable colmod, which was coded 1 if the insurgency involved a colonial or third-party incumbent, and the insurgency was fought after 1918. The choice of the end of the World War I is justified by the fact that this conflict resulted in a profound transformation of the perceived value and legitimacy of colonial rule. I then reran Lyall and Wilson's full model (1800–2005) with this new variable included. This model includes control variables for regime type, external support, an incumbent's military power and economic capacity, terrain, and distance. As predicted, the coefficient for colmod is negative and statistically significant (see Model 1 in Table 1). In contrast, the variable measuring the general shift from the nineteenth to the twentieth century is negative, yet not statistically significant. This result suggests that it is the particular shift in the nature of colonial insurgencies, not a more general shift in the nature of military technology or insurgent lethality, that can account for the decline in incumbent success from the nineteenth to twentieth centuries.
TABLE 1. Colonial context and counterinsurgency outcomes
Notes: Robust standard errors clustered on country in parentheses.
† p < .10;
* p < .05;
** p < .01;
*** p < .001.
I also reexamined Lyall and Wilson's specific models for pre- and post-1918 insurgencies. When focusing on the nineteenth-century period, the colonial variable is negative but not statistically significant. This suggests that colonial insurgents were no more difficult to defeat in the nineteenth century than their domestic counterparts (see Model 2). When examining the post-1918 period, however, the colonial variable is negative and statically significant (see Model 3). As predicted, this suggests that colonial insurgencies were more difficult to defeat than their domestic counterparts during the twentieth century.
Explaining Colonial Futility
These findings provide general support for the hypothesis that a shift in colonial powers' effectiveness was responsible for the decline in incumbent victory. Yet it remains unclear why colonial incumbents did so well in the nineteenth century, but fared so poorly in the twentieth century. I argue that the explanation can be found in a series of four reinforcing shifts in the character of international politics that both eroded the capacity of colonial incumbents to adopt effective counterinsurgency strategies, and opened up new opportunities for anticolonial movements to use violence to achieve their political aims.
Permissive international system
First, the normative features of the international system shifted in a way that disadvantaged colonial incumbents. In the nineteenth century, colonial incumbents operated in a relatively permissive international environment in which colonial domination was perceived as legitimate. European great powers negotiated bilateral and multilateral agreements in which they demarcated their colonial claims and laid out standards for effective occupation.Footnote 65 International legal standards were developed to provide justifications for colonial domination.Footnote 66 European powers competed with one another to carve out spheres of influence, but they generally refrained from funneling assistance to guerillas fighting rival colonial powers. Multilateral agreements—such as the 1890 Brussels Act that placed restrictions on the sale of firearms and ammunition—were negotiated to deprive non-European populations of the tools of resistance. In such an environment, anticolonial insurgents found few international backers while colonial powers conspired to reinforce one another's claims.
In contrast, the environment in which twentieth-century colonial powers operated was much more constricted. Beginning with U.S. President Woodrow Wilson, norms of self-determination began to be accepted as the proper standard by which to judge the legitimacy of governance institutions.Footnote 67 With the mandate system, the League of Nations accepted the principle that colonial dependencies would eventually transition into sovereign states. These trends only accelerated in the aftermath of the World War II. The United Nations (UN) itself was founded on the principles of state sovereignty and nonintervention. The UN General Assembly repeatedly rejected colonial domination as incompatible with the principles of the international system. Most importantly, both of the superpowers rejected the legitimacy of colonial empires. Communist countries in particular proved willing to provide assistance to national liberation movements that sought to throw off the yoke of colonial domination. In this environment, colonial incumbents found it harder to justify the repression of subject peoples, while insurgents found numerous suitors willing to provide normative and material support.
Domestic political will
Counterinsurgency theorists have long argued that guerilla wars depend on a balance of resolve. If an incumbent is to defeat an insurgency, it must have the domestic will to invest the blood and treasure to outlast its guerilla opponent. Most theorists assume that third-party incumbents are at a disadvantage in the balance of resolve, because domestic politicians and populations are unlikely to see wars in the periphery as worth fighting.Footnote 68 Yet in the nineteenth century, policymakers and publics in colonial powers placed a much greater value on their overseas possessions, and as a result, they were much more willing to expend precious resources acquiring and defending them against insurgents.Footnote 69 Economic interest groups viewed colonial possessions as lucrative sources of natural resources and as potential markets for products. Military interest groups sought to acquire colonial possessions to protect strategic sea-lanes and railways. Diplomats viewed imperial possessions as an important source of prestige and as critical compliment to their European diplomatic maneuverings. It is not surprising that aspiring colonial powers were willing to embark on costly and lengthy counterinsurgency campaigns in anticipation of such lucrative benefits.
It was only in the twentieth century that the perceived benefits of colonial dependencies had eroded, and the core lost the resolve to fight guerilla wars.Footnote 70 Despite investment in “tropical development” projects in the interwar period, most European powers found it difficult to derive significant economic benefits from their possessions. Attempts to fall back on colonial markets after World War II were likewise unsuccessful. Not only were the economic benefits of colonies unforthcoming, military benefits proved illusory as well. Colonial dependencies were difficult to defend and sapped valuable resources away from home defense. Moreover, far from imparting prestige, the everyday cruelty needed to maintain imperial possessions served as an embarrassing distraction. Given the eroding base of domestic support for colonial rule, it is not surprising that colonial incumbents in the twentieth century lacked the will to fight bloody counterinsurgency campaigns.
Collaborators: Finding and recruiting allies
Counterinsurgents do not win by brutality alone, and counterinsurgents depend as much on cooperation as they do coercion to achieve their goals. Collaborators are essential to a counterinsurgent's success because they can provide vital local resources and information that can be used to defeat a guerilla opponent. Collaborators also help legitimate alien rule in the eyes of local populations, thus reducing the attractiveness of insurgent narratives. But finding indigenous collaborators is not an easy task. Their availability depends on the politics of the periphery, particularly the degree of fragmentation in targeted societies. When local political authority is contested, elites lack dense ties with one another and are more vulnerable to counterinsurgent recruiting efforts.
Fragmentation may increase the pool of potential collaborators, but availability is not enough. Counterinsurgents must also be able to provide attractive incentives to collaborators to secure their support.Footnote 71 On the one hand, working with colonial authorities can yield significant benefits, including monetary compensation, personal security, as well as the authority that comes with working for the colonial state. On the other hand, collaboration entails risks: it can generate resentment, invite reprisals, and disconnect one from traditional sources of status and patronage.Footnote 72 The decision to collaborate, therefore, depends on the extent to which an individual is benefiting from the precolonial status quo and the degree to which a foreign power can credibly commit to provide collaborators benefits over the long term.
All things being equal, nineteenth-century colonial states had numerous advantages in recruiting collaborators. They were much more likely to face deeply fragmented societies, which provided conquerors with ample opportunities to forge new relationships with local leaders.Footnote 73 Economic, political, and cultural ties with social-climbing elites also generated bonds of common interest that colonial powers could exploit in service of their political expansion. Moreover, colonial powers had many advantages when seeking to “outbid” indigenous leaders for a collaborator's support.Footnote 74 Colonial authorities found it easier to make credible commitments to distribute spoils to potential collaborators, especially when unequal treaties transferred authority to colonial powers indefinitely. Investments in the colonial state—including the construction of roads and railways, barracks and bureaucracies—likewise provided multiple patronage opportunities colonial authorities could employ to attract local allies.
In contrast, colonial powers in the twentieth century found it much more difficult to retain competent collaborators.Footnote 75 Norms of self-determination led many to question both the capacity and will of colonial powers to preserve their dependencies.Footnote 76 Experiments in self-rule and local governance likewise signaled a long-term shift toward political independence. Flagging confidence at home made it harder for officials on the spot to convince actors that the promises of collaboration would be honored. At the same time, the colonial state no longer represented the most attractive avenue for personal enrichment or political advancement.Footnote 77 Economic elites sought to break out from the shadow of colonial currency and trading blocs. Political elites sought to build new constituencies by exploiting anticolonial nationalist sentiment. Educated classes bristled at the constraints to their advancement within the colonial system. All the while, Cold War competition provided collaborators with access to alternative sources of foreign patronage in the form of American or Soviet assistance. In this environment, colonial powers struggled to find influential elites willing to risk backing the colonial state against popular liberation movements.
Choosing when to fight
The process by which colonial incumbents selected fights against insurgents shifted. During the nineteenth century, in particular, colonial powers had a much greater ability to pick and choose when to fight insurgent enemies.Footnote 78 While the map of European colonial possessions ultimately stretched over much of the world, this process was far from linear or progressive. Aspiring colonial powers could choose to press their claims when they had a reasonable expectation of local support. Alternatively, they could moderate their ambitions and make strategic concessions when conditions appeared unfavorable. Nineteenth-century colonial incumbents, therefore, had considerable political flexibility to employ their scarce military resources on favorable terms along contested frontiers.
In contrast, twentieth-century colonial incumbents had much less political flexibility when responding to potential insurgent threats.Footnote 79 Because they had already staked formal imperial claims, they could not easily walk away from insurgent challenges without suffering blows to their prestige. Colonial governments also worried that granting concessions to insurgents in one territory might encourage similar unrest in neighboring colonies.Footnote 80 As a result, twentieth-century incumbents lacked the ability to pick and choose their fights or to scale back their ambitions in response to local setbacks. Each challenge had to be met with maximum response lest it set a negative precedent and spark further resistance. Colonial powers in the twentieth century also had fewer political alternatives to resolve insurgent grievances. In many cases, the only political options available to colonial authorities were repression or decolonization. While colonial powers in the nineteenth century could renegotiate the terms of a dependent relationship, those in the twentieth century often faced the unpalatable choice between a humiliating defeat on the one hand and an inglorious retreat on the other.
In sum, the political context in which colonial incumbents dealt with insurgents was much less permissive in the twentieth compared to the nineteenth century. The normative structure of the international system turned against foreign rule. Domestic support for colonial campaigns flagged. Reliable local collaborators declined in reliability and availability. And colonial powers had less flexibility to pick and choose their battles. These trends were not isolated from one another, but rather interactive and self-reinforcing. Growing calls for self-determination, for example, raised the costs of collaboration for local elites, who could now be branded as imperial stooges. Waning metropolitan support for colonial development reduced the patronage opportunities available to purchase local collaborators' loyalty. Elites who had been disenfranchised by the colonial state used opportunities provided international institutions to spread nationalist messages. Taken together, these four trends deprived twentieth-century colonial powers of the time and resources necessary to develop effective counterinsurgency strategies, while simultaneously affording anticolonial insurgents the support required to sustain resistance. The decline in incumbent success, therefore, has less to do with military strategy, and more with broader shifts in the material and normative context in which different types of incumbents operated.
Testing the Colonial Futility Argument
To examine the plausibility of these hypotheses, I first revisit the data on colonial insurgencies, generating proxy measures for each of the four causal mechanisms proposed in the previous section. I find initial support for most—but not all—of the proposed mechanisms. Second, I examine one case of counterinsurgent victory drawn from the British experience in southern Africa. This case facilitates a more fine-grained test of competing theories of counterinsurgent victory, and provides strong support for the argument that the material and ideational factors favoring colonial states in the midnineteenth century paved the way for British success.
Revisiting the Data
I proposed four mechanisms to explain the marked decrease in colonial effectiveness in counterinsurgent warfare—an increasingly hostile international environment, a decline in the will of colonial powers, a reduction in the availability and reliability of local collaborators, and an inability of colonial incumbents to pick and choose their battles. These factors are difficult to measure directly, but proxy variables provide some plausible support for their validity.
Consider the hypothesis that colonial possessions declined in value over time. Ideally, one would gather data on the economic or political value of specific colonial possessions for individual colonial powers. While economic figures are available for certain colonial powers such as Great Britain, the coverage is limited to more recent periods and select regions. Instead, I generated an alternative indicator of incumbent interest—coltrend—that measured whether the colonial incumbent had increased or decreased its total number of colonial possessions in the ten years prior to the start of an insurgency.Footnote 81 As a proxy for incumbent motives, this variable is problematic—trends in colonial expansion may reflect an incumbent's capacity for suppression, as much as its appetite for new possessions. Yet all things being equal, incumbents adding to the size of their colonial empire were more likely to value their colonial possessions than those shedding dependencies. The resulting variable corresponds to our intuition that nineteenth-century colonial powers were more motivated than their twentieth-century counterparts. Between 1800 and 1918, forty-five of the sixty-seven colonial incumbents (67 percent) were expanding colonial powers. In contrast, between 1918 and 2005, only nine of the forty-nine colonial incumbents (18 percent) oversaw enlarging empires. Using this proxy variable, I reran Lyall and Wilson's full model (1800–2005) for all 116 cases of colonial insurgencies (see Model 4 in Table 2). As predicted, the coltrend variable is both positive and statistically significant. This finding supports the hypothesis that motivated colonial incumbents are more likely to defeat insurgent adversaries than ambivalent colonial incumbents whose empires were static or in decline.
TABLE 2. Sources of colonial incumbent failure
Notes: Robust standard errors clustered on country in parentheses.
† p < .10;
* p < .05;
** p < .01;
*** p < .001.
To test the hypothesis regarding the permissiveness of the international environment, I generated the variable—colheg—that measured the extent of colonial possessions possessed by the hegemonic power in the system.Footnote 82 Following Gartzke and Rohner, the assumption is that the more dependencies a hegemon possesses, the more likely they are to use their political and economic might to establish institutions and norms that favor the maintenance of colonial domination.Footnote 83 The resulting variable corresponds to our prediction that the nineteenth-century international system was more favorable. Between 1800 and 1918, the hegemonic power—Great Britain—possessed an average of 54.5 colonial dependencies. Not surprisingly, Great Britain played an integral role in establishing the normative and legal frameworks to support colonial expansion. In contrast, between 1918 and 2005, the successive hegemonic powers—Great Britain and the United States—possessed an average of 38.4 colonial dependencies. Using this proxy variable, I reran Lyall and Wilson's full model (1800–2005) for all 116 cases of colonial insurgencies (see Model 5). As predicted, the colheg variable is both positive and statistically significant. This finding suggests that colonial incumbents found it easier to defeat insurgent adversaries when the dominant hegemon in the system possessed a large colonial empire, and provided legitimacy to the establishment and retention of colonial possessions.Footnote 84
Finding an appropriate proxy for a colonial power's capacity to recruit collaborators proved more difficult. Ideally, one would gather information on patterns of local collaboration—the proportion of indigenous soldiers in colonial armed forces, the percentage of local representation in colonial bureaucracies, and so forth. This information is available for some colonial possessions, but the historical data is sketchy, especially for the nineteenth century. As an alternative, I created the variable precol, which measured the level of precolonial development of a targeted society prior to colonial conquest, based on criteria established by Lange, Mahoney, and Hau.Footnote 85 The logic of this particular proxy is that colonial powers should have an easier time cultivating collaborators in societies with low levels of precolonial development. The absence of bureaucratized authority structures and robust bonds of collective identity should make it easier for colonial powers to cultivate local collaborators. Using this proxy variable for colonial penetration of targeted societies, I reran Lyall and Wilson's full model (1800–2005), again confining the analysis to the 116 cases of colonial insurgencies (see Model 6). As predicted, the precol variable was both negative and statistically significant, which suggests that societies with lower levels of precolonial development were more vulnerable to efforts by colonial powers to penetrate indigenous hierarchies and remake local societies.
The fourth proxy concerns colonial powers' capacity to pick and choose when to fight insurgent challenges. I coded a dichotomous variable colconq, which coded whether an insurgency was the result of a new colonial conquest or the result of a rebellion in an established colonial regime.Footnote 86 The logic of this proxy variable is that colonial powers could display a greater degree of selectivity in picking and choosing when to embark on new conquests than they could when responding to rebellions in long-standing colonial dependencies. The resulting variable corresponds to one's intuition about the character of nineteenth- versus twentieth-century colonial insurgencies. Between 1800 and 1918, forty-two of the sixty-seven insurgencies (63 percent) broke out as a result of colonial conquests. In contrast, forty-three of the forty-nine insurgencies (87 percent) that took place between 1918 and 2005 were the result of rebellions in established colonial dependencies. Using this proxy variable for colonial incumbent selectivity, I reran Lyall and Wilson's full model (1800–2005), again confining the analysis to the 116 cases of colonial insurgencies (see Model 7). As predicted, the coefficient for the colconq variable is positive, but the estimate falls short of statistical significance at the conventional p < .05 level (p = .056). This finding provides some support for the hypothesis that selection effects shaped colonial incumbent victory, but is not definitive.
To compare the explanatory power of the various hypotheses of colonial incumbent victory, I reran Lyall and Wilson's full model (1800–2005), including all four proxy variables (see Model 8). The hypotheses concerning the availability of collaborators and selection effects receive the strongest support—the coefficients for precol and colconq are both in the predicted direction and both were statistically significant. In contrast, the hypotheses for incumbent interest and international environment receive less support—the coefficients for coltrend and colheg are both positive but neither achieves statistical significance.Footnote 87 These findings suggest that patterns of precolonial development and selection effects may be more persuasive explanations of colonial incumbent success than the value of colonies or the permissiveness of the international environment. Additional research and the collection of more detailed proxies for incumbent interest and the international context will be necessary, however, to assess the comparative weight of these various explanations.
Taken together, these findings lend general support to the hypothesis that nineteenth-century colonial powers enjoyed distinct advantages over their twentieth-century counterparts. They also provide provisional support for specific hypotheses of why some colonial powers had an easier time defeating their insurgent adversaries—notably a capacity to break down local societies and access local collaborators and the ability to pick and choose one's battles along contested frontiers. To be certain, the proxy measures developed to assess the specific mechanisms could be improved. But the general picture supports the contention that nineteenth-century colonial incumbents operated in a favorable and permissive political context, while twentieth-century colonial incumbents did not.
Case Study: British Counterinsurgency in Southern Africa
To provide a fine-grained test of competing theories of counterinsurgent success, I consider the case of the British in southern Africa in the mid-nineteenth century. There are a number of reasons to consider this particular critical case. First, Lyall and Wilson identify the mid-nineteenth century as the “apex” of incumbent success.Footnote 88 Between 1840 and 1860, for example, incumbents won all twenty-five of their wars against insurgent opponents. If there is any time where the specific mechanisms of counterinsurgent success should be most evident, it is this particular period. Second, the British fought a series of small wars against guerilla opponents in southern Africa during this period. All told, four separate Cape Colony governors struggled to impose order over frontier populations between 1844 and 1854. There is considerable variation, however, in the strategies and techniques colonial authorities employed, allowing for an effective test of how force structure and strategy may have contributed to British success. Third, the documentary record allows a detailed examination of the specific dynamics of pacification and resistance during this period. By examining archival records including dispatches, reports, and private letters, one can trace how specific variables—including mechanization, brutality, and insurgent capacity—directly affected battlefield outcomes.
The British fought and won a number of small wars against a variety of African opponents during this period, yet the most sustained and intense fighting took place between the Cape Colony and Xhosa chiefs along the eastern Cape frontier. The Xhosa people were ruled by a series of “genealogically related but politically independent” chiefdoms.Footnote 89 Many of these chiefs, notably Chief Sandile of the Ngqika clan, resented the presence of British settlers in and around their traditional homelands. On two separate occasions, these tensions spiraled into war. In March 1846, the British sent an armed column across the frontier to capture Chief Sandile, who was accused of organizing the escape of an alleged thief from the custody of a colonial escort. The move sparked a widespread uprising among many of the frontier chiefdoms. The resulting “War of the Axe,” named after the axe stolen by the accused thief, lasted almost two years and required nearly 3,400 British regulars to bring it to a successful conclusion. Similarly, in December 1850, the British dispatched three mobile columns in an effort to capture a Xhosa preacher who had been prophesizing the end of white rule. Once again, the action prompted various Xhosa chiefs to rebel against the colonial authorities. The “Mlanjeni War,” named after the Xhosa preacher, lasted over two years with more than 8,600 British regulars taking part in combat operations.
In each of these cases, the British struggled to develop an effective strategy to meet the threat. With an initial force of a little less than 2,000 men, the British had to protect a colonial border stretching nearly 300 miles.Footnote 90 Not only were they undermanned—the colonial forces were also poorly equipped. Their stockpiles of arms were out of date, and ordinance stores “were but scantily supplied with muskets.”Footnote 91 There were insufficient draught oxen to move artillery pieces into the field, while British frontier fortifications were in a state of “shameful irregularity.”Footnote 92 In contrast, the Xhosa were a formidable potential enemy. The Xhosa could bring an estimated 10,000 warriors into the field, upward of 70 percent of which were armed with muskets.Footnote 93 This initial imbalance raises an important question: What can explain British success against Xhosa insurgents despite significant shortages of men and matériel?
Foraging and logistics
If Lyall and Wilson are correct, British reliance on foraging should have provided them with valuable information about the disposition and strength of rebel forces. There are a number of problems with this argument. To begin with, the British rarely relied on foraging to supply their forces. Like most colonial campaigns, the primary means of supply for British garrisons and field forces were supply columns. Lengthy trains of wagons and horses, escorted by hundreds of troops, would bring provisions forward to frontier posts over poor roads and rugged terrain. This cumbersome system inhibited their ability to conduct an effective counterinsurgency campaign. The British were forced to divert scarce manpower to “large escorts” to protect their “unwieldy commissariat trains.”Footnote 94 Delays caused by inclement weather and enemy ambushes also led to frequent interruptions in the supply line, with the result that British forces were often living “from hand to mouth.”Footnote 95 In certain cases, the British were forced to abandon posts when ponderous supply columns proved unable to reach their destinations.Footnote 96
Logistical difficulties also impeded British offensives. The ability of columns to penetrate into enemy territory, for example, was “crippled by the absence of means to carry forward supplies.”Footnote 97 To compound these shortages, hostile Xhosa warriors adopted the tactic of burning fields as they withdrew.Footnote 98 The inability to forage made advancing columns further dependent on cumbersome supply lines, which limited their ability to act with celerity. In one case, the British obtained intelligence about the location of Chief Sandile but were unable to organize a column quickly enough to intercept him. In a private letter, the Cape Colony governor complained: “It is a great pity our communication is so bad, or I could have made a move at the same time … distances, country, all conspire against military service and combination.”Footnote 99
The continued need to move supplies forward also eroded the cohesion of British forces. Cavalry troops in particular suffered from “hard service, scanty forage and unfavourable weather.”Footnote 100 In extreme cases, British forces broke down due to exhaustion. In May 1851, for example, one British officer reported that the “long and harassing” marches had left “both men and horses” suffering “severely from fatigue.”Footnote 101 A week later, the same commander was forced to call off the pursuit of a body of enemy warriors because “our men and horses were too much exhausted for us to follow them.”Footnote 102 One regiment, which was considered typical, marched an estimated 2,838 miles in its first seven months of duty, an average of nearly 10 miles a day.Footnote 103 Far from being an asset, therefore, the logistical system of the mid-nineteenth-century British army imposed serious impediments to the effective conduct of counterinsurgency operations.
Utility of barbarism
If Arreguín-Toft is correct, the explanation for British success lies in their willingness to engage in barbarism to ruthlessly suppress insurgent Xhosa chiefdoms. The British certainly did not refrain from barbaric practices. During the Mlanjeni War in particular, Governor Sir Harry Smith became a strong advocate for harsh tactics. He encouraged his commanders to punish the enemy through a scorched-earth policy of burning and looting. During one operation in March 1851, for example, the British “destroyed many kraals” and plundered “large quantities of [Xhosa] corn and mealies.”Footnote 104 After a similar raid in April, Governor Smith boasted that the operation had shown “the rude hand of war exercised in a manner to bring home to the savage that a day of retribution must succeed rebellion.”Footnote 105 By winter 1851, the fields in the lower Amatola Mountains were completely denuded of grass and corn, and reports began to filter in about bands of starving women and children fleeing from their homes. Governor Smith admitted “to prosecute war” in this manner “is revolting to the Christian mind … but no other course is open.”Footnote 106
Barbarism alone, however, is an unpersuasive explanation for British success. While Governor Smith's rampaging columns unleashed misery upon the local civilian population, they had little effect on the ability of hostile bands of Xhosa to continue their struggle. Operating from three dispersed locations—in the upper Amatola Mountains, the Waterkloof range, and the Fish River Bush—Xhosa warriors continued to harass colonial supply lines and plunder frontier farmers. Despite the brutality of British tactics, Xhosa insurgents remained a cohesive and effective fighting force throughout winter and spring of 1851. In the absence of concrete evidence that barbarism was reducing the will of the Xhosa insurgents, officials in London recalled Governor Smith. The Colonial Secretary Earl Grey complained of endless offensives that were “entirely barren of useful results,” where “ground thus hardly won could not be retained.”Footnote 107
British officials on the spot were equally skeptical of barbarism's utility. During the War of the Axe, for example, Governor Maitland prohibited the burning of fields, fearing that such a policy might produce “a nation of savages in the desperation of famine … both humanity and policy forbid us to drive them to such an extremity.”Footnote 108 During the Mlanjeni War, British commanders likewise experimented with a policy offering a pardon to insurgents who surrendered their arms and took an oath of allegiance.Footnote 109 Smith's successor, Sir George Cathcart similarly acknowledged “measures of unnecessary harshness provoke resistance.”Footnote 110 In short, barbarism was not the primary strategy employed by the British during this period, nor was it the most effective.
Insurgent weakness
Theories of insurgent lethality predict that the Xhosa should have been defeated because they lacked the technology and skill to function effectively as a fighting force. One should observe examples of disciplined and motivated British regulars easily defeating ill-equipped and poorly motivated Xhosa rebels. As I have already shown, however, Xhosa insurgents had distinct advantages in manpower, and were well equipped with firearms. British officials were astonished at the capacity of the rebellious chiefs to sustain military supplies. Toward the end of the War of the Axe, for example, Governor Maitland reported that the Xhosa “have been in possession of an extraordinary amount of ammunition throughout this war.”Footnote 111
Xhosa warriors also exhibited a considerable degree of tactical sophistication. They became experts at engaging and then withdrawing in the face of enemy fire. In one encounter, for example, a British officer described an attempt to use his skirmishers to engage the Xhosa before they could reposition from their ambush points. As his forces approached, however, “a voice from the adjacent heights invariably warned them of the vicinity, when [the Xhosa] would incline to our right or left flank in a manner that could not be surpassed by the best drilled light infantry.”Footnote 112 The Xhosa also employed reserves to descend on the vulnerable points in advancing British columns. In the Waterkloof, for example, a British officer reported “the enemy took advantage, by having men strongly posted at each defile, holding a considerable reserve, which he evidently applied to any point where the struggle became the hardest.”Footnote 113
Xhosa leaders demonstrated considerable initiative in adapting their tactics depending on the terrain and balance of forces. When outnumbered or operating in rocky mountainous terrain, Xhosa warriors would fire on colonial convoys at a distance from concealed positions. Long firefights would ensue as the British struggled to drive the Xhosa from their hiding places. In one skirmish, for example, a British column expended nearly 1,400 rounds over three hours in an unsuccessful attempt to dislodge an entrenched Xhosa force.Footnote 114 In contrast, when the Xhosa possessed numbers or were maneuvering in dense bush, they would seek to surprise British columns with close-quarters ambushes. In this way, they could nullify the advantages of British firepower.
In the rare cases when hostile chiefs lined up against British regulars in the open field, such as the Battle of Imvane in April 1851, African forces were routed. Similarly, when the Xhosa or their Khoi allies were forced to concentrate in a particular location, as in the defense of the captured Fort Armstrong, the British could bring their artillery to bear and inflict significant punishment. But for the most part, hostile chiefdoms exploited their knowledge of the local terrain to avoid set piece battles, and to harass and wear down their colonial opponents. As one British governor lamented: “these athletic brutes … can cover in an hour a distance it requires soldiers three; they have neither front nor rear, nor commissariat.”Footnote 115 Far from being impotent or unmotivated, therefore, the Xhosa warriors exhibited a considerable degree of tactical discipline and inventiveness.
Explaining British success
British officials and military commanders did not possess a coherent counterinsurgency strategy in place at the start of either the War of the Axe or the Mlanjeni War. Some officials such as Governor Maitland opted to fight on the defensive, hoping that the existing network of frontier fortifications and regular patrols would “create such an uneasiness and insecurity to the invaders” that they would be deterred from plundering colonial farms.Footnote 116 Maitland's defensive strategy, however, failed: Xhosa warriors easily evaded colonial forces and captured cattle with impunity. Other officials such as Governor Smith opted to fight on the offensive, hoping that “rapid and unexpected” advances would keep the enemy “in a state of continued excitement and apprehension.”Footnote 117 As I have shown, however, Smith's strategy was more effective at inflicting misery on the local population than eroding the capacity of elusive insurgent bands.
Over time, however, the British refined their approach by adopting familiar counterinsurgency strategies designed to isolate rebellious chiefs. During the War of the Axe, for example, Governor Pottinger embraced the use of indigenous manpower recruited from local African populations, primarily the Khoi, to help bolster the performance of British regulars. He doubled the size of the Cape Mounted Rifles, a force of irregular cavalry used in convoy and escort duty.Footnote 118 He also authorized the creation of an indigenous police force consisting primarily of loyal Xhosa. As with local security forces in other counterinsurgency campaigns, these indigenous forces possessed skills that were uniquely suited to the local environment. Unlike British regulars, African recruits had experience operating in the dense kloofs and vegetated drifts of southern Africa. They spoke the language and were experienced cattle trackers and rustlers. As one British general noted: “the only mode of effectually punishing the [Xhosa] is by taking their cattle, for which the [Khoi] are admirably calculated … they possess wonderful power of vision, great celerity of foot, and powers of endurance … the British soldiers are not equal to doing this effectually.”Footnote 119 The governor likewise praised the indigenous police, viewing them as “a leading step towards controlling the [Xhosa] by means of their own countrymen.”Footnote 120
The British profited from familiar counterinsurgency strategies in the Mlanjeni War as well. Governor Cathcart, in particular, abandoned punishing raids of his predecessor and instead embraced a policy to “systematically and … progressively” clear rebel Xhosa from their sanctuaries.Footnote 121 As part of this plan, Cathcart proposed the construction of an elaborate network of twenty-five fortifications—a “defensible nucleus”—from which British regulars could operate in conjunction with local irregulars. Cathcart recognized that this strategy might appear “slow and inglorious,”Footnote 122 yet he assured officials in London that the fortifications would impress upon the Xhosa that “their stronghold is not again to be abandoned as soon as marched through.”Footnote 123 To complement his system of fortifications, Governor Cathcart also employed indigenous police forces to ensure territory cleared of insurgents would not be reoccupied, including a “partisan corps” of Mfengu warriors whose sole mission was to “hunt down” rebel bands.Footnote 124 While the British were tied to their supply lines, this force could be “constantly on patrol, with knowledge equal to [the rebels] of all the intricacies of the country.” Similarly, while regular troops could focus on only “one operation at a time,” the police would be able to respond quickly to “sudden outrages” and thus help elicit “the vigilance and prompt aid of the inhabitants themselves in their own cause.”Footnote 125
As predicted, the capacity of British colonial officials to employ effective counterinsurgency strategies was facilitated by the permissive political context in which they operated. First, colonial officials received support from policymakers back home who recognized the Cape Colony's importance as a naval base connecting Britain and India.Footnote 126 Although thrifty mid-Victorian officials complained about the cost of frontier conflicts, they consistently provided the Cape Colony with the funding and manpower necessary to defeat its local adversaries. British regulars were dispatched to supplement the Cape Colony's burgher and native levies. British naval power was mobilized to help move men and matériel forward from Cape Town to the eastern frontier.Footnote 127 In this way, support from home allowed successive governors to amass the manpower and supplies needed to implement effective counterinsurgency strategies.
Second, the British operated in an accommodating international environment in which there was broad support for the acquisition and retention of colonial possessions. None of the major powers intervened on behalf of the Xhosa. Hostile chiefdoms did acquire firearms and ammunition in large quantities, but their primary sources were Cape Colony merchants, not foreign powers.Footnote 128 Given Britain's hegemonic position in networks of global trade and finance, none of her putative rivals were in the position to challenge her capacity to support the war effort. Similarities between British suppression of the Xhosa and equally brutal campaigns by the French in Algeria or the Russians in the Caucuses likewise muted any criticism of British policy on normative grounds.
Third, the British were able to exploit social ties with frontier chiefs, as well as divisions between rival chiefdoms, to recruit a reliable cadre of local allies and collaborators. As I have shown, indigenous military manpower proved critical in British efforts to defeat the rebellious Xhosa chiefs. All told, Africans provided upward of 70 percent of the total manpower available to the colonial authorities.Footnote 129 Frontier collaborators were indispensable in this recruitment effort. During the War of the Axe, for example, the Mfengu accounted for more than a fifth of the indigenous contribution.Footnote 130 Local collaborators proved even more important during the Mlanjeni War, when the British secured valuable assistance from many of the southern Xhosa clans. Chief Phatho of the Gqunukhwebe-Xhosa, to take one prominent example, provided some 1,400 warriors to fight alongside colonial forces, as well as to protect vulnerable supply routes from the coast to interior forts.Footnote 131 Other chiefs, such as Chief Toyise of the Ndlambe-Xhosa, provided warriors to protect mission stations and scouts to accompany colonial forces.Footnote 132
Fourth, the British were able to choose their fights, thereby freeing up resources to implement an effective counterinsurgency campaign. During the Mlanjeni War, for example, the British cut various side deals with Griqua and Sotho chiefs residing along the northern frontier, in order to concentrate their military assets against the hostile Xhosa chiefs to the east. The British likewise reached accommodations with neutral Xhosa clans, such as the powerful Gcaleka, in order to focus their war effort against hostile clans, such as the Ngqika.Footnote 133 The British also took advantage of flexible and undefined war aims to enhance their counterinsurgency strategy. During the War of the Axe, for example, the British granted major concessions to hostile chiefs to deprive the war party of broad support. Chiefs Tyali and Bhotomane, for example, laid down their arms in exchange for the right to reside on fertile land in the vicinity of the Tyumie River.Footnote 134 While these selective incentives prevented the British from imposing the draconian peace many white settlers lobbied for, they weakened the war party and brought a rapid end to the insurgency. Had the British been committed to a particular territorial settlement or felt obligated to secure a decisive victory, they could not have employed these concessions to such positive effects.
In sum, the British defeated the Xhosa on two separate occasions in the midnineteenth century. But their victories were not due to British strength of arms or superior skill over weak and incompetent opponents. In both cases, the British were initially checked by an enemy that was well equipped, superior in numbers, and operating on familiar terrain. In both cases, the British found themselves fighting guerilla conflicts for which their regular forces were ill suited and unprepared. Yet in each of the cases, colonial officials had sufficient time and resources to develop effective counterinsurgency strategies. The frontier wars threatened to become total war of white against black. By taking advantage of favorable political circumstances, the British were able to confine these wars to isolated theaters, deprive rebels of the freedom to operate, and ultimately secure victory.
Conclusion
In contrast to existing explanations of incumbent victory, which posit a shift in the character of counterinsurgency warfare from the nineteenth to the twentieth century, I have argued that there has been a fundamental continuity in counterinsurgency strategies. The primary challenge of incumbents has always been to identify and locate insurgents who hide in sanctuaries or among the population. The primary goal of counterinsurgency operations has always been to restrict guerillas' capacity to conduct military operations by clearing and holding territory. Like their contemporary counterparts, historical counterinsurgents struggled with how to adapt their military forces to this unique and challenging task, as well as to balance repression with conciliation. Similarly, insurgents throughout history have exploited their knowledge of the terrain and their ability to move undetected and to harass their adversaries.
To explain shifting patterns of incumbent victory, I have argued that the political context in which counterinsurgency wars are fought is critical. In particular, most nineteenth-century incumbents were European colonial powers who were highly motivated to acquire new territories, who operated in a relatively permissive international environment in which overseas conquest was viewed as legitimate, and who exploited divisions in and ties to elites in targeted societies to crush local resistance. These factors contributed to a relatively high rate of incumbent success across this period. In contrast, twentieth-century colonial powers struggled to maintain their tenuous grip on power. The international community no longer condoned colonial exploits, while a decline in the availability of local collaborators frustrated efforts by increasingly ambivalent colonial powers to defeat local resistance. As a result, twentieth-century colonial incumbents struggled in conflicts they did not want to fight, had little chance of winning, but could not afford to lose.
These findings have a number of important implications. First, they call into question those who claim that success in counterinsurgency operations is simply a matter of choosing the right strategy or adopting the proper force structure. By contrast, the findings presented in this article suggest that the main challenge of counterinsurgency warfare is not intellectual, but political. Unfavorable conditions, such as those facing twentieth-century incumbents, can deprive incumbents of the time and resources necessary to formulate effective strategies, all while enhancing insurgents' opportunities to exploit violence for maximum political effect. Recent studies support this emphasis on structural constraints over strategic tinkering. Andrade, for example, argues that U.S. failure in Vietnam stemmed less from American strategic myopia than the sheer scale of the challenge posed by the potent combination of Viet Cong guerillas and North Vietnamese main force units.Footnote 135 Hack likewise emphasizes that many of the touted tactical innovations Britain employed to win “hearts and minds” in Malaya took place after a combination of military sweeps and population-control measures had degraded the strength of an already fractious communist insurgency.Footnote 136 In short, sophisticated counterinsurgency strategies are not a panacea when the underlying structure of material and ideational conditions favors insurgents rather than incumbents.
Second, the findings echo conclusions of recent studies, which emphasize the centrality of international factors in shaping patterns of insurgent violence. Kalyvas and Balcells, for example, highlight how shifts in the capacity of states during and after the Cold War influenced the “technology of rebellion” across a variety of civil wars.Footnote 137 Much like this article, they emphasize how superpower competition during the Cold War encouraged the emergence of “robust insurgency,”Footnote 138 whose international support and local legitimacy made them difficult to defeat. Not surprisingly, these movements were especially effective when targeting colonial powers, whose repressive capacities were hamstrung for the reasons described earlier. More work needs to be done to understand the complex ways in which international factors interact with domestic or subnational forces to shape counterinsurgency outcomes across different types of political actors in both the nineteenth and twentieth centuries.
Finally, the findings have important implications for contemporary debates about the role of counterinsurgency warfare in American grand strategy. Some maintain that the U.S. military should continue to plan for and invest in capabilities dedicated to counterinsurgency operations, while others contend that the military should revert to its traditional mission of fighting and deterring wars with conventional opponents. The fate of twentieth-century colonial powers, however, should instill caution in counterinsurgency enthusiasts. Many of the same factors that drove colonial incumbent futility—a hostile international environment, a skeptical domestic public, and a thin base of local support—will likely be present in future manpower-intensive U.S. counterinsurgency operations. Policymakers should keep in mind these inherent limitations and consider alternative tools for achieving their desired ends. Foreign security assistance, for example, can build state capacity before insurgent movements gather in strength. Counterterrorism operations can prevent hostile groups from exploiting ungoverned spaces created by insurgent violence. While neither of these approaches will completely eliminate the danger posed by insurgents, they may be superior to the alternative of fighting difficult, often unwinnable, wars.
