Many Americans have long been concerned that their national security agencies do not work closely enough together and that this problem can have fatal consequences. In the aftermath of the September 11 (9/11) terrorist attacks, numerous political leaders, security experts, and official reports came to the conclusion that various government agencies had significant prior information and leads on the 9/11 plot – but they failed to prevent it because of inadequate coordination.Footnote 1 The US government introduced a number of organisational reforms in response to this failure.Footnote 2 It also went on to deplete the Al-Qaeda core, kill Osama Bin Laden, and develop intelligence that enabled the prevention of many terrorist plots.Footnote 3 Despite this progress, however, there were also several reminders of the ongoing problems in American counterterrorism. Experts and congressional investigations have highlighted inadequate coordination between security agencies prior to the 2009 Fort Hood shooting, the Christmas Day ‘underwear’ bomb plot that same year, and the 2013 Boston bombings.Footnote 4 Why, more than 14 years after 9/11 and despite significant government attention and reform, do these weaknesses in US inter-agency operations and intelligence still persist?
Through a comparison of the United States with the United Kingdom, this article critiques existing answers to this question and provides a new explanation for the deficiencies in the American response, drawing on 34 author interviews and a range of primary sources. The article documents how inadequate coordination and conflict between security agencies in the United States have had damaging effects on its development of intelligence and its operations against Islamist terrorist networks since 2001. The British agencies, on the other hand, have developed a more integrated approach to inter-agency counterterrorism and they tend not to engage in turf battles to the extent that their American counterparts do.Footnote 5 To explain these differences between the American and British cases, I argue that a combination of two factors – state institutions and the organisational routines of security agencies – has led to the development of different approaches to inter-agency operations in each national setting. In the United States, institutional authority is divided between a range of actors including the executive branch, Congress, and state and local government. This has contributed to a proliferation of security agencies with overlapping jurisdictions, and a reliance on informal routines and relationships that often prove inadequate for the purposes of inter-agency coordination. In the UK, a combination of centralised institutions and formal routines has proved more conducive for the development of coordinated inter-agency operations.
The analysis offered here challenges some beliefs that are widely held in both academic and policy circles. Officials and observers of the US government often assume that inadequate coordination and turf battles are an inevitable, if regrettable, feature of the relationship between national security agencies.Footnote 6 Theoretical underpinning for this view has been provided by influential models of bureaucratic politics, which hold that government agencies are strongly motivated by a commitment to their own parochial interests.Footnote 7 From this perspective, clashes of bureaucratic interest make inter-agency turf wars a persistent and intrinsic feature of the national security landscape.Footnote 8 But is this always the case? Are significant levels of inter-agency conflict really inevitable in the field of security? This article suggests that the answer is no. In some countries, such as the United States, we do indeed see agencies behaving as models of bureaucratic politics would expect. In other cases, such as the United Kingdom, however, there is a far lower degree of inter-agency conflict and a higher degree of coordination. We cannot account for this variation, I argue, without reference to institutions and routines. As fundamental features that appear in a variety of national settings, state institutions and organisational routines are analytically prior to bureaucratic politics, which appear in some cases but are largely absent from others.
This article brings a comparative perspective to bear on the ongoing scholarly and policy debate on the coordination of the United States’ response to terrorism.Footnote 9 To my knowledge, this is the first study that seeks to explain the performance of the US on inter-agency intelligence and operations against terrorism - in comparison with that of another Western democracy.Footnote 10 The comparative method can offer additional analytical leverage, enabling one to better identify the precise conditions that have led to different outcomes across the cases.
It is crucial for any national response to terrorism to have an effective coordination of operations carried out by the various security agencies involved in the mission. Indeed, a series of terrorist attacks carried out in 2015 and 2016 highlighted how the coordination of counterterrorism is a salient issue not only in the United States but also in France, Belgium, and many other countries. ‘Coordination’ here refers to the process of organising people or groups so that they work in a mutually supportive way towards a common goal.Footnote 11 Research indicates that inter-agency coordination is one of four elements that are particularly important for an effective operational approach to preventing terrorist attacks, the others being: precise and actionable intelligence; a robust but discriminate use of force; and international cooperation.Footnote 12 Inter-service coordination is important for putting together pieces of information, which may be dispersed across the nation’s security agencies, in order to develop accurate intelligence on the overall threat and particular plots facing the country. Without effective coordination, there is also an increased risk that one agency’s actions may compromise the intelligence, military, or arrest operations of another agency.Footnote 13 Finally, as Bruce Hoffman argues, counterterrorism strategy can only work if it utilises in a coherent way all the elements of national power against the challenge at hand. ‘Success’, he writes, ‘will ultimately depend on how effectively the U.S. can build bridges within our own governmental structure and … improve the ability to prioritize and synchronize inter-agency operations.’Footnote 14
Full coordination may not be appropriate in every context. Public policy scholars such as Aaron Wildavsky and intelligence experts such as Gregory Treverton point out that duplication and redundancy may increase reliability and bring different perspectives to bear on a problem. Yet, as Treverton also points out, the kind of duplication that we observe in the US case is not always this purposeful and is often wasteful and counter-productive instead.Footnote 15
I also recognise the inherent difficulty of preventing terrorist attacks. Terrorist operatives usually operate inconspicuously in small cells, they blend in with the general population and do their utmost not to reveal their plans to others. Consequently, there are, as Paul Pillar puts it, some ‘permanent and ineradicable’ limits on the ability of intelligence and security agencies to prevent attacks.Footnote 16 Good coordination is important for averting some of the more avoidable errors committed by security agencies and increasing their ability to prevent terrorism. But even the best-organised security response will never be able to stop all attacks.
This article examines domestic intelligence and law enforcement agencies with responsibility for combatting terrorism on the national territory. There are two main reasons for this focus. Firstly, as Erik Dahl’s research has shown, domestic intelligence and law enforcement activities are key to preventing terrorist attacks against America.Footnote 17 Consider also the nature of the threat. Britain has faced a significant degree of ‘homegrown’ terrorism since 2003–4.Footnote 18 In more recent years, US citizens and residents have played an increasingly prominent role in Islamist terrorist plots against the United States.Footnote 19 Responding to this evolution of the threat, the US government has emphasised how the latest iteration of its counterterrorism strategy is the first one to ‘designate the homeland as a primary area of emphasis in our counterterrorism efforts’.Footnote 20 While it was the domestic/foreign dividing line between the Federal Bureau of Investigation (FBI) and the Central Intelligence Agency (CIA) that stymied efforts to uncover the 9/11 plot, the dividing lines between domestic agencies are of crucial importance in the current context. Given that most Islamist terrorist plots against the US and the UK are being substantially prepared inside their respective homelands,Footnote 21 it is important to study how domestically-focused security agencies work or do not work together to identify and combat such threats.Footnote 22
The United States and the United Kingdom share some important characteristics – as liberal democracies based on the rule of law – and they have both been facing a significant threat from Islamist terrorism. Among Western countries, the US and UK have been the top priority targets of jihadist terrorism for most of the post 9/11 era.Footnote 23 Britain experienced the ‘7/7’ London bombings in 2005, the killing of an off duty soldier, Lee Rigby, in London in 2013 and several other substantial plots by Islamist extremists, which were foiled. In the United States, the Boston bombings and the Fort Hood and San Bernardino shootings caused death and injury, while a number of substantial plots have been intercepted.Footnote 24 Overall, the US and Britain have faced a broadly similar threat from Islamist terrorism during the period under study here.
The United States’ national security bureaucracy is much larger than that of Britain. However, any argument that size explains the two countries’ records on inter-agency coordination does not hold up, especially if we expand our comparative frame of reference. Studies have shown how – similar to the United States – the coordination of inter-agency operations has been problematic in two other prominent cases: France and Germany.Footnote 25 Yet both of these countries have national security bureaucracies of a comparable size to Britain – not to the US. If size was a key factor, we should not observe similar outcomes in America, Germany and France, with a different outcome in Britain. The fact that we do see such outcomes indicates that other factors are more important.Footnote 26
While the main objective of this article is to shed new light on a national security problem, the analysis also has broader theoretical implications. This is because the counterterrorist policy field can be seen as a hard test for institutional and organisational routine theories, particularly for their claim that historical legacies shape current policy in ways that reduce the likelihood of efficient outcomes. One might have assumed that such inefficiencies would have been stamped out more than 14 years after 9/11 – given that the protection of citizens from Islamist terrorism is one of the highest priorities of the US government. This has not been the case, however. Since a focus on institutional legacies and organisational routines explains much about how government performs, not just on low-level issues, but also in this top priority area of policy, I argue that the theories have passed a difficult test.
The article proceeds in five steps. The first section critiques both bureaucratic politics explanations of security agency behaviour and a second widely-used approach – that of organisational culture. I outline my alternative analysis in the next two sections, which discuss in turn the role of national institutions and the influence of organisational routines in the field of security. These two sections are structured similarly, each beginning with a theoretical discussion, followed by an outline of the relevant institutions or routines in the US and British cases respectively. It is the interaction of these two factors, I argue, that best explains the performance of the two countries on the development of inter-agency intelligence and operations against Islamist terrorism. Evidence is presented for this argument in the case of the United States in the fourth section and Britain in the fifth section. The conclusion presents the implications of these findings for theory and practice, arguing that an emphasis on institutions and routines offers not only a new explanation but also provides a realistic framework for analysing the viability of proposals for national security ‘reform’ in the United States. The article draws on a range of sources, including author interviews with 34 current and former counterterrorist officials in the US and the UK.
Beyond bureaucratic politics and organisational culture
When asked to account for organisational problems in field of national security, analysts often refer to either the bureaucratic politics of security agencies or to negative cultural characteristics of these agencies. The most important proponent of the bureaucratic politics paradigm in recent decades has been Graham Allison, starting with his widely-read 1971 study of the Cuban missile crisis, Essence of Decision. Though this work actually contained three models, it was Model III – ‘Government Politics’ – that proved most influential.Footnote 27 This model holds that government officials are ‘substantially affected by’ the worldview and bureaucratic interests of their own particular department or agency.Footnote 28 Drawing on Allison, Matthew Kroenig and Jay Stowsky write that government agencies are constantly seeking ‘to protect their own parochial interests’ and maximise ‘their own autonomy, resources and prestige’. Applying this perspective to counterterrorism, they trace how the FBI and other agencies protected their own interests in domestic intelligence in the years after 9/11.Footnote 29 In a similar vein, Amy Zegart writes that US national security officials engage in a ‘zero-sum battle for agency autonomy and power’. For her, this strong commitment to narrow bureaucratic interest partly explains the prevalence of inter-organisational conflict, the inadequacy of inter-agency coordination and the failure of the US government to effectively reorganise its intelligence agencies after 9/11.Footnote 30 Yet, while such explanations capture some of the dynamics of government, it may be misleading to place them centre-stage in one’s analysis. A comparison of security agencies in different national settings reveals the limitations of a bureaucratic politics perspective. It cannot explain why organisations with similar bureaucratic interests do not act in the same way in the field of inter-agency counterterrorism. In particular, if the model was accurate, we should see Britain’s security agencies engaging in turf battles to maximise their interests. However, as a subsequent section of this article will show, we do not observe such battles.Footnote 31
A second widely-accepted analysis is that the organisational cultures of US security agencies are at least partly responsible for the outdated or ineffective aspects of their response to terrorism.Footnote 32 Culture is usually understood to be the values, beliefs, and identities of an organisation’s employees. While these cultural analyses have offered useful insights into some issues,Footnote 33 they have not provided a robust explanation for why inter-agency coordination problems persist in the United States. For example, in Amy Zegart’s analysis of the CIA, one of the main ‘cultural pathologies’ identified is a ‘debilitating sense of agency parochialism’, according to which CIA personnel developed a strong sense of loyalty to their home agency, rather than to the intelligence community as a whole. Zegart argues that this parochial culture partly explains why some CIA officials avoided sharing information with other agencies prior to 9/11. In other words, inward-looking behaviour is said to stem from an inward-looking or parochial ‘culture’. This explanation conflates the dependent and independent variables. Similarly, ‘resistance to change’ is identified as a characteristic of CIA culture and used to explain the agency’s failure to undergo meaningful change or reform.Footnote 34 In these sections of Zegart’s analysis, it seems that ‘culture’ is just shorthand for bad practices. Furthermore, it is not clear that the concept of organisational culture can help us to explain variation across national settings. Why does Britain have less coordination problems than the United States – because it has a ‘good’ or unified inter-agency culture? And what would explain why the culture of the British agencies is good, while that of the American agencies is ‘bad’ or parochial? Rather than focusing on broad conceptions of culture, it may be more fruitful to base our analysis on more specific concepts that have been developed in the literatures on institutions and organisational routines.
How national institutions affect security agencies
The next two sections will trace the relationship between the ‘macro’ institutions of the nation state and the ‘micro’ organisational processes of security agencies. I draw on the ‘institutionalist’ literature, which argues that, at the very least, institutions can be understood to comprise formal rules and procedures, such as ‘the rules of a constitutional order’ and ‘the standard operating procedures of bureaucracy’.Footnote 35 One strand of the literature – sociological institutionalism – stresses how these rules and procedures are underpinned by legitimising ideas or norms.Footnote 36 For example, as will be outlined below, the rules and procedures of the US constitution concerning the separation of powers cannot be considered in isolation from the anti-statist ideas and norms of American society that underpin them. For the purposes of this article, institutions are defined as the formal rules, procedures, and political norms which regulate the relationships between units of government (such as the legislature, executive branch departments, and state and local governments).
Institutions are circumscribed by their historical origins. As Lynn Eden has written, the institutionalist literature ‘do[es] not assume rational, efficient or adaptive outcomes’ but instead stresses ‘how older ways of understanding and acting persist’ and shape governments’ solutions to the problems they face.Footnote 37 Institutions thus develop in a ‘path dependent’ manner, as Walter Powell explains: ‘Choices made at one point in time create institutions that generate recognizable patterns of constraints and opportunities at a later point’, he writes. Outcomes cannot be explained simply by ‘the preferences of actors … but must be explained as the product of previous choices’.Footnote 38 Institutionalists analyse how such choices can become self-reinforcing and may even become ‘locked-in’ or resistant to radical reform. Institutions can and do change but new developments will usually be broadly compatible with and follow the same logic as the existing institutional order.Footnote 39 These theoretical expectations are well exemplified in the case of the United States’ political institutions.
America’s anti-statist institutions
The long-standing American suspicion of state power has a profound effect on its institutions of government. Anti-statism has been an important feature of US political life from the foundation of the republic to the Cold War and the post 9/11 period.Footnote 40 Reflecting this widespread wariness of the excessive concentration of power, the Constitution sets out how institutional authority in the United States must be divided.Footnote 41 Building on work by Amy Zegart and Martha Crenshaw, we may identify three types of division and decentralisation in US institutions that are particularly relevant to counterterrorism. First, regarding the separation of powers between the three branches of national government, Zegart has shown how the presence of a strong legislature, independent of the executive branch, means that the design and reform of national security agencies is subject to messy political compromises.Footnote 42 Second, as a federal union, the United States has multiple levels of government – national, state, and local – which reflect the political importance of the states as well as a long-standing determination to forestall the development of an excessively powerful central government.Footnote 43 While the US has a national investigations bureau – the FBI – it does not have a national police force partly because of fears that this would represent an excessive concentration of coercive power on the domestic scene.Footnote 44 This power is decentralised rather to state and local governments, which have their own police departments. Apart from constitutionally-ordained diffusions of power, a third key factor is the tendency towards fragmentation even within the national executive branch. Responsibility for dealing with terrorism is widely distributed across the federal government to agencies such as the FBI, the Department of Homeland Security (DHS), and the Department of Defense (DOD).Footnote 45 Indeed, there is a general tendency towards the proliferation of executive branch agencies in the field of security, whether it is America’s 16 intelligence services, for example, or its various federal law enforcement organisations.
The divided nature of US institutions has three effects of particular interest to us here. Firstly, the combination of federalism and a fragmented executive branch has produced a large number of domestic police and intelligence agencies with important roles in the American response to terrorism. Secondly, divided institutions have also led to a dispersion of authority in the field of counterterrorism. Thirdly, messy political compromises in Congress or between Congress and the executive branch have contributed to a situation in which the jurisdictions of security agencies often overlap or are unclear. Within the legislative branch, various congressional committees seek to maintain their own importance by ensuring that the security agencies and departments that they oversee play a role in crucial missions such as counterterrorism. This favours dividing such missions or creating overlapping jurisdictions so that multiple agencies and congressional committees all get a ‘piece of the action’.Footnote 46 These three features of US counterterrorism are considered in more detail below.
Britain’s centralised institutions
Britain has a more centralised set of institutions than the United States, organised around a powerful executive branch, which is usually supported by its majority in Parliament.Footnote 47 Since 1999, Westminster has devolved considerable powers to elected assemblies in Wales, Scotland and Northern Ireland, as well as to local government. Yet this reform did not lead to any major changes in the institutional structures and rules most relevant to policing, counterterrorism, and national security.Footnote 48 In the area of general law enforcement, there is a network of 51 police forces spread across Great Britain. However, counterterrorist law enforcement has long been organised from the centre by the London Metropolitan Police. The domestic intelligence agency, the Security Service – or MI5 – is responsible for developing intelligence on terrorist threats. Both the police and MI5 report to the UK’s interior ministry, the Home Office, and there is little evidence in the British government of the kind of fragmentation that one observes in the American executive branch.Footnote 49
In the context of its more centralised institutions, Britain differs from the United States along three key dimensions. The number of agencies with important roles in the British domestic response to terrorism is lower than in the United States, while authority over counterterrorism in the UK has been concentrated in central government. Thirdly, in the context of a generally acquiescent legislature (unlike the assertive American Congress), the parliamentary majority tends to follow the government’s lead on security issues, avoiding the need for the sorts of political compromises that have led to overlapping agency jurisdictions in the US. We shall return to these three features of UK counterterrorism in the next section.
The organisational routines of security agencies
National institutions help to form the structures and routines of counterterrorist agencies. Yet these organisational routines also take on a significance and a momentum of their own, which has a major impact on the quality of inter-agency responses to terrorism. For James March, organisational action stems less from a logic of consequences (the considered weighing of alternatives, envisaged by rational choice theory) and more from a logic of appropriateness. This means that organisations tend to resort to pre-existing repertoires of action when they recognise a situation ‘as being of a familiar, frequently encountered, type’.Footnote 50 Where organisational responses are marked by the appearance of such recurrent patterns of action, March considers them to be instances of ‘routinized’ activity.Footnote 51 Such organisational routines are commonly defined as ‘recurrent interaction patterns’ between multiple actors within and across organisations.Footnote 52
Similar to national institutions, organisational routines tend to develop in a path-dependent manner. Markus Becker emphasises that while routines can change in response to challenges in the external environment, choices made in the past also have ‘feedback effects’, which favour the continuation of certain routines and make the development of others less viable.Footnote 53 Actors also tend to reproduce organisational routines in habitual and unreflective ways.Footnote 54 Such routines are all the more powerful because they are not up for debate and are taken for granted in their particular contexts. It is through these path-dependent and habit-based mechanisms that historically-grounded routines shape organisations’ responses to contemporary challenges.Footnote 55
Organisational routines are context-specific and may ‘strongly differ’ across cases.Footnote 56 Considering the routines of counterterrorist agencies in various national settings, the key differences between them may be captured in the concepts of formal and informal organisational routines – two terms that have a specific meaning in the context of this study. In this article, a formal routine is indicated by the presence of regularised interaction patterns between agencies, based on rules laid down by a central authority. Conversely, an informal organisational routine is indicated by the presence of irregular interaction patterns between agencies, based on interpersonal relationships.Footnote 57 I argue that the American counterterrorist agencies rely for the most part on informal organisational routines, while their British counterparts’ routines are formal in nature.
Macro institutions hold an important key to understanding why micro organisational routines may differ across national settings. As outlined above, national institutions have an important influence on three key variables; what we may call the antecedent structural conditions of organisational routines. These are (each followed by two ideal types): (i) the number of core counterterrorist agencies in the country (few/many);Footnote 58 (ii) the nature of their respective jurisdictions (distinct/overlapping); and (iii) the distribution of authority between agencies (concentrated/dispersed). Variation in these structural conditions gives rise to different types of organisational routines, which in turn shape levels of inter-agency conflict and cooperation. The four stages of this analysis are outlined in Figure 1.Footnote 59
Figure 1 A model of inter-agency counterterrorism.
In the remainder of this section, I outline how the three structural conditions helped to shape the development of different organisational routines in the American and British cases.
The United States’ informal routines
The organisation of US domestic counterterrorism is complex and dispersed, with several core agencies playing a central role in the effort. While the FBI has ‘lead responsibility’ for both counterterrorist intelligence and law enforcement, its jurisdiction over these areas overlaps with that of DHS, DOD, and state and local police forces.Footnote 60 As noted above, these overlapping or unclear jurisdictions are sometimes the result of messy political compromises. Thus, in 2002, pressure from Congress led to the foundation of DHS, which was given a broad mission to prevent and respond to terrorism within the United States. The department was also mandated to develop its own Office of Intelligence and Analysis with responsibilities that overlapped with those of the FBI.Footnote 61 That same year, Congress passed legislation that consolidated DOD intelligence activities under a weighty new position, the Under Secretary of Defense for Intelligence. DOD units increasingly gathered terrorism intelligence within the United States, overlapping with the FBI’s intelligence mission. In 2011, following a compromise between the legislative and executive branches, Congress mandated DOD to play a role in the detention of foreign terrorist suspects arrested in the United States, stepping this time into areas of the FBI’s law enforcement mission.Footnote 62 Beyond the federal government – at state and local level – the police forces of large cities such as the Los Angeles Police Department and in particular the New York Police Department (NYPD) also have significant counterterrorist investigative units of their own with missions which overlap with that of the FBI. Overall, as Gregory Treverton has written, there is ‘confusion and ambiguity about the roles of particular agencies’ and ‘uncertainty about who is responsible for what parts of the effort’.Footnote 63 One bureaucratic actor at the centre of government – the Assistant to the President for Homeland Security and Counterterrorism – oversees counterterrorist policy and operations, but she lacks the authority to direct departments and agencies.Footnote 64 Authority over domestic counterterrorism is dispersed rather across the key agencies, most notably, the FBI, DOD, and DHS.
The presence of several core agencies with overlapping jurisdictions, generally not subject to direction by a central authority, are conditions that have given rise to informal organisational routines between the US counterterrorist services. A former senior DHS intelligence official illustrated this dynamic: ‘You do have, legitimately, FBI and DHS components all saying “We have the jurisdiction … to run that investigation.” And really, they’re all right. They’re all correct.’ The response to such jurisdictional overlap, he explained, was to ‘rely on [interpersonal] relationships … you have to negotiate that stuff out, and what I tried to do in my job was have as many meetings and build as many relationships as I could across the country with state and local people, FBI people, National Guard people. You name it, I was out making relationships.’ In order to work with other agencies, he underlined, ‘You gotta meet these guys and they gotta trust you … that you’ll protect their secrets and that you’ll help them as much as you can.’Footnote 65 Though such efforts usually bore fruit, they also meant that interactions between the agencies often relied on ad hoc negotiations and thus did not follow a regular pattern. As one experienced FBI agent put it: ‘A lot of the way in which we work is personality-driven, and so, you could have a Special Agent in Charge [of an FBI Field Office] who has got a great relationship with the Police Commissioner of City X, but once of those guys leaves and they don’t like each other any more, the relationship could sour very quickly.’Footnote 66 Also speaking about informal inter-agency relationships, a former senior FBI counterterrorist agent made a revealing comment: ‘those issues take vast quantities of energy to resolve every day. It’s amazing the tending [to relationships] that is required, and if you ignore it or don’t tend to it appropriately, there are breakdowns, and then lack of cooperation.’Footnote 67 Overall, there was a general recognition among the practitioners who were interviewed that US inter-agency counterterrorism relies heavily on informal routines.
Britain’s formal routines
In Great Britain, from the early 1990s until 2006, responsibility for domestic counterterrorist intelligence and law enforcement lay with relatively few agencies: one domestic intelligence service – MI5 – and two branches of the London Metropolitan Police, which performed distinct missions. The police’s ‘Special Branch’ gathered intelligence on terrorism, while its ‘Anti-Terrorist Branch’ was responsible for law enforcement. The leading counterterrorist official at the Metropolitan Police is also the Senior National Coordinator of Terrorist Investigations and the London force has for decades had a mandate to investigate terrorist activity anywhere in Great Britain, supported by local police forces.Footnote 68 A central authority – the government – issued clear guidelines to the agencies, which stated from 1992 that MI5 was the ‘lead agency’ on terrorism intelligence and that Special Branch’s role was to ‘assist’ MI5 in this area.Footnote 69 These guidelines also stated that Special Branch must provide all of its terrorism intelligence to MI5.Footnote 70 While MI5 has primacy over counterterrorist intelligence, the agency has no competence in law to make arrests or do police-type investigations. Counterterrorist law enforcement is the exclusive competence of the British police. With their respective roles set out in legislation and government guidelines, the law enforcement and intelligence services working on counterterrorism in Great Britain have had a clear understanding of the division of labour between them over the last two decades.Footnote 71
Thus, in contrast to the US, Britain has relatively few core counterterrorist agencies, each of which has a distinct jurisdiction and operates on the basis of rules laid down by a central authority. These conditions have led Britain’s counterterrorist police and its domestic intelligence agency to adhere to a set of formal organisational routines, notably by developing procedures for regularised cooperation between their units. Whereas the interaction between the agencies in the United States tends not to follow a regular pattern, Britain’s MI5 and its police appear to begin with an assumption that their officers will be regularly carrying out operations together. MI5 desk officers have a mandate to task operatives from both their own agency and from police Special Branch. Indeed, MI5 and Special Branch agents have worked together even on sensitive tasks like the recruitment or handling of informants.Footnote 72 When a particular case reaches a critical point, an Executive Liaison Group (ELG) of MI5 and police investigators is formed to make an operational decision on the case. Reflecting the police’s lead responsibility for law enforcement, the chair of the ELG is a police officer and it is the police who have the final say on whether, when and how the suspects will be arrested.Footnote 73 As we shall see below, the formal routines of the British agencies were reinforced in the years after 9/11.
Explaining levels of inter-agency coordination in the United States
The next two sections will treat of the United States and Britain in turn, tracing how their divergent institutions and organisational routines have shaped the degree of inter-agency cooperation and conflict found in the two cases. In the US, significant coordination problems have affected intelligence work and operations against terrorism, according to several expert reports by organisations such as the Government Accountability Office, the Project on National Security Reform and the Bipartisan Policy Center (BPC).Footnote 74 In a 2011 report, the BPC’s national security group, which is a follow-on from the 9/11 Commission, echoed a widely-held view when it identified an improvement in cooperation between the CIA and the military against terrorism. It cautioned, however, that ‘on the domestic side, there has been less unity of effort and much slower progress among multiple agencies’.Footnote 75 Such assessments are borne out when we examine the relationships between arguably the four most important agencies in US domestic counterterrorism: the FBI, DHS, DOD, and the NYPD. This section considers firstly the relationship between FBI and DHS; secondly, the FBI’s interaction with DOD (including the case of the 2009 Fort Hood shooting); and thirdly, the FBI’s coordination with the NYPD (including their prevention of the Najibullah Zazi-led plot to attack New York in 2009). It will also assess the significance of post-9/11 reforms and improvements to information-sharing among US agencies. The FBI’s prominence in the analysis reflects its role as the agency with lead responsibility in US domestic counterterrorism – but one that takes part in a complex set of relationships as several other organisations play an increasingly important role.
FBI and DHS
In the years following the establishment of DHS in 2002, its Office of Intelligence and Analysis (I&A) became a member of the US Intelligence Community and began working on domestic terrorism intelligence. The FBI remained the lead federal agency for terrorism intelligence within the United States. However, the broad and unclear nature of the DHS intelligence mission and its overlap with that of the FBI enabled the new organisation to take an expansive view of its mission. A CIA veteran, Charlie Allen, was appointed head of I&A in 2005 and, according to a close colleague of his at DHS, ‘[Charlie] wanted to do everything’ in the field of terrorism intelligence analysis, from ‘borders’ to ‘prison radicalization’ to ‘homegrown terrorists’ and beyond. It was ‘a very big agenda’. Yet even as I&A moved into aspects of intelligence analysis traditionally conducted by the FBI, he admitted, ‘we didn’t have good connectivity with each other’. Sometimes ‘there was duplication of effort’ between DHS and FBI which ‘could have been avoided if we’d had better cooperation’. He wondered ‘whether we were all doing the same things in a vacuum: whether we’re working on a Somali issue for Columbus, Ohio; they’re working on a Somali issue for Columbus, Ohio. So you’ve got two teams doing the same thing. Would it be better if maybe they can combine?’Footnote 76
This duplication of effort was confirmed by one FBI counterterrorist agent, though he doubted the value of coordination with the DHS intelligence office: ‘[DHS/I&A] were trying to get their oars in the water in a way that I think was repetitive and actually hurt the mission, because people were spending time away from actually doing things to now having to coordinate with somebody who really had nothing, no value added.’Footnote 77 At one point, some I&A officials felt that they were receiving insufficient information from the FBI, he added, and ‘there was a lot of animosity’. If they did get some information, they would ‘start digging along by themselves’, he said. When this intelligence work was presented back to the FBI, it would not go down well, according to the agent: ‘The FBI guy would [say], “Well, we actually knew that. We’ve been doing this forever, you should have asked us to begin with, now please stop.” There was still a lot of that going on.’Footnote 78
Charlie Allen left his post at I&A in 2009 and though his successors may not have taken as expansive a view of DHS intelligence as he, the conditions favouring conflict between DHS and FBI remain. Unclear and overlapping missions mean that the two sides have different understandings of the appropriate division of labour between them in the field of intelligence analysis. A former senior official summed up the view from DHS: ‘They [the FBI] are a prosecutorial department; we’re not. We’re an intelligence department.’ DHS had to remain engaged in mainstream terrorism intelligence, he believed, because it was more attuned to the need to build a broad intelligence picture than the FBI with its focus on arrests and evidence.Footnote 79 FBI officials reject that characterisation, pointing to their efforts to develop a fully-fledged intelligence programme.Footnote 80 From their point of view, the FBI is responsible for developing intelligence on terrorist trends and networks, while DHS (I&A) should concentrate on analysing the potential for terrorist threats to expose vulnerabilities in the United States’ critical infrastructure, border security and other areas relevant to DHS’s component agencies.Footnote 81
The FBI and DHS have also been at cross-purposes concerning who has primary responsibility for fostering cooperation between the different levels of America’s federal system. They do work together to produce joint intelligence reports and bulletins that are disseminated to state and local governments and police.Footnote 82 At the same time, however, the two agencies are in charge of parallel networks that have a very similar raison d’être to coordinate information-sharing on terrorism between the federal and state and local authorities. Since the 1980s, state and local police forces have increasingly participated in FBI-led Joint Terrorism Task Forces (JTTFs). Now existing in over 100 cities and states around the US, JTTFs enable joint investigations and intelligence-sharing on terrorism between federal and local agencies. A Washington-based National JTTF manages these local task forces and acts as a national focal point for terrorism information-sharing between operational agencies.Footnote 83 After 9/11, however, Congress also gave the DHS intelligence office a substantial mandate in this area. As its website proudly states, ‘I&A has a unique mandate within the Intelligence Community and is the federal government lead for sharing information and intelligence with state [and] local … governments.’Footnote 84 Since 2003, DHS has also used its grant money to promote the creation of inter-agency fusion centres in states and cities, which perform information-sharing tasks some of which are similar to those conducted at the FBI-led JTTFs.Footnote 85 The FBI’s level of engagement with these fusion centres varies considerably and only about one-third of the centres are viewed as effective.Footnote 86 Asked about the various FBI and DHS-led networks, one FBI counterterrorist agent referred to them as ‘surreal parallel environments’, which were ‘very cumbersome and tiresome to deal with’.Footnote 87 A former senior DHS official also acknowledged that these parallel networks ‘are at times very redundant’. There is, he said, ‘so much room for improvement’ in this area.Footnote 88 Rather than seeing it as purposeful, officials within the FBI and DHS view the duplication of effort and rivalry between their two agencies as a cumbersome obstacle to their development of terrorism intelligence.
FBI and DOD
The Department of Defense has taken on an increasing role in the United States’ domestic response to Islamist terrorism since 9/11. Between 2002 and 2005, DOD expanded its main domestically-focused intelligence unit to 1,000 staff with the authority to give orders to a further 4,000 military investigators in the United States, making its potential counterterrorist investigatory ranks comparable to the FBI’s.Footnote 89 The role of DOD domestic intelligence is to protect military bases in the US from terrorist attack and conduct counterintelligence activities to safeguard the military from potential double agents or insider threats. However, this mission was broadly interpreted to include more general investigations into terrorist activity. For example, the Army’s 902nd Military Intelligence Group launched a programme under which its special agents and analysts gathered intelligence and made assessments on the general terrorist threat to the US homeland from Al-Qaeda, Hezbollah, and other groups. Because of the US system’s informal routines, the 902nd was able to take this independent initiative to carry out work already covered in depth by the FBI and other agencies at the expense of giving full attention to its own core missions.Footnote 90 On the other side, the FBI has arguably been overzealous in insisting that it should have lead responsibility for investigating members of the US military suspected of involvement in terrorism. As a Senate Committee report has detailed, DOD disputes the FBI claim and insists that it should have the lead in this area.Footnote 91 These kinds of incursions across fuzzy jurisdictional boundaries had two negative consequences in the case of Army Major Nidal Hasan, an Islamist extremist who killed 13 DOD employees and wounded 32 others at the Fort Hood military base in Texas in November 2009.
Firstly, the Army’s 902nd Military Intelligence Group – distracted by its ‘external’ activities – did not focus sufficiently inwards to detect Major Hasan’s radicalisation.Footnote 92 Secondly, a lack of cooperation between FBI and DOD played a role in their inability to identify the risk posed by this individual. When Hasan sent several emails to a known inciter of Islamist terrorism, Anwar al-Awlaki, in 2008–9, the FBI decided to open an investigation into the army major. The FBI at that time tended to informally share information with DOD about such investigations into DOD employees (officials from both agencies work together on JTTFs around the country, and a later FBI review found that information about such investigations was shared in the vast majority of cases).Footnote 93 However, these informal routines did not serve them well on this occasion because the FBI did not share its information about Hasan with DOD counterintelligence officials. This was a significant error, according to a Senate Committee investigation into the shooting, because DOD counterintelligence was better placed to evaluate the threat posed by its service member and would likely have mounted a deeper probe into Hasan at the very least as a potential espionage risk.Footnote 94 Instead, however, the investigation wound down in mid-2009 without any action being taken against Hasan.
In the aftermath of Major Hasan’s attack on Fort Hood later that year, the FBI itself recognised that its coordination with DOD had been suboptimal. The Bureau introduced procedures to ensure that all of its counterterrorist investigations that implicated DOD employees would be communicated to DOD counterintelligence. FBI and DOD also signed a consolidated agreement governing their operational coordination.Footnote 95 However, the Senate Homeland Security Committee expressed its ongoing concern in 2011, pointing out that the information sharing failures in the Hasan case partly stemmed from the FBI-DOD dispute over jurisdictional boundaries, which remains unresolved.Footnote 96
Beyond the field of domestic intelligence, the US Congress has sought to increase the role of DOD in another area that has traditionally been the preserve of the FBI. In December 2011, the legislative branch passed the National Defense Authorization Act (NDAA), which included a law mandating DOD to take the lead role in the detention of foreign Al-Qaeda-linked terrorist suspects arrested in the United States.Footnote 97 President Obama reluctantly signed the measure into law as part of a broader compromise with Congress, though the White House believed that it was ‘ill conceived’ and ‘remain[ed] concerned about the uncertainty that this law will create for our counter-terrorism professionals’.Footnote 98 This uncertainty was illustrated by the then Director of the FBI, Robert Mueller, who warned lawmakers: ‘The statute lacks clarity with regard to what happens at the time of arrest.’ He was concerned that the new law could create a situation in which ‘FBI agents and military [show] up at the scene at the same time on a [terrorist suspect] … with some uncertainty as to who has the role and who is going to do what.’Footnote 99 Mueller’s concerns were addressed in February 2012 when President Obama issued a policy directive, which included extensive waivers to the NDAA’s requirement for military detention of foreign terrorist suspects, ensuring that they will continue to be placed in FBI custody in the vast majority of cases.Footnote 100 The FBI remains in pole position in this area under the Obama administration. However, the law remains on the statute books and any future president or secretary of defense who is more sympathetic to the idea of military detention could interpret that law accordingly, opening the door for confusion and conflict between FBI and DOD.Footnote 101 The NDAA was a further example of how political compromises involving Congress create overlapping jurisdictions in law, which may in future require counterterrorist agencies to come up with informal work-around solutions or – if that is not possible – fight over who is responsible for what.
In sum, fuzzy jurisdictional boundaries and informal routines between DOD and the FBI contributed to deficiencies in their coordination of intelligence, most notably in the case of Nidal Hasan. Rather than clarifying their respective roles, changes to the law rather threatened to extend this confusion into another domain – the detention of terrorist suspects.
FBI and NYPD
After 9/11, the New York Police DepartmentFootnote 102 developed a major counterterrorist capability based around two distinct units – the Counterterrorism Bureau, which focused on law enforcement and specialised programmes, and the Intelligence Division, which was charged with developing terrorism intelligence. However, during most of the period since 2001, the Intelligence Division did not participate in the FBI-led Joint Terrorism Task Force (JTTF) in New York. Instead, the two organisations have tended to develop their terrorism intelligence sources independently. One FBI agent said that in New York, there were ‘parallel investigations … all the time. I would say that is common’.Footnote 103 The NYPD Intelligence Division has often been circumspect about sharing its information with the FBI. One former senior Bureau agent spoke of this in late 2008, more than seven years after 9/11: ‘They [the NYPD] have their own intelligence entity and they are running their own sources. And they are not telling. There is no sharing of that information. All done independent. So you’ve got two huge, very good organizations stepping on each other.’Footnote 104 NYPD intelligence agents have even sought to infiltrate some of the same groups being monitored by the FBI/JTTF and collect information in some of the same mosques, bookstores, and other locations as the FBI without notifying the Bureau.Footnote 105 This lack of cooperation has the potential to compromise not only the intelligence operations of the two agencies, but also their ability to intervene against terrorist suspects. In mid-2008, the NYPD did not inform the FBI that it was monitoring a suspect named Abdel Shehadeh until they saw that he was heading to John F. Kennedy Airport. The two agencies scrambled to respond but ultimately had to let a potentially dangerous man board a plane and fly to Pakistan.Footnote 106 Some months later, the then Attorney General, Michael Mukasey, wrote to the head of the NYPD, Ray Kelly, that such ‘documented failure[s] of the NYPD … to share information in a timely manner … are unacceptable and make New York and the country less safe’.Footnote 107
The FBI and NYPD’s Counterterrorism Bureau do work together on many investigations and have assigned almost 150 detectives each to the New York JTTF.Footnote 108 Despite this commitment, problems have also arisen here because of their failure to fully coordinate operational decisions on some cases. The inquiry into US resident, Najibullah Zazi, is often praised as an example of good inter-agency cooperation and it did involve effective coordination of foreign intelligence with a domestic investigation.Footnote 109 Yet even this good example had revealing shortcomings. In September 2009, the FBI was following Zazi as he drove towards New York with what agents believed were explosive materials in the trunk of his car. At a bridge leading into New York City, the FBI used Port Authority police to stop Zazi for a ‘random search’ but their sniffer dog failed to detect the materials, leaving police without a pretext to open his trunk and so they left him on his way. The fact that the FBI had used the Port Authority rather than NYPD angered the latter and soon the police department was taking its own initiatives on the case.Footnote 110 The Intelligence Division began asking their sources about Zazi, including an imam, Ahmad Afzali, who proceeded to inform the suspect by phone that law enforcement were onto him. FBI and NYPD officials have blamed each other both anonymously and publicly in the press for tipping off Zazi.Footnote 111 In any case, by this time, he had jettisoned his bomb-making materials and was soon taken in for questioning. As later court cases would show, a significant plot had been foiled but in a haphazard way which cut short efforts to develop intelligence on Zazi’s network and allowed any co-conspirators ample opportunity to flee.Footnote 112 Indeed FBI officials stated that they would have preferred to monitor Zazi and others for longer to gather further intelligence but could not because of the NYPD intervention.Footnote 113 Independent initiatives on both sides and a failure to cooperate on operational decision-making contributed substantially to this suboptimal outcome.
In sum, separate lines of responsibility at federal and state level and the lack of a central authority for counterterrorism gives the NYPD and the FBI freedom to informally implement both general policies and specific operational decisions without consulting each other – a pattern that led to conflict and confusion between the two sides. Such informal organisational routines, which enable rapid and nimble action, can certainly be a strength in counterterrorism.Footnote 114 However, when informal routines and relationships are relied upon to manage inter-agency coordination, it rarely proves a sustainable solution. According to a then Deputy Commissioner of the NYPD, Michael Sheehan, the FBI-NYPD conflict reached a high point in early 2006. Later that year, however, when the FBI appointed Joe Demarest as head of the New York JTTF, cooperation ‘improved dramatically’ due to his leadership style and good personal relationship with the NYPD Commissioner, Ray Kelly.Footnote 115 Yet when Demarest left the FBI in early 2008, relations deteriorated as a dispute between federal officials and the NYPD over the latter’s approach to electronic surveillance was revealed in the press.Footnote 116 Demarest was brought back to the New York Field Office in December of that year partly to patch up relations. He had some success in this endeavour, but when he was transferred to FBI HQ in Washington, DC in 2010, conflicts broke out again. In one joint investigation, the NYPD unilaterally sought a search warrant without informing the FBI, leading an angry Bureau to halt information-sharing with the NYPD Intelligence Division and suspend meetings of their JTTF for a period.Footnote 117 One former senior FBI agent reflected the Bureau’s irritation when he told me: ‘the NYPD is a sort of in-your-face type of thing, and its infuriating what they do, how they treat the FBI Field Office’.Footnote 118 During 2012–13, complaints about deficient information-sharing were once again being aired in the press.Footnote 119 By 2015–16, however, changes of personnel at the top of the NYPD and the FBI’s New York Field Office led to improvements in the exchange of information between the two sides.Footnote 120 Nevertheless, previous experience indicated that relations could again turn sour when the key personalities moved on. Overall, conflicts and inadequate coordination between the FBI and the NYPD in the post-9/11 era have been ‘serious enough to affect operations’, according to Michael Sheehan, hampering intelligence-development and allowing terrorist suspects to flee notably in the Zazi and Shehadeh cases.Footnote 121
Formal coordination through NCTC
If Congress’s substantial involvement in national security often leads to confusion over agency missions, the activist legislature and political class in the United States has also been a significant driver of reform, including the introduction of some formal coordination mechanisms into American counterterrorism. The creation of the National Counterterrorism Center (NCTC) in 2004 was one of the main reforms driven by the 9/11 Commission and Congress aimed at improving inter-agency coordination in response to the organisational failures that preceded 9/11.Footnote 122 NCTC brings together representatives from the United States’ various counterterrorist agencies for regular meetings in which they pool their information on terrorism. The centre also produces ‘all-source’ analyses of the threat, which aim to integrate all terrorism-related intelligence possessed by US government departments, agencies, and intelligence organisations.Footnote 123 NCTC is widely considered to have added value in these areas. As one senior FBI intelligence official put it: ‘on threat analysis and information-sharing, I think they do a pretty good job, and most people here [at the FBI], including the Director, would say that’. NCTC has direct access to the databases of the FBI and other agencies, the official explained: ‘We’re not only giving it to them … they can draw it out themselves.’Footnote 124 These procedures for automatic information-sharing with NCTC, along with the regular inter-agency meetings at the centre, indicate how the introduction of formal organisational routines can strengthen coordination.
The second key mission of NCTC concerns strategic operational planning; that is the assigning of specific roles in counterterrorism to US government agencies. NCTC negotiated with the various agencies over ten months in 2005–6 to produce a National Implementation Plan (NIP) for counterterrorism.Footnote 125 ‘There was a lot of wailing and gnashing of teeth’, recalled one former senior NCTC official, as agencies sought to maintain their prerogatives. Ultimately, NCTC lacked the authority to ensure that the agencies implemented the plan. The former official admitted: ‘I’m not certain how well [the NIP] has been followed subsequently.’ He also pointed out that NCTC does not direct tactical-level intelligence, law enforcement, or military operations: ‘We were almost like air traffic controllers. We weren’t operational. We didn’t tell them: “You should do this. You should do that.”’Footnote 126 While NCTC has added value in some areas, it does not have major authority or an operational role, nor has it changed the overall nature of US inter-agency counterterrorism, which retains its reliance on informal routines.
Overlapping missions and informal organisational routines create a fluid situation in which agencies can take independent actions without necessarily consulting their peers or a superior central authority. DHS took initiatives after its foundation, which the FBI reacted angrily to, leading to rivalry and duplication of effort, which both sides view as an obstacle to their development of terrorism intelligence. Fuzzy jurisdictional boundaries between DOD and the FBI contributed to deficiencies in their intelligence work. The NYPD also developed its activities independently of the FBI, with negative operational consequences in a number of instances. These case studies show how inadequate coordination and conflict between domestic security agencies have had damaging effects on the United States’ development of intelligence and its operations against Islamist terrorist networks.
Explaining inter-agency coordination in BritainFootnote 127
In contrast to the US case, the UK’s security agencies have tended to closely integrate their activities and refrain from turf battles. An analysis of MI5’s relationships with various police units shows that this higher degree of coordination owes much to the formal organisational routines of the British agencies.
MI5 and the London Metropolitan Police
During the 1990s, the missions of Britain’s three core domestic counterterrorist agencies were shaped by a sharp distinction between intelligence and evidence. MI5 devoted itself to collecting intelligence; the law enforcement officers of the Metropolitan Police Anti-Terrorist Branch (ATB) worked purely on gathering evidence that could be admitted to court; and Metropolitan Police Special Branch straddled the line between the two, linking intelligence to evidence-development.Footnote 128 After 9/11, however, facing a perceived threat of mass-casualty terrorism, the ATB police began arresting suspects earlier in the inquiry process than they had in the past (when they had faced the Irish Republican Army). In this context, the Anti-Terrorist Branch began to work more intensively with MI5 early on in particular cases to facilitate the assembly of evidence and enable consequent arrests. Senior ATB law enforcement officers, such as Peter Clarke, confirmed that they were now working with MI5 at an earlier stage of inquiries and were being given greater access to sensitive intelligence than they had previously.Footnote 129 ‘Operation Crevice’ in 2004 was a significant example of the ATB being involved at an earlier stage of an inquiry as it worked with MI5 to gather evidence for two months before the suspects were arrested.Footnote 130 MI5 also developed its role, playing a greater part in the provision of evidence for trial, alongside its traditional intelligence-gathering function.Footnote 131
MI5 and the ATB were now working directly together on linking intelligence to evidence-development. In effect, they were carrying out the role traditionally fulfilled by Metropolitan Police Special Branch. In this context, the Metropolitan Police decided to merge Special Branch with the ATB into a reformed division called the Counter Terrorism Command, which would bring together in one agency the traditional métiers of the two old branches: intelligence and law enforcement. Peter Clarke, then the leading officer at the ATB, was appointed head of the new CT Command, which was launched in October 2006.Footnote 132 The Metropolitan Police Special Branch was one of Britain’s two major police counterterrorist units, with a good reputation in police circles internationally and a history of operations that stretched back to the nineteenth century. Aware of the significance of their organisation, some Special Branch officers were unhappy when they learned that it was to be closed down and its personnel absorbed into a new agency.Footnote 133 Nevertheless, no major conflict ensued between Special Branch and the other agencies. Two factors help to explain why.
Firstly, we need to understand how the UK’s formal organisational routines – which involve distinct missions for each agency, laid out in government guidelines – have fostered stable expectations among the counterterrorist services. As noted above, these government guidelines have specified since the early 1990s that Special Branch’s role was to ‘assist’ MI5’s work on terrorism intelligence. In this context, there has been an expectation among Special Branch officers in recent decades that MI5’s role in terrorism intelligence would continue to be enhanced – at their expense.Footnote 134 As the ultimate outcome of what one senior police officer called a set of ‘incremental changes’, the decision to put an end to Special Branch as a distinct entity was wholly in tune with the established trend.Footnote 135 Secondly, the overall balance that the UK’s reforms maintained between the roles of the intelligence and police agencies softened the impact of this change. Since 2004, the government has allocated funding to more than double the staff of both MI5 and of the police’s main counterterrorism entities.Footnote 136 While such expansions may encourage mission grabs in some countries, the core counterterrorist agencies in Britain have less room to take such initiatives because each service has a distinct and well-insulated mandate. MI5 has a clear lead on the intelligence mission while the police maintains responsibility for law enforcement.
Formal organisational routines thus reduce the likelihood that agencies will take rapid or independent initiatives, which can provoke conflict between them and other services, as we have seen in the US case. In Britain, rather, changes were introduced incrementally either by the central government or with their support. The Metropolitan Police’s decision to close Special Branch was congruent with both contemporaneous and previous changes approved by the British government to promote MI5 primacy on terrorism intelligence. With a central authority supporting these developments, the British reforms maintained clear lines of responsibility and a balance between the roles of the police and intelligence services. In this context, the core agencies continued and, in some respects, intensified their close and regularised cooperation on counterterrorism. The government and agency leaderships were the key decision-makers in the British reforms with Parliament taking a back seat, in contrast to the United States where Congress played an active role in national security reform.
MI5 and regional police
While the British agencies foiled several substantial terrorist plots in the post-9/11 era, their failure to prevent the 2005 London bombings revealed deficiencies.Footnote 137 MI5 had strengthened its relationship with the London Metropolitan Police, as outlined above, but it was not sharing sensitive terrorism intelligence with provincial police forces to the same extent.Footnote 138 Some analysts believed that this contributed to the agencies’ failure to detect the London bombers’ attack plans in advance.Footnote 139 At the same time, intelligence was uncovering a growing Islamist terrorist threat in certain regions of the UK, which local police forces were ill-equipped to deal with. In this context, both the police and MI5 were given funding from 2006–7 to create significant regional counterterrorism units or stations.Footnote 140 Several sources indicate that this has improved MI5’s operational coordination with regional and provincial police forces. The inquest into the London bombings by Lady Justice Hallett, which was highly critical of MI5 in some areas, concluded in respect of this issue that ‘on the evidence, the gathering of intelligence around the country and the liaison between the Security Service and the various police forces has changed beyond recognition and brought with it considerable benefits’.Footnote 141 As an example of this liaison, the then head of MI5, Eliza Manningham-Buller, told of how her service’s regional station in the Midlands had worked with the regional police’s Counter Terrorism Unit and others on a sensitive operation in 2007, which disrupted a plot to kidnap and kill a British soldier: ‘It is clear to me that having an established [MI5] station in [the West Midlands] that is able to work very closely with both the Special Branch and the CT unit in [the West Midlands] on this case is extremely helpful’.Footnote 142
UK counterterrorism has departed from its centralised model to some extent with the introduction of regional police Counter Terrorism Units. However, these reforms have also provided for central control of the new regional units and introduced common procedures to ensure interoperability between them.Footnote 143 The London Metropolitan Police and MI5 remain the key actors and they have retained their authority to direct counterterrorism operations throughout Great Britain. Inter-agency coordination in the UK is not without its problems. A parliamentary report in 2014 recommended that the police and MI5 should work closer together to bring ordinary criminal charges against people that they suspect, but have insufficient intelligence, of being linked to terrorism. The agencies considered bringing drug charges in 2012 against a man who went on to kill Lee Rigby the following year, but they did not give it sufficient priority to make a successful disruption.Footnote 144 Notwithstanding this flaw, however, it is clear that the British agencies overall have considerably less coordination problems and turf conflicts than their US counterparts.
Implications for theory and policy
In the United States’ domestic response to Islamist terrorism, there has been considerable conflict between core agencies such as the FBI, DHS, DOD, and the NYPD, while their reliance on informal inter-agency ties has had negative operational consequences. The more formal arrangements of the British case have led to higher levels of inter-agency cooperation on counterterrorist intelligence and law enforcement operations and little conflict between core organisations. To explain this variation, we have shown how an interaction between macro-institutions and micro-organisational routines shapes the development of intelligence and operations against terrorism within the United States and Britain respectively. These findings have implications for both theory and policy.
The findings raise questions about the ability of some influential theoretical paradigms to shed light on the dynamics of security organisations. From a bureaucratic politics perspective, security agencies in Britain and the US may be assumed to have a similar interest in maximising their autonomy and sphere of activity. If this is the case, however, the model cannot explain the variation in levels of inter-agency cooperation between the two cases, nor does it shed light on why turf battles are prevalent in the US but rare in Britain even when agencies’ bureaucratic interests are severely threatened as in the case of Special Branch. Rather than placing interests at the centre of the analysis, it is more fruitful to examine the jurisdictional boundaries between security agencies (whether they are distinct or overlapping). It is equally important to focus on how informal organisational routines enable individual agencies to take rapid and independent initiatives, which are more likely to cause conflict and stymie cooperation than changes that are introduced incrementally with the support of a central authority.
Routines are sometimes linked to, or seen as a component part of, organisational culture.Footnote 145 In this sense, my conclusions build on and are complementary to cultural studies. Nevertheless, this article has suggested that a focus on organisational routines offers more specific mechanisms than culture for the analyst to examine. It has specified three institutionally-based structural conditions, which shape the formation of these routines. This interaction between national institutions and specific organisations is worthy of further study. As argued above, since institutional and organisational routine theories can explain how historical legacies lead to suboptimal practices – even in a top priority area such as counterterrorist policy – these theories have passed a difficult test.
Turning to policy implications, this study identifies certain key conditions for the development of a high level of inter-agency cooperation on counterterrorist intelligence and law enforcement operations. The presence of few core counterterrorist agencies, whose work is regulated by distinct missions and clear guidelines laid down by a central authority, has been found to give rise to formal organisational routines (see Figure 1). These routines entail regularised interaction between services and favour the development of a relatively high level of cooperation between core organisations. All of the conditions leading to this outcome were present in the British case and absent in the American case.Footnote 146 It may be useful for US policymakers to take these conditions into account when making future alterations to the agencies; for example, the importance of giving distinct missions to security organisations. Furthermore, the record of NCTC shows how formal routines and coordination mechanisms can be introduced into parts of the system in a way that brings concrete benefits. Some degree of reform to the coordination of US counterterrorism is possible.
However, the analysis offered in this article also highlights the severe constraints on reform and indicates why more substantial changes to the coordination of US counterterrorism have not been introduced. For example, some expert panels have made proposals for a moderate centralisation of US counterterrorism, in which the President of the United States would ‘empower’ NCTC to ‘serve as an integrating mechanism for CT in the U.S government’.Footnote 147 From this perspective, if clashing bureaucratic interests are a key driver of inter-agency turf wars and coordination problems, then what is needed is stronger direction from the top. However, such proposals encounter a formidable obstacle in the American suspicion of concentrated power. This anti-statist impulse is so pervasive that it is even expressed by high-level security officials who one might have expected to be sympathetic to some degree of centralisation. A former senior NCTC official spoke of inter-agency coordination problems that he had personally witnessed, adding:
But that’s the way we are in this country. We decided long ago, we’re not gonna have … concentrated power … This country is much more comfortable with a fragmented [law] enforcement and intelligence system. That comes with a cost: conflicting operations and conflicting analysis and everything else. But we have decided that we’d rather have that than have a KGB.Footnote 148
In the context of such attitudes, anything that smacks of centralisation is likely to face rigorous opposition.Footnote 149 Even if these obstacles could be overcome and a moderate centralisation of US counterterrorism was introduced, my analysis suggests that it would not make a great deal of difference to operational coordination. The informal routines of the US agencies have, over time, taken on significant momentum and staying-power. As noted above, the theoretical literature on the subject indicates that organisational routines are taken for granted in their particular contexts and tend to be reproduced in unreflective ways. Cross-jurisdictional incursions and turf battles have become routine and expected behaviours among US security agencies. It would take more than a moderate increase in central authority to change these deeply-embedded organisational routines. In American politics and society, however, a more than moderate centralisation is unlikely to be acceptable.
A second idea for reform is that policies designed to alter the incentive structure of security officials might reduce their focus on their own agency’s bureaucratic interest. For example, Amy Zegart has suggested making rotational assignments to other agencies a requirement for promotion. She argues that this would create incentives and opportunities to establish informal networks and build trust between officials across agencies.Footnote 150 It is true that good relations between individuals can improve inter-agency cooperation for a certain period, as was outlined in the FBI-NYPD case above. Yet, as this example also showed, such informal links and routines do not provide a sustainable solution to the problems of inter-agency coordination and conflict over turf.
These and other reform proposals are influenced by the widespread assumption that bureaucratic self-interest is a key driver of deficiencies in inter-agency coordination. Perhaps surprisingly, it turns out that this assumption leads to overly optimistic conclusions. A comparison of the United States with the British case, focused on their deep-seated routines and institutions, leads one to be more doubtful about the prospects for significant change in inter-agency counterterrorism.
When a terrorist attack takes place in the United States, the response of many members of Congress in the weeks and months afterwards is to criticise the security agencies that failed to prevent it. They do not usually acknowledge that the fragmented counterterrorist system that they criticise stems from a deep-rooted set of anti-statist institutions of which Congress itself is a key component. Rather than simply blaming the agencies, lawmakers and citizens would do well to acknowledge more explicitly the uncomfortable trade off between avoiding excessively strong government on the one hand and developing effective inter-agency counterterrorism on the other.
Biographical information
Dr Frank Foley is Lecturer in International Relations at the Department of War Studies, King’s College London. His research focuses on counterterrorism, human rights, intelligence and police agencies, particularly in Britain, France and the United States. He is the author of Countering Terrorism in Britain and France: Institutions, Norms and the Shadow of the Past (Cambridge University Press, 2013). Dr Foley holds a PhD in Political Science (2008) from the European University Institute in Florence and was the Zukerman postdoctoral fellow at Stanford University’s Center for International Security and Co-operation (CISAC) in 2008–9. His works have appeared in various outlets, including Security Studies, the Review of International Studies, and the European Journal of Criminal Research and Policy.
Acknowledgements
I would like to thank Martha Crenshaw, Erik Dahl, Stephen Marrin, Robert Pape, Pascal Vennesson, and the EJIS reviewers and editors for their helpful comments and criticism, as well as participants in seminars and panels at the International Studies Association Conference 2015, the START Conference at the University of Maryland, the Center for International and Security and Cooperation (CISAC) at Stanford University, and the US Department of Defense (Office of the Assistant Secretary for Homeland Defense). Thanks also to the practitioners who agreed to be interviewed for this project. I gratefully acknowledge the financial support of the AXA Research Fund, the Centre for Political and Constitutional Studies, Madrid, and the Zukerman fellowship programme at CISAC Stanford.
