The deliberate distortion of the work achievements of local governments, a thorny issue for China's leaders during Mao's era, has emerged once again as a concern for Chinese Communist Party (CCP) leaders in recent decades. Studies show that information distortion is not a sporadic, occasional or locally specific phenomenon: it is found at various levels of local governmentFootnote 1 and is widespread across different regions.Footnote 2 In addition, information distortion is not restricted to particular policy areas: it is evident in the reporting of GDP growth rates,Footnote 3 industrial statistics,Footnote 4 peasants' net incomes,Footnote 5 death tolls from coal mining accidents,Footnote 6 the number of mass complaints,Footnote 7 and so on. These observations call the reliability of China's official data into serious question.
A number of these studies indicate that the deliberate distortion of information has been driven mainly by the implementation within local government of the target-based cadre evaluation system, which began in the late 1980s.Footnote 8 This system links local officials' career prospects to how well they meet top-down allocated socio-economic development policy targets. Under pressure to meet these targets, some local officials manipulate or even falsify performance data, sometimes just to survive their evaluations and sometimes to stand out from their peers in the performance competition.
CCP leaders use the target-based cadre evaluation system as a key means to improve local compliance with the party-state's policy goals. Local leaders' performance targets are derived from the macroscopic policy goals set by the party-state at various levels, and the evaluation mechanism is designed to ensure the realization of these goals at each corresponding level. The system encourages local governments to report the extent to which policy goals are being met, thereby keeping the party-state informed of the progress of local socio-economic development. Based on the feedback received, the state then adjusts its policy plans for China's transition, reformulating and fine-tuning policies to improve them. Sebastian Heilmann and Oliver Melton describe the mutually reinforcing interaction of the evaluation system and policy formulation as a “plan-cadre nexus.”Footnote 9
It is clear then that the distortion of local government accomplishments can have very serious consequences, including decision-making failure at various levels and the erosion of effective state control over local agents. More importantly, it can undermine the basis on which policymakers rely to formulate appropriate short- and long-term development plans – the very plans that will be disaggregated into performance targets for local officials in the following years.
Given that reliable government performance data are so important, how do Chinese leaders tackle local officials' deliberate information distortion? Why is China's statistical system, the main mechanism used for collecting information, not very effective at collecting reliable data on the government's work? What strategies do Chinese leaders use to remedy the problems? How effective are these strategies in improving the quality of data? This study attempts to answer these questions.
China's information-collection mechanisms have had little scholarly attention. Several studies, however, are noteworthy. For example, in his articles on administrative monitoring in China, Huang Yasheng indicates that, in the reform era, China's leaders have strengthened their ability to collect local information.Footnote 10 Their strategies include establishing offices tasked with reporting reliable information direct to central policymakers, using the letters-and-visits system to check on the performance of local cadres, and collecting information for economic policymaking through the statistical and auditing systems. Cai Yongshun has examined information collection in the area of anti-corruption. His research shows that China's anti-corruption agencies mainly rely on reports by citizens through the letters-and-visits system and information revealed in other corruption cases under investigation as well as media, inspection and audit exposure.Footnote 11 More recently, Tom Orlik has examined the major reform measures undertaken by China's National Bureau of Statistics (NBS) in 2008–2013 to improve the quality of economic data. These measures include strengthening the independence of local statistics offices to protect against interference from local leaders, and ratcheting up punishments for false reporting.Footnote 12
Notwithstanding their remarkable contributions, these studies do not adequately explain the measures that have been developed to tackle artificial information distortion at local levels. Huang Yasheng's studies mention that the NBS conducts periodic performance checks on bureaucrats to detect deviations from central policy guidelines, but his research has little to say regarding the design of these information-collection strategies and the deficiencies of China's statistical system in obtaining quality data. Orlik's work advances our knowledge in this regard by explaining the role of the NBS in detecting and preventing data falsification since 2008; however, it offers little discussion on the collection and quality of information that is beyond the main statistical system but still important for policymaking (i.e. information on sector-initiated programmes). Cai's study, by contrast, does address how sector-based data such as corruption cases might be collected, but because it focuses on anti-corruption, it does not attempt to explain the CCP information-collection mechanisms that are used across the board.
This study examines the CCP's mechanisms for tackling artificial distortion of information related to local government work achievements. It argues that, in the past two decades, the party-state has developed “bypassing strategies” to tackle information distortion at local levels. The main feature of these strategies is that both central and local authorities which are in need of accurate information reach out to those who have the raw data, and in this process, avoid the lower-level officials who have the incentives to misreport their work accomplishments. This study presents evidence at both the local and national level that these bypassing strategies are improving the reliability of data. Distortion is not entirely eliminated, and the quality of information on local performance is uneven, but the bypassing strategies ingeniously reduce both the motives and the opportunities for information distortion and improve the party-state's ability to collect key information needed for target setting and policymaking.
This study contributes to the existing literature in the following ways. First, the development of the bypassing strategies signals an important change in the politics of China's information-collection system, away from the bottom-up, level-by-level reporting that has been practised in the past, and towards the direct collection of information by higher-level authorities – a centralization of power in their hands. This shift has not received the attention it deserves, and this study will provide empirical evidence of this change. Second, the quality of information on a local government's accomplishment of the top-down allocated targets is a key issue in understanding the role and effectiveness of China's planning system. It is important to know how reliable that information is to understand how effectively the target–cadre nexus can guide resource allocation, as debated in a recent symposium.Footnote 13 This study will shed some light on this issue. Third, information distortion is not unique to China. It also appears in Western and some Asian countries that have implemented target-based performance measurement systems. This study introduces China's experience of tackling this problem which may offer useful lessons for reformers and practitioners in other countries, especially those with large bureaucracies such as China's.
This study mainly draws on data collected from several rounds of field research in the provinces of Guangdong and Shaanxi during 2010–2015. To obtain insider viewpoints on the quality of government performance information, the author interviewed a total of 45 Party and government officials who play a key role in collecting, auditing and reporting data in these localities. Some officials were interviewed more than once in several rounds of fieldwork. The interviewees also included a team of researchers in Guangdong who worked as an independent think tank to provide information for local government. This study also uses evidence from a number of Party and government documents, including internal documents that were collected during the field trips.
The Rise of Information Distortion in the Reform Era
China's official statistics during the Maoist period are regarded as largely reliable, despite the nationwide epidemic of false reporting that occurred during the Great Leap Forward. Several economists examined the quality of statistical data during that period thoroughly, and a general observation arising from their findings is that deliberate falsification of statistics was highly improbable, although statistical inaccuracies did exist owing to deficiencies in collection, compilation, calculation and publication of data.Footnote 14 As Thomas Rawski indicates, China's statistical system during the Mao era “steadily increased its ability to develop meaningful and reliable quantitative indicators of magnitudes selected for investigation,”Footnote 15 and produced information that “withstood scrutiny from an often skeptical international community.”Footnote 16
Falsification and embellishment were increasingly observed in local statistical reporting in the 1980s, however. Studies found sporadic evidence of the distortion of agricultural outputs, industrial outputs, the amount of arable land, and population control in several provinces.Footnote 17 Having learnt a bitter lesson from the Great Leap Forward, CCP leaders took immediate action to strengthen the monitoring of statistical data. The 1983 Statistics Law, which applied to all levels of governments from the county level up, stipulated that misreporting and falsifying statistical data, or requesting others to do so, had legal consequences and would be punished. However, the efficacy of the Statistics Law was undermined by lax local enforcement, and statistical distortion and violations of the Statistics Law remained routine among local officials throughout the 1980s and 1990s.Footnote 18
Towards the end of the 1990s, the international community grew increasingly suspicious of the reliability of China's socio-economic statistics in general, and of the plausibility of GDP data in particular. Rawski, who had earlier expressed confidence in the reliability of China's pre-reform statistics, was one of the forerunners of this criticism. In several articles published in 2000 and 2001, Rawski questioned the reliability of the official GDP data gathered in 1998 and 1999. His research shows that the maximum growth of China's GDP in each of these two years was about 2 per cent, and cumulative growth over the period 1998–2001 was somewhere between 0.4 per cent and 11.4 per cent, which was much lower than the official figure (34.5 per cent). He observed, “Beginning with 1998, standard GDP data contain exaggerations that extend far beyond the technical difficulties,”Footnote 19 and “intentional falsification of economic performance indicators is commonplace throughout the business community and at every level of government.”Footnote 20 A few others expressed similar doubts.Footnote 21
By the mid-2000s, information distortion had become an alarming and urgent issue for central leaders. Distortion of the government's performance was frequently reported in various localities and policy areas, and at various government levels. According to the NBS, about 60 per cent of the 59,200 disclosed cases of illegal statistical behaviour that occurred between 2001 and 2003 involved false reporting.Footnote 22 In addition, in 2003, the sum of provincial GDP for the first time exceeded the national figure. Since then, the gap between the two sets of data has been widening. Between 2004 and 2013, the gap between the national GDP and the sum of provincial GDP grew from 0.3 trillion yuan to 6.1 trillion yuan.Footnote 23
It should be noted that certain other economists, such as Carsten Holz and Gregory Chow, hold different viewpoints. They argue that China's economic statistics remain largely trustworthy, although not without problems.Footnote 24 Their studies mainly defend the quality of China's national statistics; however, they also agree that data falsification at local levels is widespread, as “evidence appears overwhelming.”Footnote 25
The Statistics Bureaus' Dilemmas
Since its foundation in the 1950s, the statistical system has been the key mechanism for collecting data on China's socio-economic development. Hence, when cases of data falsification are discovered, the public blames the NBS or local statistics bureaus because they are the authorities that collect, interpret and release the data. China's local statistics system would like to address data falsification, but it faces two fundamental dilemmas when dealing with the problem.
The first dilemma is that local statistics bureaus take their orders from local governments regarding reporting data. The NBS does not have the authority to manage local statistics bureaus and does not have much control over local information – that authority rests with the localities. The Statistics Law established a statistical system characterized by “unified leadership and tiered responsibility.” “Unified leadership” means that the NBS organizes, leads and coordinates the nation's statistical work. The statistics bureau at one level has a business (or professional) leadership relationship with the statistics bureaus directly below it in the governmental hierarchy. “Tiered responsibility” means that statistics bureaus at and above the county level are responsible for the statistical work at their own level. However, at each level, the local government has a direct (administrative) leadership relationship with its local statistics bureau.Footnote 26 The local government plays a major role in managing the established posts, leadership selection and the finances of the local statistics bureau, although advice on these matters would be sought from the statistics bureau at the next higher level.
Under this bifurcated system, the local government rather that the NBS has de facto management authority over the statistics bureau at that particular level. Before releasing data on the local government's work performance, the statistics bureau needs to get approval from local authorities. Put another way, even if the statistics bureaus collect reliable data, if local leaders wish to alter the data, they can lean on the local statistics bureaus to manipulate the information before it is reported to the higher-ups and released to the public. The implementation of the target-based evaluation system of cadre management in the late 1980s provides exactly the sort of high-powered incentive that would lead local leaders to make that request.
The second dilemma is that information collection is segmented in practice: there is the comprehensive statistical system (zonghe tongji xitong 综合统计系统) and the sector statistical system (bumen tongji xitong 部门统计系统).Footnote 27 The NBS, local statistics bureaus and other relevant statistical organizations (such as the survey teams) take charge of the comprehensive statistical system and collect major socio-economic data, while the statistics divisions in all other departments collect the sector-based data. For a variety of data, the NBS and local statistics bureaus rely on data collected by these other departments; they are not able to dictate the types of data to be collected or the precise collection method.Footnote 28 Nor do local statistics bureaus have the authority to manage the established posts and personnel selection in these statistical divisions. The authority of the statistics bureaus is limited to a weak business “guidance” relationship with the statistical divisions within other departments; the departmental divisions in effect monopolize sector data.
Bypassing the Lying Mouths: the CCP's Countermeasures
The reform-era CCP leaders have taken various steps to tackle local information distortion, including conducting regular socio-economic censuses and strengthening the legal measures against false reporting. At local levels, a strategy developed in the 2000s whereby authorities that wanted reliable information (the information seekers) would bypass lower-level officials or departments which had incentives to distort data (information reporters) and go directly to the origins of the raw data (information providers), either by sending their own teams or through independent third-party organizations. The information seeker might be the NBS, a local government or a specific department in charge of collecting data on a certain type of work. The information reporter might be a lower-level government or department (either directly below or several levels below the information seeker). The information provider would be individual citizens and enterprises.
Why would local officials, who sometimes distort statistics themselves, spend time and effort bypassing lower-level agents to collect reliable data? My fieldwork shows that their incentives are at least twofold. On the one hand, local governments need accurate and timely statistics on the progress of policy implementation for decision-making purposes. Whether they can lay their hands on reliable information is a different issue from how this information is to be reported. On the other hand, the higher-level authorities constantly bypass local officials to dig out dishonest reporting and may hold them accountable if violations are exposed. By using bypassing strategies of their own, local officials deter lower-level agents from unscrupulously falsifying data and thus reduce the risks of being penalized themselves.Footnote 29
The bypassing strategies are a good fix, although not a perfect solution, to the two dilemmas described above. The following section will examine three examples of widely used bypassing strategies to illustrate how they are used to improve the quality of both statistical data (data collected by local statistics bureaus under the comprehensive statistical system) and sector data (data collected by each department).
Statistical reforms: improving the reliability of statistical data
In the mid-2000s, tackling the distortion of economic statistics was tabled as an urgent issue on the top leaders' policy agenda. Since then, a series of reforms has been launched, with a focus on the comprehensive statistical system. The first noteworthy measure was to strengthen the NBS survey teams' capacity for independent data collection and reporting. There is a historical background for adopting this measure. In 1984, the NBS established two survey teams – the Rural Sampling Survey Team and the Urban Sampling Survey Team – under its direct control to conduct independent investigations on key socio-economic issues such as agricultural outputs, net incomes of rural and urban residents, price indices, etc. But, only two years later, the NBS delegated the management of these two teams to local statistics bureaus. Decentralization greatly reduced the advantages of the survey teams in information collection and reporting. In 1994, a third survey team, the Enterprise Survey Team, was also established under the same management framework.Footnote 30
Under the current statistical system, it is not easy for the NBS to enhance the independence of local statistics bureaus, as they fall under the authority of the local party-state. An alternative is to establish a relatively more independent team of statistical professionals that reports directly to the NBS. Between 2004 and 2007, the NBS implemented nationwide reforms of the three survey teams (rural, urban and enterprise). At all local levels, they were merged into one new survey team. In 2011, the new system comprised 32 general survey teams in provincial-level governments, 15 teams in municipality-level governments, 333 teams in prefecture-level governments and 857 teams in selected county-level governments. The total staff of the government statistical system at and above the county level numbered 98,568, with approximately 14,500 people in established posts working in the NBS survey teams at various levels.Footnote 31
The survey teams still perform their work at the local level, but they are managed directly by the NBS. The survey teams carry out investigation tasks assigned by the NBS, independent of the local government, and report the investigation results directly to the NBS. In addition, the rank of the survey teams is now equal to the rank of the local statistics bureaus (previously the survey teams had been ranked half a grade lower than the local statistics bureaus).
An interviewee from the Foshan 佛山 City Survey Team in Guangdong province indicated that this round of reforms had improved the quality of data in at least two aspects.Footnote 32 First, since the reform, local government has had little influence over the information that the survey team reports to the NBS. The survey team collects data directly from information-reporting units such as enterprises and residents. The team works side by side with Foshan city's bureau of statistics in the compilation of local statistical data, and they share data with one another, but neither can intervene in the other's data collection. Second, there is a work division between the statistics bureau and the survey team. For example, the statistics bureau is responsible for collecting data from the 6,300 industrial enterprises above a designated size through a complete survey method, while the survey team works on the 33,000 smaller enterprises below the designated size through a sampling survey method. The two sets of data can be cross-checked to verify their credibility. If the statistics bureau reported that the bigger enterprises' revenue income was rapidly growing while the survey team's investigation showed an obvious declining revenue income for smaller enterprises, the contradictions in the two sets of data would indicate a possible manipulation of data at one end.
The second reform measure was to change the GDP calculation method. In 2004, the NBS required local governments at all levels to shift gradually to a one-level-down calculation system to improve the accuracy of local GDP data. Previously, each locality had calculated its own GDP and then reported to the higher-level government for verification. However, under the one-level-down calculation system, the statistics bureau at one level calculates the GDP for the government level directly below it, thereby circumventing governmental attempts to misrepresent its GDP. For example, in Foshan city, prior to the reform, every district and county in Foshan city calculated its own GDP and then reported the results to the city. During this process, data could be distorted to make it appear that the locality had met its pre-established economic targets. The city statistics bureau would check the reported data and then release the “verified” data to districts and counties. The verified data might or might not reduce the inflated figures, and sometimes even exaggerated them. The same practice was repeated when the cities reported to Guangdong province. Under the one-level-down system, by contrast, the NBS first calculates the GDP for Guangdong province, and then the provincial statistics bureau calculates the GDP for Foshan city. Foshan city then calculates the GDP for all its districts and counties.Footnote 33 The data collected by the survey teams at each level, which local governments cannot interfere with, are used as references in calculation.Footnote 34 In this way, the incentives – and ability – to distort local GDP data are greatly reduced.
Given the complexity of GDP calculation and the difficulties in unifying the calculation methods and different local situations, the NBS has indicated that it will be several years before nationwide implementation of the one-level-down GDP calculation system in local governments at all levels is complete. During this process, the NBS must first lay the groundwork by getting a true sense of each locality's economic situation by conducting censuses. Before the transition to the one-level-down GDP calculation system is complete, a one-level-down management system is put in place, whereby the statistics bureau at one level must verify the data reported by the statistics bureau at the level directly below it (the verification process is explained above).
In 2004, the NBS selected ten localities, including Shandong, Guangdong, Zhejiang and Jiangsu – all developed areas – to test the system. Available evidence suggests that the system achieved positive results. For example, in Guangdong province, one year after the adoption of the one-level-down management system, the discrepancy between the sum of its 21 cities' GDP and the GDP calculated by the province in 2005 was reduced from 3.7 per cent to 2 per cent.Footnote 35 Another piece of supportive evidence is that, under these reforms, the discrepancy between the national and provincial GDP in 2014 was reduced to 4.8 trillion yuan, down from 6.6 trillion yuan in 2013, after almost ten years of continual growth.Footnote 36
The third reform measure was to establish a unified national data system. This reform is one of the well-known “four big tasks” launched in 2008 by Ma Jiantang 马建堂, the head of the NBS. The other three tasks are the establishment of a unified, up-to-date list of qualified enterprises; the implementation of a standard reporting form for statistical information, which is to be used by all enterprises on the list (qiye yitaobiao 企业一套表); and the achievement of direct reporting by firms to the national database. Eventually, the project aims to establish a national system for collecting and managing the data, under which local, provincial and national statistics bureaus will all work with the same numbers as soon as those numbers enter the system.Footnote 37
From 2009 to 2011, to test the feasibility of a unified data collection system, the NBS conducted four rounds of experiments in different regions. In 2012, the four-task project was launched on a nationwide scale for the first time.Footnote 38 About 900,000 enterprises directly reported raw data to the national database, and this number has increased by roughly 20–30 per cent every year since.Footnote 39 Any records of alterations or modifications of the raw data are saved in the database and hence are traceable. An internal report from the Foshan city statistics bureau indicates that by April 2015, a total of 9,800 enterprises had registered on the national list. These enterprises contribute 66 per cent of Foshan's GDP, and they report directly to the national database. Through online direct information reporting, townships, counties, cities, provinces and the NBS can view the reported statistical data simultaneously, which greatly reduces the possibility of deliberate manipulation of statistical data and improves the efficiency of data collection.Footnote 40
Spot checks: improving the reliability of tangible sector data
Ensuring the quality of sector data is very challenging because it depends on the willingness and capacity of each sector and department, and both vary from situation to situation. As mentioned earlier, the NBS and local statistics bureaus do not have the authority to take on the collection of sector data, and adoption of a unified approach to collection and management of this type of data is not feasible. Although certain measures were initiated at the national level to strengthen the quality of sector data, an important strategy is to send down inspection teams to conduct spot checks at the grassroots levels.
Spot checks are not a reform-era innovation. They have played an important role in local politics, governance and policy implementation throughout the history of the People's Republic of China. Depending on the needs, inspection teams can be organized and dispatched by various sectors. In addition, they can be organized by the central organizations to check up on work in one or several localities, or by higher-level authorities in a locality to check up on the work of government at lower levels.Footnote 41 During my fieldwork in Guangdong and Shaanxi, I observed spot checks being used in areas such as forestry, education, investment of fixed assets, and family planning. These are all policy areas partially or fully funded by higher-level authorities.
Take the example of spot checks on the implementation of population policies in Shaanxi province. In the 2000s, the Shaanxi population and family planning commission have conducted two rounds of field inspections every year, and cities in this province organize their own spot checks. The inspection teams adopt several strategies to make sure they access the most authentic information. First, the inspection teams sidestep the government to be reviewed and visit the villages directly. An interviewee explained that when the provincial commission wants to examine the cities' work, they conduct random spot checks in villages under the administration of each city. This information will be used to cross-check the information reported by Xi'an city. Likewise, when the Xi'an city commission examines the counties' work, they conduct random spot checks in villages under the administration of each county and district. In 2011, the city inspected 51 villages under its 13 counties and districts.Footnote 42
Second, the selection of the inspection sites is often kept secret from the villages as well as from their superior governments so as to avoid beforehand preparation. Inspection teams sent by Shaanxi province and Xi'an city adopted the same method to keep the information secret.Footnote 43 On the day of inspection, once all team members are gathered on the coach, the team leader opens a sealed envelope that contains the name of the county they are to visit. When the coach enters the county, the team leader opens another sealed envelope with the names of the villages to be inspected. Changes of inspection sites are strictly forbidden and only granted in exceptional cases, for example, when the name of the selected site has been leaked. The team leader is held strictly responsible for guaranteeing the confidentiality of the inspection information.
A third strategy is for inspection team members to talk directly to selected households in the villages. The inspectors will ask the residents to show their certificates (marriage certificate, birth certificate, one-child honour certificate, etc.). If the household has no prior warning of the visit, extra births are easily discovered. If the inspectors cannot meet members of the household, they will randomly talk to at least three residents in the same village about the birth-control situation in that household.Footnote 44 The inspectors use a set of very detailed forms to check on the population control and family planning situation in the locality. Township and village cadres are not allowed to be present during any part of the visit. When all the work is done, the inspection team meets to discuss problematic cases. A second round of spot checks takes place if there are such cases.Footnote 45
The spot checks kill two birds with one stone. First, they enable the inspection teams to see the real situation in the localities at first hand. For example, in 2010, Xi'an city required one of its counties (Z County) to keep the new birth rate to within 13.2 per cent, among which 94 per cent or more were to be legal new births. Furthermore the accuracy of the reports of new births was to be above 97 per cent (3 per cent statistical error acceptable). However, an internal inspection report showed that by the end of 2010, two out of the 22 townships had failed to achieve the specified birth rate, and ten townships had failed to control the statistical error to within the required level. In addition, nine townships failed to meet the required sex-ratio rate set by the city, and 17 townships failed to perform satisfactorily on the “three-checkups.”Footnote 46
Second, the randomness and unpredictability of the spot checks has driven local officials to make improvements, in part because fabricated data might be exposed. For example, because Shaanxi province and Xi'an city both bypass Z county to conduct random spot checks in villages, the leaders of Z county take their population work very seriously. In 2010, they conducted four rounds of checks in villages. In each round, the county randomly investigated around 50 villages under its 22 townships as samples. Adding up the two rounds of inspections organized by both the province and the city, there were at least eight rounds of inspections in the villages devoted solely to population and family planning work. The intensity of the inspections can be even higher if the central government also sends inspection teams.Footnote 47
Township leaders are the most nervous community in these frequent checks, as they are bypassed by all the higher-up inspection teams and held directly responsible for any problems discovered in the villages. One township official indicated that it is hard for townships and villages to know in advance which villages will be selected for the review. Even if local cadres manage to get this information, the inspection team may not come. Either way, the townships and villages have to prepare for the spot checks. One township Party secretary complained that in 2011, the county leaders informed him nine times that the higher-level authorities might come to examine the population work in his township. However, the inspection teams did not visit the township even once that year. With the threat of inspection hanging over his head, this Party secretary strictly controlled extra births that year and received a first-class award in the year-end evaluation for his outstanding performance of this work.Footnote 48
Third-party evaluation: improving the reliability of intangible sector data
When the outcome of government work is not tangible, methods such as spot checks are not useful in improving the trustworthiness of data collected. One such example is the task of improving social order, a policy goal with overriding importance for the Chinese party-state in recent decades. Conventional evaluations of government work on improving social order use objective measures such as the crime rate, the number of mass incidents, the rate of dispute resolutions, and the like. The data are mainly collected by the department/sector that has charge of local political and legal affairs. Even if the higher-level authorities do conduct spot checks, they can only obtain limited useful additional information on the local safety situation. More importantly, such evaluations do not reveal how the local people feel about social order in their area, which would be an indication of their satisfaction with Party rule.
In this context, since the mid-2000s, CCP leaders have emphasized the use of external evaluations for this type of government work, and many local governments delegate information collection to independent, third-party organizations. Local practices started in the 1990s and have flourished since the 2000s. In 2006, central leaders explicitly required local governments to conduct citizen satisfaction surveys in order to analyse public perceptions of local economic, political, cultural, social and Party development progress. The order from the central government spurred further enthusiasm for using citizen satisfaction surveys to evaluate local government performance.Footnote 49
The survey on citizens' feelings regarding public safety is a typical case. In 2001, the central leaders required statistics bureaus at all levels to conduct such surveys in order to collect reliable, fair, objective and trustworthy information. Since 2009, the surveys have been conducted by public opinion research centres (sheqing minyi zhongxin 社情民意中心), which are social institutes under the administration of the statistics bureau at the same level. The statistics bureaus and the research centres are regarded as third parties that are independent from both the subject under evaluation (i.e. lower-level governments) and from the evaluators (the citizens). By 2010, public opinion research centres were established under the statistics bureaus in 28 provinces, autonomous regions, and municipalities directly under the central government. With the strong support of the central leaders, public-safety surveys have now become a nationwide practice.Footnote 50
The statistics bureaus and/or the research centres mainly use computer-assisted telephone interviewing (CATI) technology to conduct the surveys, which means that the centre staff call a sampling of local residents in each jurisdiction to seek their opinions on local safety. The interviews consist of a questionnaire that asks respondents about their feelings regarding local safety, satisfaction with local police work, adequacy of local safety publicities, and the like.Footnote 51 In Shaanxi province, the public-safety surveys were introduced in 2001 and have been conducted on an annual basis. The survey results and the ranking of each local government are released to the public through local newspapers.
Guangdong province goes even further. In 2012, the province developed a total of 26 “safety performance indicators” to evaluate the safety work of lower-level governments. Of the 26 indicators, 24 are objective measures, while two indicators measure citizens' subjective opinions on general safety and local political-legal work, respectively. These two subjective indicators are weighted more heavily than all the other measures combined.Footnote 52 In other words, a locality's safety performance is mainly decided by the citizens' opinions.
The Office of Maintenance of Comprehensive Social Order, under the administration of the Guangdong Provincial Commission of Political and Legal Affairs, then launched an open bidding process to invite qualified official and civic organizations to conduct the surveys. This sent a clear message that more reliable information is obtained by comparing two sets of data. Five organizations submitted proposals. Eventually, the Guangdong Public Opinion Research Centre (the official research centre) and the Pearl River Delta Public Safety Research Centre (PRDPSRC) at the Guangdong Police College (a civic research centre) won the bidding. The two organizations each designed their own public-safety questionnaires and conducted their own surveys. The bidding document stipulated that the surveys had to get 13,600 samples from the 21 municipalities and 132 counties and districts in Guangdong province.Footnote 53
In early 2013, with a questionnaire they developed after extensive field research in other parts of China, the PRDPSRC conducted a survey on perceptions of local safety by directly calling selected residents above the age of 16 in each of the jurisdictions. The interviewees were selected by standard sampling methods. In total, they interviewed 26,010 residents and obtained 14,438 usable surveys. Out of 100 points, the average mark citizens bestowed for general satisfaction was 67, with the lowest at 61.2 and the highest at 81.2. About 12 per cent of the interviewees thought that their locality was “safe,” 28 per cent chose “passable,” 32 per cent chose “not so safe,” and 28 per cent chose “not safe.” In other words, about 60 per cent of the interviewees were unsatisfied with the safety situation in their localities.Footnote 54 The results are not flattering.
Guangdong province then calculated the mean of the marks gathered by the PRDPSRC and the Public Opinion Research Centre. The mean was used as the final grade for each city, county and district. A new ranking was created accordingly and was released only to the cities, counties and townships that received rewards or punishments. The top eight (out of 21) cities, and the top 30 counties and districts (out of 121) received rewards, while the bottom three cities and the last ten counties and districts were warned and penalized.Footnote 55
The above discussion provides some evidence regarding the effectiveness of each of the three bypassing strategies in improving the reliability of data. Further evidence from the NBS also shows that data distortion appears to be reduced on a nationwide scale, although it should be acknowledged that the reduction is the result of a variety of statistical reform measures taken in the past decade. In 2001, when the NBS conducted a nationwide inspection of government and civic organizations responsible for providing statistical data, it uncovered a total of 62,000 violations of the statistics law.Footnote 56 In 2004–2007, this number fell to around 20,000 per year on average. In addition, in 2001 more than 19,000 of the 62,000 cases were filed for legal investigation, whereas that number was reduced by nearly half, to approximately 10,000, by 2013.Footnote 57
Limitations of the Bypassing Strategy
Notwithstanding the positive evidence at both the national and local levels regarding their partial effectiveness, the bypassing strategies are limited in their ability to tackle information distortion in two aspects, which partly explains why information distortion continues to be so prevalent.
First, although bypassing strategies such as spot checks and third-party evaluations do improve the quality of some types of sector data, it is very costly to use these measures for all types of sector data. Therefore, the quality of sector data remains uneven. An interviewee from the construction department of Guangdong province gave an example of an instance when bypassing strategies do not ensure the quality of sector data. In 2014, the department was required to redevelop old and dilapidated rural housing. However, when the provincial leaders came to collect information on the progress of this project, the first problem they met was the ambiguous definition of “old and dilapidated housing.” Officials at different levels had a different understanding of how to carry out this project, based on their own definition. In addition, the department did not have the extra budget to invite the statistics bureaus or any other third-party organization to collect data. They had to rely on their own resources – spot checks and data reported from below. Third, local officials were not given sufficient time to conduct thorough investigations and spot checks. It would probably have taken a year to collect reliable information on this project, if indeed it was at all possible, but the province was often asked to report the data in weeks, sometimes even in hours. Under such circumstances, data distortion was inevitable. The interviewee commented,
Collection of major socio-economic development information has institutional guarantees because these institutions function for many years. Such data are relatively reliable. But the reliability of sector data depends on the situation: the individual leader's willingness, the clarity of the targets, the adequacy of human and financial resources at the grassroots level, time allowed to collect data, etc. Hence, there is no firm guarantee of the reliability of sector data, and distortion is hard to avoid.Footnote 58
Second, the effectiveness of the bypassing strategy is based on the assumption that the information providers are honest and will report the authentic raw data; however, there is no guarantee of this. For example, in 2013, the NBS exposed a case of false reporting in Henglan 横栏 township, which falls under the administration of Zhongshan 中山 city, Guangdong province. It was reported that the NBS stipulated that any enterprises which report directly to the national database should be above a designated size. The township reported that 249 industrial enterprises qualified for direct reporting, but when the NBS conducted spot checks in 73 of those enterprises, it found that 38 were not of the designated size, and 19 were non-functional or had ceased operations. The investigation also found that the township's reported industrial output (851 million yuan) was fabricated. The actual industrial output was 222 million yuan. It turned out that the officials from the township's economic development and scientific information bureau had fabricated data at their offices, and they reported the data to the national database in the name of the enterprises via the Published Switched Telephone Network (PSCN), which concealed their real addresses.Footnote 59 This case shows that local leaders can find loopholes in the online data reporting system in order to fake the identity of information providers.
On the bright side, the spot checks worked well in this case and compensated for the flaws of the online data collection system. In other cases, local officials were able to game the system of spot checks by sabotaging the collection of raw information. The lesson may be that it is more effective to use several bypassing strategies together to tackle false reporting. In addition, it is interesting to note that new technology has been adopted to verify the identity of the information provider. For example, personal digital assistants (PDAs) were used in the third economic census in 2013. PDAs can, amongst other things, give the precise geographic location of an accountant, record the personal particulars of the accountant, and send photos taken at the site and which contain the enterprise's identity information directly to the national database. Measures such as these offer better guarantees of the credibility of raw data and the identity of the information provider.Footnote 60 These sorts of backups and safety systems led Ma Jiantang, when commenting on the case of Henglan township, to declare that “some individual cases of local statistical false reporting will not affect the reliability of the macro-level data.”Footnote 61
Conclusion
This study examines the bypassing strategies adopted by Chinese leaders to reduce the artificial distortion of government performance data at local levels. It shows that these leaders are focusing their efforts on ensuring the reliability and accuracy of key information – the information that matters most for governing localities and formulating policies and plans. Unfortunately, bypassing strategies have had only limited success in ensuring the quality of certain types of sector data, where loopholes make it possible to undermine the overall effectiveness of the strategies. The battle over information distortion is tough and ongoing.
A key advantage of the bypassing strategies is that they reduce the opportunities for information distortion and collusion. Information distortion can only occur if the lower-level officials who stand to benefit from it are given the opportunity to convey the distorted information to their superiors. Bypassing strategies shift power from the information reporter to the information seeker: in some cases, the lower-level officials are deprived of the chance to report data at all (for example, data may be collected directly from the source, as when enterprises report straight to a unified national database). In other cases, although the lower-level officials still report their data, their superiors audit and verify those reports, comparing different data sets, conducting spot checks and seeking out third-party evaluations. Under such circumstances, deliberate distortions of information are more easily caught and collusion is less useful.
The adoption of bypassing strategies indicates a shifting focus in China's data-collection system. In Mao's era, the information-collection system was highly decentralized. Top-down approaches were less institutionalized and were utilized more or less according to the needs and preferences of individual leaders. The bypassing strategies of the reform era represent an institutionalization of top-down information collection. China's current information-collection system has become a hybrid of both top-down and bottom-up approaches, with the top-down approach increasingly used in collecting key information and the bottom-up approach used chiefly to provide information that is of secondary importance.
It should be noted that China's leaders have done much more to curb information distortion than this paper can cover. For example, the NBS is working on how to use the “big data” provided by internet giants such as BAT (Baidu, Alibaba and Tencent, the three leaders in China's internet and big-data industry) in statistical work. Such data are entirely beyond the control of local governments and could be additional reliable input for decision making in some policy areas. Future studies should examine the content and effectiveness of these strategies. In addition, this paper focuses on data collection, but even reliable data are wasted if not used in policymaking. How information is used in actual policymaking, target setting and disaggregation processes is beyond the scope of this paper and is another fruitful topic for future studies.
Acknowledgement
This study is funded by two grants: a start-up grant from the Faculty of Arts and Social Sciences, National University of Singapore (WBS No.: R-108-000-069-133) and the Singapore Ministry of Education (MOE) Academic Research Fund (AcRF) Tier 2 grant (MOE2015-T2-2-074).
Biographical note
Jie Gao is assistant professor at the department of political science, National University of Singapore. Her current research focuses on cadre evaluation and management, government performance measurement, citizen satisfaction surveys and work safety regulation in China.
