Social programmes can be effective only when their intended beneficiaries are able to participate. Yet, evidence suggests that many people who are both eligible and needy fail to access benefits because the bureaucratic processes to enrol and stay enrolled are too cumbersome. Across OECD countries, uptake of social assistance and housing programmes generally ranges from 40 to 80 per cent of those eligible and uptake of unemployment compensation from 60 to 80 per cent (Hernanz et al. Reference Hernanz, Malherbet and Pellizzari2004).
For income support in the USA, low uptake translates into fewer resources reaching families and children experiencing poverty. Receipt of Temporary Assistance for Needy Families (TANF), the primary income support programme for the poor, is at a historic low, and much of the recent decline in TANF receipt is due to declines in uptake, rather than changes in the number of eligible families (Falk Reference Falk2017). In 2014, the latest year for which data are available, 27.6 per cent of families who met TANF eligibility requirements participated in the TANF programme (USA Department of Health and Human Services et al. 2017c). This rate of uptake declined by well over half in less than 20 years – it is down from 69.2 per cent at the programme’s inception in 1997 and 36.0 per cent in 2007.
A growing body of public policy research demonstrates that failure to enrol in programmes is often a result of behavioural and informational barriers, rather than optimal or preferred household choices. Getting employees to start saving for retirement, for example, has been found to greatly depend on defaults (Beshears et al. Reference Beshears, Choi, Laibson, Madrian, Kay and Sinha2008). Much less is known, however, about how to support continued participation in programmes that require sometimes-onerous steps to remain eligible. When a social policy does require significant action to recertify one’s eligibility, how can social policymakers best facilitate successful recertification for eligible participants? And when deadlines come with weighty consequences, how can agencies best encourage timely recertification among needy populations?
We test whether a variety of potential avenues that social scientific theory offers can address low rates of continued participation, when recertification is difficult and deadlines are consequential. We apply insights from theories of cognitive ease, implementation intention, and deadline anchoring to try to overcome several frictions in ongoing TANF receipt for families who have already successfully engaged with the programme in the past. We detail our construction of a behaviourally informed communication and find that sending this communication increased families’ likelihood of successfully recertifying for TANF by 14 per cent. Additionally, we find that anchoring recertifying households to open rather than specific action dates is somewhat more effective.
We consider our primary contributions to be threefold. First, much of the behavioural public administration literature is about initial programme uptake, as we describe below. We focus on the much less-tilled process of recertification. There are many reasons we might expect this interaction with the state to differ from initial participation decisions: participants have more knowledge about the programme, they may have emotional responses to experiences in the programme, and they may be more motivated to act due to loss aversion. These features suggest that, if participants are already invested, recertification may be a much more difficult stage to affect with behavioural interventions than is initial application. Second, a large subset of the experimental behavioural science literature on programme continuation focuses on retirement savings and the role of defaults or health insurance and choice architecture [see, for e.g. Beshears et al. (Reference Beshears, Choi, Laibson, Madrian, Kay and Sinha2008); Johnson et al. (Reference Johnson, Hassin, Baker, Bajger and Treuer2013); Blumenstock et al. (Reference Blumenstock, Callen and Ghani2018); and Hoffmann et al. (Reference Hoffmann, Cam and Camilleri2019)]. In our work, continuation requires an onerous set of paperwork and a substantial in-person visit. It is important to assess behavioural interventions in challenging circumstances such as these. Third, where other work explores the effect of different relative deadlines (Homonoff and Somerville Reference Homonoff and Somerville2021), we hold actual deadlines constant, but test how best to frame deadlines in difficult recertification contexts – whether anchoring participants to a specific date for an in-person visit promotes more or less successful recertification than an open date.
Below, we first describe how theories of administrative burden interact with cognitive biases and the implications of this interaction for TANF operations and social programme administration more broadly. In a randomised field experiment, we test whether pathways suggested by theory can be exploited, deploying a behaviourally informed reminder letter. Finally, we discuss how behavioural insights can be used to reduce psychological frictions and increase the uptake of public benefit programmes generally. We describe how agency partners have already expanded the scope of the original intervention we test here. Our findings suggest guidance for public policy implementation and communication specificity across a wide range of social programmes, particularly those with critical deadlines and extensive documentation requirements.
Administrative burden, cognitive bias, and recertification deadlines
Administrative burden
Standard economic models traditionally attribute low uptake of social programmes to the costs of learning about eligibility and application rules, the costs associated with benefit receipt, and the transaction costs of programme applications (Currie Reference Currie2004). Literature in public policy and administration associates these costs with administrative burden, “an individual’s experience of policy implementation as onerous” (Burden et al. Reference Burden, Canon, Mayer and Moynihan2012). Moynihan et al. (Reference Moynihan, Herd and Harvey2015, p. 43) extend the definition of administrative burden to include the “learning, psychological, and compliance costs that citizens experience in their interactions with government.” In the context of social policy, learning costs refer to the time and effort it takes to learn about and access a programme; psychological costs include the stigma, loss of autonomy, and stress surrounding the programme, and compliance costs are the burdens of following administrative rules and requirements (Moynihan et al. Reference Moynihan, Herd and Harvey2015).
While some administrative burdens may legitimately serve public values, others – sometimes inadvertently, sometimes intentionally – impose costs on individuals and hinder programme goals. For example, administrative burdens have been shown to limit uptake among eligible individuals for programmes such as TANF and Medicaid, and have also been cited as major barriers to naturalisation for Hispanic immigrants, access to abortion services, and voting (Gonzalez-Barrera et al. Reference Gonzalez-Barrera, Lopez, Passel and Taylor2013; Moynihan et al. Reference Moynihan, Herd and Harvey2015; Herd and Moynihan Reference Herd and Moynihan2019). Indeed, requirements to re-establish eligibility more frequently have been consistently found to reduce participation in the United States’ Supplemental Nutrition Assistance Program (SNAP) (Ratcliffe et al. Reference Ratcliffe, McKernan and Finegold2008). Churn – when individuals enrol in a programme, exit when faced with recertification, then quickly re-enrol – suggests that the administrative burden of recertification can specifically discourage eligible individuals from accessing social supports continuously (Herd et al. Reference Herd, DeLeire, Harvey and Moynihan2013). Churn in TANF has at least two types of costs: first, participants experience discontinuity in needed income supports; second, programme staff must devote additional resources to reapplications. In the programme we study, it takes staff twice as long to process a reapplication as a recertification. For the participant, reestablishing benefits requires 3 to 4 times as much administrative interaction time. National surveys illuminate the extent to which administrative burden may be responsible for families exiting TANF or its predecessor. Ten to fifteen per cent of families exiting these programmes cited such reasons for leaving as “Did not follow programme rules” or “Didn’t want or need/too much hassle/system too frustrating” (Brodkin and Majmundar Reference Brodkin and Majmundar2010).
Cognitive bias in the face of administrative burden
Cognitive biases may generate disproportionate responses to administrative burdens. For instance, small situational factors – how decisions are framed, the small “hassle” factors associated with achieving a goal, and procrastination – may have outsized influence on programme participation decisions (Tversky and Kahneman Reference Tversky and Kahneman1981; Bertrand et al. Reference Bertrand, Mullainathan and Shafir2004; Kahneman Reference Kahneman2011). Administrative burdens have been shown to elicit negative emotions – especially confusion, frustration, and anger – which make bureaucratic interactions exhausting and more difficult to successfully navigate. These emotional reactions bias individuals’ cognitive processes as they weigh the costs of compliance against the benefits of applying to a government programme (Hattke et al. Reference Hattke, Hensel and Kalucza2020). Related constructs help explain why individuals avoid decisions, such as applying or reapplying to means-tested social programmes, that require significant paperwork. For example, ego depletion describes the draining of the individual’s finite pool of cognitive, emotional, or physical energy (Baumeister et al. Reference Baumeister, Bratslavsky, Muraven and Tice1998); hyperbolic discounting describes people’s tendency to greatly overvalue short-term consequences, leading to procrastination and decreased programme participation when deadlines pass. The level of cognitive ease associated with programme application and continuation also affects the likelihood of participation (Fox et al. Reference Fox, Feng and Stazyk2020). Populations experiencing poverty, like the one we study, are particularly susceptible to these phenomena. The stress of poverty can induce a scarcity mindset where cognitive bandwidth, and the attention available to negotiate complex processes, are limited (Mullainathan and Shafir Reference Mullainathan and Shafir2014, p. 160). Scarcity creates, as Christensen et al. (Reference Christensen, Aarøe, Baekgaard, Herd and Moynihan2020, p. 127) aptly put it, “a human capital catch-22, increasing people’s likelihood of needing state assistance while simultaneously undermining the cognitive resources required to negotiate the burdens they encounter while seeking such assistance.” Ultimately, the learning, psychological, and compliance costs associated with administrative burden can be more than a nuisance; activating cognitive biases, they can have significant impacts on citizens’ access to services, well-being, and relationships with government.
Governments can reduce administrative burden by simplifying bureaucratic processes and increasing the cognitive ease of interacting with social programmes; they can thereby increase uptake and continuation. When processes are slow to change, programme implementers can still promote cognitive ease by applying behavioural-scientific findings to interactions with participants. There is a need, as Hattke et al. (Reference Hattke, Hensel and Kalucza2020, p. 61) describe, “for public servants to counterbalance the negative emotional consequences of bureaucratic organization.”
Policymakers can limit learning, compliance, and psychological costs by reshaping state-citizen interactions in ways that positively interact with individuals’ cognitive biases (Moynihan et al. Reference Moynihan, Herd and Harvey2015) and by shifting administrative burden from citizens onto the state itself (Herd et al. Reference Herd, DeLeire, Harvey and Moynihan2013). “Seemingly minor variations in enrolment and renewal policies” can significantly reduce administrative burden (Fox et al. Reference Fox, Feng and Stazyk2020, p. 105). For example, simplifying and standardising application processes – even through small measures such as including simpler language on forms – has been shown to increase participation in social programmes (as well as public sector job recruitment) and improve service delivery (Madrian and Shea Reference Madrian and Shea2001; Behavioral Insights Team 2016; Linos and Riesch Reference Linos and Riesch2020; Fox et al. Reference Fox, Feng and Stazyk2020). In particular, sending reminder letters about government benefits has been shown to increase participation. Bhargava and Manoli (Reference Bhargava and Manoli2015) find that sending letters about the Earned Income Tax Credit (EITC) to eligible individuals increased uptake by approximately 25% and that letters with simplified layouts were more effective than complex, textually dense versions. Similarly, Finkelstein and Notowidigdo (Reference Finkelstein and Notowidigdo2019) find that elderly individuals who received simple reminder postcards were more likely to enrol in SNAP.
Anchoring deadlines
A critical component of TANF recertification is timeliness. If a household fails to successfully recertify by the end of the relevant month, they must file a new initial application, churning to recover benefits. Deadlines – even artificial or self-imposed deadlines – induce people to avoid postponing action repeatedly (Ariely and Wertenbroch Reference Ariely and Wertenbroch2002; Zamir et al. Reference Zamir, Lewinsohn-Zamir and Ritov2017). They may also spur action by signalling scarcity. Generally, anchors may influence decisionmakers to make judgments biased towards an initially presented value (Furnham and Boo Reference Furnham and Boo2011). While the literature on anchoring is strong, particularly for judgments around knowledge, probability estimates, forecasting, legal judgments, valuations, negotiation and self-efficacy, less is known about how dates may anchor decisions on when people will act. As a result, we aim to contribute evidence to the open question of how programme administrators should frame administrative deadlines.
Many studies of deadlines and reminders are based on appointments and highlight the importance of citing specific dates and times. For example, Anzelone et al. (Reference Anzelone, Timm and Kusayeva2018, p. 12) attributes the success of their intervention to “(1) creating a clear deadline for action by setting an appointment; and (2) orienting the rest of the mailed materials to help a parent prepare for and attend the appointment.” Similarly, the medical literature finds consistent evidence that reminders which include the date, time, location, and contextualising information increase appointment attendance (McLean et al. Reference McLean, Booth, Gee, Salway, Cobb, Bhanbhro and Nancarrow2016).
In our setting, however, there are not true appointments; required interactions with programme staff are first come first served. In the absence of appointments, the evidence is mixed on whether listing a specific date, rather than a more general one, better overcomes procrastination. However, existing research also lacks a pure test. A study asking students to voluntarily provide exam feedback at Hebrew University found that students were more likely to respond if they were given a short, specific deadline (Tuesday, July 23 at noon) than if they were given a long, imprecise deadline (“during the coming weeks”). Unfortunately, this intervention design conflates the length and precision of the deadline (Zamir et al. Reference Zamir, Lewinsohn-Zamir and Ritov2017). A different study of deadlines provides some evidence that length of the deadline does not matter, but offers no insights into precision. In response to appeals from a known charity, most people donated immediately after the appeal was made, or a reminder sent, regardless of the length of the deadline (Damgaard and Gravert Reference Damgaard and Gravert2017). In both cases, the presence of the deadline, rather than its content, seems important. We seek to fill this gap in prior work and isolate the causal effect of deadline precision.
In particular, for administrative deadlines, which are often based on the end of specific periods (months, quarters, calendar or fiscal years), it is not clear whether people respond with more timely action to specific deadlines (e.g. Saturday, 30 September 2017) or less precise ones (e.g. September 2017). A license plate renewal project tested the impact of a reminder postcard that emphasised the expiration month and then asked recipients to select specific dates for performing required actions. The researchers posited that part of their success in increasing timely renewal was due to the postcards setting “an early anchor – the month [the recipient’s] license expires rather than the last date to renew” (Behavioral Insights Team 2016). However, in the absence of data on the dates of renewal, it is not clear whether these individuals’ anchor is an early one (e.g. by September 1) or a late one (e.g. by September 30). To summarise, while a specific date reminder may simplify follow through for the recipient by setting a clear anchor for action, recipients may be discouraged if they are unable to make that date. On the other hand, setting the month as an anchor (an open date) may invoke less urgency or reinforce procrastination to the end of the month. We provide evidence that, in fact, participants given less precise deadlines recertified earlier.
Adding to the existing experimental literature, our study informs theories of motivation, showing that an early, but open, deadline anchor date may best help participants overcome procrastination. It appears to do so, while still encouraging participants to complete the paperwork even after the anchor period has begun. Our study shares some features with Bhargava and Manoli (Reference Bhargava and Manoli2015) (such as simplified letters) and Lopoo et al. (Reference Lopoo, Heflin and Boskovski2020) (such as a recertification setting with intensive paperwork). However, our combination of testing open and specific deadline anchors, using the common social programme communication mode of letters, the recertification setting, and the requirement of an in-person visit appears rare in the literature.
Despite the uniqueness and challenges to participants in our experimental setting, our primary hypothesis was that our behaviourally informed communications would increase TANF families’ likelihood of successful recertification. If this communication could be effective, we sought to learn whether anchoring participants to the deadline month or to a specific date could better promote programme continuation. From the mixture of imperfect findings described above, it was unclear which direction would be implied by theory. Thus, our study asks two primary empirical questions:
• Is there a difference in successful recertification between families who are sent a behaviourally informed reminder letter and those who are sent standard communications only?
• Among those who are sent a reminder letter, is there a difference in successful recertification between families who are sent a reminder letter emphasising the deadline month (open date) and those emphasising a specific date?
Methods
Designing the intervention
Nationwide, families required to recertify for TANF face many barriers to successfully doing so. Footnote 1 In Washington, DC, during the first quarter of 2017, high levels of churn suggested that these barriers needlessly created unstable support for families and additional burden for programme staff. More than one-third (34 per cent) of families whose benefits were terminated successfully reapplied within 90 days. Moreover, this figure excludes families who might have remained eligible under rules such as income disregards that were more generous when recertifying than when reapplying (see Appendix A). In the absence of the recertification burdens, at least this 34 per cent would likely have remained eligible and continued their participation uninterrupted, without needing to start over and reapply.
DC TANF is administered by the Department of Human Services (DHS), which operates several Service Centers to accommodate required in-person visits for TANF application and recertification. DC TANF households encounter learning costs such as identifying the nearest Service Center, understanding which documents are needed, and parsing eligibility requirements. Since recertification documentation had to be supplied in-person at the time of the study, compliance costs include everything associated with planning a trip to the Service Center (e.g. scheduling, transportation, and child care) as well as assembling proper documentation. There may also be psychological costs associated with recertification, such as stigma around receiving government benefits or a sense of diminished autonomy. These costs could contribute to ego depletion and procrastination among recipients, who experience the stress of poverty and therefore may have more limited attention available to negotiate the complex application process (Mullainathan and Shafir Reference Mullainathan and Shafir2014). Using chunking, implementation intention, loss framing, simplification, and a checklist to facilitate organisation and a gradual sense of accomplishment, the reminder letter intervention in this study aims to increase the proportion of families who successfully recertify for TANF by reducing the learning and compliance costs associated with the recertification process.
We attempt to facilitate the first step in a multi-step recertification by providing families with specific, relevant information about logistics and deadlines in the TANF reminder letter. DHS chose to send physical letters for three reasons: similar letters have proven promising in similar contexts (Bhargava and Manoli Reference Bhargava and Manoli2015; Farrell et al. Reference Farrell, Smith, Reardon and Obara2016), most DC TANF recipients can receive letters (as opposed to text messages, e.g.), and letters are low-cost. It was not possible to quickly revise, legally approve, and test revisions to the official notices.
Informed by the theory of administrative burden and behavioural science, we designed a reminder letter to reach households facing recertification. As a first step to designing this intervention, we conducted a behavioural audit to better understand the potential bottlenecks from the participant’s perspective (see Figure 1). This audit involved a detailed analysis with several DHS staff as well as a visit to a DHS Service Center. With DHS staff, we enumerated every physical and cognitive step required for a participant to successfully recertify, from recognising the need to recertify to being present in front of a frontline staff member with the correct paperwork in hand. During the Service Center visit, we walked through the recertification experience with frontline staff members and specified exactly the documentation required. The audit sought to systematically uncover the participant’s experience of the concomitant administrative burdens – specifically the learning and compliance costs – of recertification. We mapped out each required step and the associated challenges that could be addressed by a letter.
Figure 1. Behavioural audit.
Figure 1 displays these steps, the challenges participants might encounter in completing each, and aspects of the letter designed to help overcome these challenges. First, the letter needed to stand out sufficiently to be noticed, but also clearly convey its nature as an official communication from a government agency. Next, the recipient needed to understand the letter, the consequences of not complying with recertification, and, critically, the steps required to comply. In order to facilitate this, our intervention attempts to reduce the cognitive demand by employing a clear statement of consequences resulting in loss, chunking the material into discrete sections, prioritising the needed steps, and providing checkboxes to promote self-efficacy. Then, the recipient needed to intend to fulfil the requirements. To promote this, we include an explicit implementation prompt asking, “How will you get here?” Finally, the recipient needed to follow through on these intentions by appearing for an in-person meeting. A map of the nearest Service Center makes clear where to do so. At each step, visual and text components of the letter sought to enable the participant to recertify with minimal cognitive load.
Because of constraints around possible interventions, this process sought to uncover, but could not address, challenges like travel costs and access to childcare, or factors that would make the participant ineligible for DC TANF (such as moving out of DC). We then identified the research-based design insights that might address the key cognitive bottlenecks and applied these insights to our letter design.
The behavioural audit resulted in two versions of a reminder letter. Both letters are designed as a trifold
$8{1 \over 2}$
” × 11” sheet (see Figure 2 and Supplementary Figure 7). The outside of the trifold (the “envelope”) is designed to attract recipients’ attention and generate sufficient interest to open the letter. While there is limited empirical consensus on what design elements are likely to drive open rates for official communications, evidence from direct mail campaigns suggests that visual design (e.g. special formats or sizes), a clear sender identity, and personalisation may all affect whether recipients open mail (Feld et al. Reference Feld, Frenzen, Krafft, Peters and Verhoef2013). We incorporate those elements in our design.
Figure 2. Reminder letter.
The inside of each letter starts with a simple statement to motivate the head of household to recertify in a timely fashion: “Your family’s cash benefits will expire this [month] unless you renew your TANF eligibility,” then lays out the three steps required for recertification. We break the text into steps because research suggests that recipients may benefit from signposting to engage with all key points. In an Indiana study, notices that provided parents with more detailed information and a checklist clearly explaining the required documentation led to more parents recertifying their eligibility for child care subsidies (Dechausay and Anzelone Reference Dechausay and Anzelone2016). To help recipients parse the task and lower learning and compliance costs, we divide the “ask” into discrete steps and transfer the administrative burden of figuring out the appropriate paperwork (e.g. which pay stub) from the recipient to the service provider.
Each step includes an implementation prompt to encourage follow through and increase recipients’ engagement in the process. Implementation prompts – asking people to plan – may make them more likely to follow through as they cause people to think more deeply about accomplishing a task than they otherwise would. These prompts take the discrete steps described above and ask recipients to perform concrete actions immediately. In the literature, successful outreach letters from an agency asked Federal Health Insurance Marketplace applicants to write on the letter the time and date they planned to return to an online form to complete enrolment (White House Office of Science and Technology Policy and Subcommitte on the Social & Behavioral Sciences Team 2015). Such prompts may mitigate forgetfulness, cause discomfort for failing to follow through, and outweigh people’s tendency to procrastinate by encouraging them to develop specific strategies to overcome logistical hurdles and follow through on their intentions (Gollwitzer Reference Gollwitzer1999).
The letters also include simplified instructions (e.g. required documents and a map to the nearest Service Center) to limit the informational costs of compliance. The concise, direct language, and illustrations are designed to increase the cognitive ease of reading the letter, since long and dense text taxes readers’ limited bandwidth. Numerous studies of communications to public benefits recipients and public sector job applicants have shown that simplified forms and applications can increase participation (Linos and Riesch Reference Linos and Riesch2020; Fox et al. Reference Fox, Feng and Stazyk2020). In Georgia, successful redesigned outreach notices for child support cases simplified language and used formatting to make the most important information prominent (Anzelone et al. Reference Anzelone, Timm and Kusayeva2018). Presenting citizens with a set of limited choices (e.g. options for transportation to the nearest Service Center) can also mitigate choice overload and prevent indecision or poor decisionmaking (Moynihan et al. Reference Moynihan, Herd and Harvey2015).
While our letter versions are largely identical, they differ in how they emphasise the deadline for action: one has an open date and the other a specific date (see Figure 3). In the first step, the open date version anchors recipients on the month their benefits are scheduled to end and asks them to “Set a Date” to recertify. The specific date version anchors recipients on the suggested date from DHS’s legally required Initial Notice and asks them to “Save the Date.”
Figure 3. Reminder letter detail: open date and specific date versions.
To test the effects of the behaviourally informed reminder letters, we randomly allocated one-third of those needing to recertify each month to receive the standard communications only (no reminder letter), one-third to receive the additional letter with a specific date, and one-third to receive the additional letter with an open date.
Participants
Trial participants included all 3,539 households required by DHS to recertify their TANF eligibility by July, August, September, October, or November 2017 who were eligible for reminder letters.
Outcomes
We drew all data for the study from DHS’s administrative benefits eligibility system. Individual records are updated in the system as TANF recipients provide eligibility information, such as during their recertification visits. All recertification outcomes are measured at the end of the recertification period, that is, after the last day in which a household was able to recertify their eligibility; these were then first shared with the study team in late December 2017.
Our primary outcome of interest is whether the household successfully recertified. We define this as a dichotomous measure of whether each trial participant’s household submitted all necessary proofs for DHS to determine their continued eligibility for TANF and were deemed eligible by DHS. We measure each household’s outcome once their recertification deadline has passed.
Secondarily, we examine the temporal pattern of successful recertification in each condition, calculate estimates that account for undeliverable mail, estimate effects on intermediate steps of the recertification process, and consider administrative costs.
Randomisation
Each month, a cohort of households was due to recertify and thus eligible for participation in our study. Within each cohort, we grouped eligible households by their nearest Service Center and assigned visit date. In particular, among those assigned to recertify at each Service Center each month, we created trios based on households’ assigned appointment dates using the blockTools package in R (Moore and Schnakenberg Reference Moore and Schnakenberg2016; R Core Team 2018). Within these trios, we then randomly assigned one household to no reminder letter, a reminder letter with an open visit date, and a reminder letter with a specific visit date.
This trio creation (blocking, in the literature) was exact with respect to Service Center (i.e. each trio’s three households were associated with the same Service Center). With respect to visit dates, this blocking was approximate, but nearly exact (i.e. each trio can vary slightly in its three assigned visit dates). Footnote 2 Since visit dates cover the 25th of the prior month to the 14th of the recertification deadline month, there were about 21 days and thus 15 weekdays of appointments each month.
Our blocking produced treatment conditions that are very well balanced across appointment dates. Across the 25 Service Center-by-month combinations with more than a handful of participants, the median absolute deviation (MAD) of appointment dates is about 0.15 days (about 3.5 hours) or less in 99% of the trios. In other words, the medians of the treatment and control groups tend to vary by much less than a day across the months and Service Centers. Footnote 3 Thus, any differences we observe between treatment conditions cannot be the result of differences in appointment dates.
Of the 3,539 households in the sample, 1,177 were assigned to receive no letter; 1,181 to receive a reminder letter with an open visit date; and 1,181 to receive a reminder letter with a specific visit date. Footnote 4
Blinding
There is minimal chance that knowledge of a household’s study group affected outcomes. The research team, including administrative staff at DHS, carried out the random assignment and electronic production of the behaviourally informed letters. As a result, during the study, only the research team had access to the data files identifying which participants were sent each reminder letter. DHS administrative staff, who printed and mailed the letters, could observe treatment status based on the individual letters but were unlikely to know experimental status systematically across the sample. Footnote 5 None of the staff who processed or input data on recertifications were aware of the experiment, nor which families were in the study sample. Individuals who received a reminder letter could have shown that letter to DHS staff during recertification, but because the business rules and processes for making final eligibility decisions are rigidly structured, it is highly unlikely that eligibility decisions were influenced by any staff awareness of the reminder letter.
Statistical methods
For each research question, we estimated the causal effect of reminder letters or letter type (open versus specific date) with the least squares linear regression coefficient on an indicator for letter or letter type. We used the standard error around the estimated coefficients to construct 95 per cent confidence intervals around the estimates. For each research question, we also estimated the Bayesian posterior probability that one condition results in more successful recertifications.
Results
Of the randomly assigned sample, our intent-to-treat analysis included 1,172 households assigned to receive the standard communications only and 2,348 households assigned to standard the usual communications plus either version of the reminder letter. This included 1,176 households assigned to receive a reminder letter with open date, and 1,172 households assigned to receive a reminder letter with the specific date.
Attrition was minimal and included just those households whose outcome data could not be matched to the baseline sample. For three per cent of the sample with observed outcomes, participation was terminated prior to the deadline separately from the recertification process. This happened, for instance, if DHS knew from their participation in another benefit programme that they were no longer eligible for TANF. Because it is not obvious that the terminations were independent of the treatment (e.g. whether the household was identified because the reminder letter led them to contacting DHS), we leave these cases in our sample. To be conservative in our estimates, we categorise these households as unresponsive to the recertification requirement.
Unless otherwise specified, each of the following analyses was preregistered with the Open Science Framework following best practices for conducting and preregistering field experiments (Gerber and Green Reference Gerber and Green2012; CONSORT and Transparent Reporting of Trials 2016). The registration is available at https://osf.io/ae6n5/. Replication materials are available in Moore et al. (Reference Moore, Gan, Minnich and Yokum2022).
We find that the combination of cognitive aids in our behavioural reminder letters did increase successful recertification. Among those who received the standard communications, 40.1 per cent successfully recertified (Figure 4, left panel, and Supplementary Table 2). Among those receiving an additional reminder letter, 45.6 per cent successfully recertified, a difference of 5.5 percentage points. This represents an increase in recertifications of 14 per cent over the baseline. Because the 95% confidence interval around the difference covers (2.1, 9.0), we reject the null hypothesis that there is no difference in successful recertification between those sent a behaviourally informed reminder letter and those who were not. Further, the Bayesian posterior probability that a behaviourally informed letter is more effective than the standard communications only is approximately 1.
Figure 4. Percentage successfully recertifying, by experimental condition.
Our results also suggest that open date letters, which only note the overall deadline month, not the specific date, generate higher recertification rates than do those listing a specific visit date (Figure 4, right panel, and Supplementary Table 3). Among those sent the open date letter, 46.4 per cent successfully recertified. Successful recertification was 44.8 per cent among those sent letters listing specific dates. This represents a difference of about 1.6 percentage points or an increase in recertifications by the open date letter of 4 per cent over the specific letter.
While this difference is suggestive, we do not reject the null hypothesis because the 95 per cent confidence interval around the difference of 1.6 percentage points includes zero (-2.4, 5.7). We also calculate the Bayesian posterior probability that the open date letters are more effective than the letters with the specific date to be about 0.79. That is, there is a 79 per cent chance that the letter with the open date is more effective at driving successful recertification than is the letter with the specific date.
At current TANF recertification levels (12,106 households required to recertify in 2017) and using current administrative procedures, sending the same mixture of the two additional letters could amount to 667 additional families successfully recertifying over the course of a year. Using the open date letter could amount to as many as 766 additional families retaining benefits.
Supplemental analyses
Recertification days and dates
To better understand how households respond to the two versions of the letters and implications for DHS operations and staffing, we conducted an analysis of the specific days and dates households recertified. These analyses were not preregistered. Figure 5 presents the cumulative distribution functions (CDFs) of recertifications using two different reference time periods. Footnote 6
Figure 5. Days and dates of recertification activity.
The left panel of Figure 5 shows when households recertified relative to the specific date they were given in their Initial Notice and, for some, in their Reminder Letter. The specific dates ranged over approximately a 20-day period, slightly more than midway through the recertification window. Households with the open date reminder letters were more likely to recertify prior to the specific dates in their Initial Notices (12.0%) than were those reminded of the specific date (8.5%) or who received the standard communications only (6.6%). The story shifts at the specific “appointment” date. Those reminded of a specific date were more likely to recertify on that date (8.6%), than those with the open date reminder (6.4%) and the standard communications (6.6%). This difference is sizable enough to almost offset the early recertifications from the open date letter. After the specific date is passed, households that receive the open date and specific date reminder letters appear to act similarly. While anchoring individuals to specific dates may encourage a subset to act on that date, it appears that “appointments” may also discourage individuals from acting early.
The right panel shows when households recertified relative to the 60 calendar days available for recertification, from the Initial Notice to the deadline. We particularly note how individuals act at the end of each month. At the far right, there is a sharp increase in recertification activity by all three groups as they approach the end of the deadline month (0 days). More surprising, it appears that there is a similar increase among households with the open date reminder as they approach the beginning of the deadline month – or end of the preceding month (
$ \approx - 30$
days). This suggests that some individuals may interpret the broad month (e.g. July) in the open date letter as an early deadline for action (e.g. before July 1). There is also suggestive evidence of that date serving as an early anchor. Among those who received the open date letter, those whose Initial Notice dates were in the first month (e.g. June 25–30) recertified an average 6.0 days later than those dates; those whose Initial Notice dates were in the second, deadline month (e.g. July 1–14) recertified an average 0.3 days earlier than those dates. Supplementary Figures 9 and 10 display the noncumulative distributions.
Return to sender
The intervention relied on physical mail in part because addresses were available for all of the study population, while other contact information was more sparse. However, some addresses for this population were out of date or otherwise incorrect, undermining receipt of the intervention. Of the 2,362 letters we sent out, we know that 230 did not reach the intended recipient. The postal service returned these letters either due to an incorrect address (216) or insufficient postage (14). Letters with open appointments were slightly more likely to reach the intended recipients. Among those with open appointment letters, 9.0 per cent were not delivered due to an incorrect address; an additional 0.5 per cent were not delivered due to a missing stamp. The analogous figures for those with specific appointment letters were 9.4 and 0.7 per cent, respectively.
While our main estimate is the intent-to-treat effect, we also calculate the effect of the letters on only those who had a chance to receive them – a local average treatment effect or complier average causal effect (Angrist et al. Reference Angrist, Imbens and Rubin1996; Gerber and Green Reference Gerber and Green2012). This estimate was not preregistered. The noncompliers represent a relatively small proportion of the treatment group (about 10 per cent of the intended recipients), and our estimate of the complier average causal effect is similar to our overall average treatment effect. Compliers were 6.1 percentage points more likely to successfully recertify and continue TANF participation than those in the control group.
Additional outcomes
While the primary outcome of policy interest is continued participation in TANF, we also analysed the impacts of the letters on two intermediate outcomes. Successful recertification requires both (1) being sufficiently motivated and informed to show up at a Service Center to recertify and (2) being sufficiently informed and organised to bring the proper documentation. Each of these is necessary, but not sufficient steps, to successful recertification. Footnote 7
The reminder letters’ impacts on starting and completing recertification are consistent with their impacts on successfully recertifying (Supplementary Tables 2 and 3). The behaviourally informed reminder letter empowers recipients to initiate the recertification process, which, in turn, usually leads to those families continuing their participation in TANF. Very few households attempt to recertify and subsequently fail to retain benefits. Specifically, households sent an additional letter were 13 per cent more likely to start recertification and 13 per cent more likely to fully comply with recertification requirements (provided all required proofs even if they did not meet eligibility criteria) than are those sent only the standard notifications only. The letters have a statistically significant impact on all steps in the recertification process. Considering just those sent letters, those with the open date are slightly more likely to start recertification (3 per cent) and fully comply with requirements (3 per cent) than are those with the specific date, but these differences are not statistically significant.
Cost-benefit analysis
Finally, to inform whether the letters were worthwhile, we present a rough cost-benefit analysis. This analysis was not preregistered. The reminder letters led to 129 more study families with low incomes keeping their TANF benefits. The reminder letters themselves cost $1,087 to send, not counting staff time costs or materials (paper, ink) DHS already owned. Footnote 8 This translates into $8.43 per additional family that retained its benefits. Stated differently, the $6,000 annual cost of sending the letters to all recertifying families in DC represents less than 0.005 per cent of what DC spends on TANF each year. This calculation does not include the additional savings in staff time that successful recertifications entail. In the study, these savings would amount to roughly 77 Service Center staff hours.
Limitations
Informed by social science work in administrative burden and behavioural science, our supplementary reminder letter was designed to be simple, with clear action steps for a reader who is presumably overwhelmed with many other things. However, we still have added another communication from the government, as opposed to replacing the original notifications. This reflects the pragmatic fact that we could not, for legal reasons, gain clearance to change the original letter for the study cohorts. Our additional communication could have decreased communication effectiveness if recipients had become overwhelmed or annoyed by the volume of mail. However, the variation in impacts across reminder letters suggests that the content, not just the volume, of the communications matters.
Relatedly, we rely on others’ findings that presentation and contextualising, motivating content matters beyond the mere presence of a reminder (Bhargava and Manoli Reference Bhargava and Manoli2015; McLean et al. Reference McLean, Booth, Gee, Salway, Cobb, Bhanbhro and Nancarrow2016). Based on these findings and statistical power concerns we had at the design stage, our experiment does not include a fourth condition in which some participants received a nonbehaviourally informed “pure reminder” after the Initial Notice. Thus, our findings should be understood as demonstrating the effect that a behavioural communication can have above just a nonbehavioural Initial Notice, but not relative to a second nonbehavioural notice.
Our finding that recipients are more responsive to the letter with the deadline month, rather than a specific date, may be limited to reminder letters. The comparison of the specific date and open date letters is meant to provide a window into whether it is better to give a personalised, specific visit date or instead provide more flexibility. That said, all recipients are sent a specific visit date in their Initial Notice. Thus, the test here is not strictly of anchoring to no visit date versus to a visit date, but rather of whether or not the supplementary letter re-anchors recipients to a specific visit date. As a result, our estimates are likely to underestimate the benefits of the open date.
While our intervention attempts to overcome several behavioural barriers to successful recertfication, it may not be targeting the most serious hurdles. For example, if recipients all know exactly where the nearest Service Center is, but cannot get to it during its open hours, then the cognitive assistance provided by the map will not help. A stronger intervention directly targeting this access barrier would be required. Though such interventions might be able to further increase successful recertification, they are likely to be more expensive and often beyond limited agency resources.
Last, our experiment necessarily takes place in a single jurisdiction. However, like other important social policy experiments, two generalisation conditions are met here: programme rules comply with national standards and the behavioural phenomena occur widely across policies. Finkelstein et al. (Reference Finkelstein, Taubman, Wright, Bernstein, Gruber, Newhouse, Allen and Baicker2012, Reference Finkelstein, Hendren and Luttmer2019) and Finkelstein and Notowidigdo (Reference Finkelstein and Notowidigdo2019), for example, describe significant results in single-state social policy experiments of national programmes.
Discussion
This study reinforces and builds upon existing research showing that communication that reduces cognitive load can overcome administrative burden to improve access to public benefits. Moynihan et al. (Reference Moynihan, Herd and Harvey2015) suggest that public administration literature should uncover mechanisms to minimise administrative burdens or shift them from citizens to the state. Our study reveals the effectiveness of one such mechanism – a reminder letter with simple language and specific implementation prompts – in increasing TANF recertification rates in Washington, DC. While other studies have also examined the impacts of physical reminder mailings on enrolment in public benefits programmes such as SNAP and EITC, this study makes several additional contributions to the literature. First, while previous studies primarily encourage eligible individuals to apply for benefits they do not yet receive, we focus on existing recipients who must complete a challenging recertification process in order to continue receiving benefits. Herd et al. (Reference Herd, DeLeire, Harvey and Moynihan2013) hypothesise that reducing the administrative burden of this type of process could either reduce exits from social programmes or could encourage participants to drop out because they perceive them as less difficult to return to. Our results provide evidence that the former hypothesis is more accurate: reductions in cognitive burden decrease, rather than increase, programme exits. These findings have important implications for policy practitioners, since churning of participants in and out of programmes has two types of costs: households suffer from discontinuity in needed flows of support, while street-level bureaucrats and their clients are required to spend resources on longer, more extensive reapplication procedures that could be avoided. Given that it can be inexpensive to avoid churn, governments should prioritise continuous maintenance of benefits for existing recipients in addition to enrolling new families.
Second, our study highlights the promise of incorporating human-centred design into government processes. The reminder letter in the study was designed using insights from a behavioural audit and attempts to address administrative burdens citizens face during the recertification process. Human-centred design can expose specific learning and compliance costs (e.g. the cognitive burden of planning a trip to a Service Center or understanding which documents are required) and lead to solutions that increase the cognitive ease of application or recertification. Behaviourally designed reminder letters may prove to be a cost-effective means for reducing administrative burdens, particularly in cases where governments or researchers are constrained from enacting more sweeping changes.
Third, adding to the numerous studies that have shown the promise of deadlines to spur action, this study adds nuance to the evidence on how deadlines should be used. In cases with administrative deadlines, we offer suggestive evidence that anchoring people to a deadline month may (1) be sufficient to overcome procrastination and (2) do so while avoiding the downsides of anchoring people to a specific day that does not work for them. Even if open dates are only marginally more effective than specific dates, they may be administratively or operationally preferable. Specific appointment dates may be more logistically challenging to schedule than establishing a walk-in policy. Overall, as Homonoff and Somerville (Reference Homonoff and Somerville2021) argue, recertification deadlines that allow for less scheduling flexibility can negatively impact future participation.
Fourth, our article strengthens the foundations of important meta-analytic work in red tape, administrative burden, and related concepts. As George et al. (Reference George, Pandey, Steijn, Decramer and Audenaert2021) points out in its meta-analysis of the effects of red tape on employees and organisations, surprisingly little of the published work in this conceptual area is experimental – experimental evidence features in only two of the 25 papers in George et al. (Reference George, Pandey, Steijn, Decramer and Audenaert2021). As DellaVigna and Linos (Reference DellaVigna and Linos2022) show, scholars should cast a wide net in such meta-analyses, as even experimental published results differ systematically from unpublished ones in public policy.
Finally, this study demonstrates the value of embedding social scientific research within public policy implementation. The research team and DHS were close partners throughout the research process and afterwards. After the conclusion of the experiment, the research team continued to support DHS, creating open date reminders for subsequently recertifying families. After we presented final findings to DHS, senior staff there decided to adopt a policy of mailing the open date letter to all recertifying families going forward. DHS then relied on training and assistance from the research team to take over this process themselves. Having taken ownership of a new policy that it helped design and rigorously evaluate, DHS then decided to expand the reminder mailer policy to another foundational social policy, the Supplemental Nutrition Assistance Program. Furthermore, experience with this study inspired the Department to take on other data-driven behaviourally informed practice improvements, including developing a user testing plan for a coaching website tool to implement with case managers, simplifying the common application form used to apply (and recertify) for TANF, SNAP, and some medical benefits, and independently conducting a difference-in-differences analysis of recertification rates for those receiving full versus partial TANF benefits. DHS used the last of these to decide to eliminate their 60-month time limit’s administrative burdens. As a result of this study and these related improvements, TANF recertification has been significantly higher – about 70% during the poststudy period. Ultimately, the practice of conducting behavioural audits, designing customised and research-informed interventions, and then testing their efficacy should be applied to a wide range of programme communications and policy issues. Our test demonstrates that, for relatively minimal costs – under $9 per additional family – well-designed reminder letters can help ensure that eligible families in poverty retain access to needed public resources and, in turn, can help public programmes meet their anti-poverty goals while simultaneously reducing burdens on administrative staff.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0143814X21000131
Acknowledgements
The Lab @ DC would like to acknowledge the support and contributions of the DC Department of Human Services Director Laura Zeilinger and the Economic Security Administration staff, particularly Administrator Anthea Seymour, Senior Policy and Program Advisor Brian Campbell, Deputy Administrator for Innovation and Change Management Stephanie Bloch, and Deputy Administrator for Data Analytics, Research, and Evaluation Won-ok Kim. We thank Julia Greenberg for outstanding research assistance and Margaret E. C. Moore for graphic design work.
Data availability statement
Replication materials are available in the Journal of Public Policy Dataverse at https://doi.org/10.7910/DVN/5WTSUZ
