2.1 Goals of retrospective review

Sometimes the goal of retrospective review is to “clean up the books” by identifying rules that are outdated, redundant, or obsolete – no longer applicable, or lacking statutory authority – and removing them. We refer to this goal as the rule relevance goal.

Another frequent goal of retrospective review is to improve the outcomes of regulation – in particular, to revise each rule, taken one at a time, to improve its performance. We refer to this goal as the rule improvement goal. In practice, this has often meant identifying specific past rules that have turned out to pose high costs and seeking to reduce those costs through revisions.

When pursuing the rule improvement goal, there are choices as to which outcomes to analyze and the scope of impacts to assess. Retrospective regulatory analyses can in principle assess the performance of a rule against several outcome criteria, potentially including: costs, benefits, rule effectiveness, cost-effectiveness, ancillary impacts, social well-being, and distributional equity.

Broadening the scope of analysis to cover more impacts will likely raise the costs of data collection and analysis, but will often also raise the value of the information for decision making and thus result in greater improvements in social outcomes from the review (Wiener, Reference Wiener1998).

The above objectives focus on evaluation and revision of single rules. A different objective of retrospective analyses may be to learn from multiple past rules and analyses, in order to improve future rules and analyses. We refer to this goal as the regulatory learning goal. Retrospective review can contribute to broader learning about regulation in several ways.

First, retrospective review can improve our understanding of the performance of alternative policy designs or instruments, to evaluate how well they actually work in practice compared to predictions in theory. For example, retrospective review has enabled better understanding of the relative performance of technology standards, performance standards, information disclosure, taxes, tradable permits, and other policy instruments.

Second, retrospective review could help improve the accuracy of methods used to conduct ex ante regulatory impact analyses (RIAs). Retrospective reviews can be used to compare regulatory forecasts and counterfactual scenarios with actual outcomes over time, and this information could then be used to improve forecasting methods.

Finally, retrospective review could lead to improved understanding of the interaction effects of multiple regulations. For example, retrospective review could assess the cumulative impacts of multiple rules on an industry. And retrospective review could be used to understand a broad range of interaction effects among rules.

Such efforts at regulatory learning could involve not just one lookback exercise to conduct retrospective review and improve the rule, but an ongoing and planned process of monitoring, data collection, periodic reviews and adaptive updating (McCray et al., Reference McCray, Oye and Petersen2010; Bennear & Wiener, Reference Bennear and Wiener2019a). And – whether or not they are repeated in an ongoing planned adaptive process – such efforts at regulatory learning would involve retrospectively assessing not only one rule at a time, but larger representative samples of multiple rules and their ex post RIAs, compared to their ex ante RIAs, with variation across policy designs in order to test their comparative performance, and/or with variation across forecasting methodologies in order to test and improve the accuracy of the methodologies (Greenstone, Reference Greenstone, Moss and Cisternino2009; Office of Information and Regulatory Affairs, 2016, p. 6; Wiener & Ribeiro, Reference Wiener and Ribeiro2016a).Footnote ¹ Such a broader multi-rule learning process may go beyond the domain of each individual agency, to include multi-agency comparisons. It amplifies the tradeoff of greater costs of information versus greater value of information for policy improvement. The more a regulatory system relies on ex ante RIA to design and approve new rules, the more it can gain from ex post RIA to improve the design of those rules and the accuracy of forecasting their impacts (Coglianese, Reference Coglianese2012, p. 65; Aldy, Reference Aldy2014, pp. 22–26). Adam White argues that “retrospective review’s greatest virtue actually has nothing to do with repealing regulations. Rather, retrospective review’s greatest value is forward-looking … to confront how accurate or inaccurate the agencies’ own projections were in forecasting the rules’ impacts in the first place” (White, Reference White2016).

While retrospective review has benefits, it also has costs (Bennear & Wiener, Reference Bennear and Wiener2021). Agencies must spend time and resources engaging in reviews. In addition to the direct resources costs of these reviews there is an opportunity cost in staff and resources not being allocated to other activities. Agencies may appear reluctant to review prior decisions as there may be reputational costs associated with revising prior rules – in essence it may be viewed as agency error which weakens credibility with stakeholders. While revising rules may not represent error in the original rulemaking – circumstances can and do change – these reputation costs may still be real (Bull, Reference Bull2015; Wiener & Ribeiro, Reference Wiener and Ribeiro2016a; Bennear & Wiener, Reference Bennear and Wiener2021). Retrospective review may also impose costs on society. Conventional wisdom may be that regulated actors would want to see rules changed if they prove overly costly, and certainly that can be true. But it can also be true that having spent the money to comply with a regulation, industry does not benefit from revisiting it, and repeated changes to rules can yield instability that is costly both to regulated actors and to those benefitting from regulatory protections, when they rely on the stability of past rules (Bennear & Wiener, Reference Bennear and Wiener2019b).

There may also appear to be tradeoffs among different types of regulatory analysis, for example, whether agencies should spend more time and effort on improving prospective analysis or on improving retrospective analysis. This tradeoff may be illusory as the two types of analysis can be complements. As articulated, the regulatory learning goal uses retrospective analysis in large part to improve future prospective analyses and rulemakings. But for more typical retrospective analyses that focus on rule relevance or rule improvement, there may well be tradeoffs between time spent on retrospective analysis and time spent improving prospective analysis.

Recognizing both the benefits and costs of retrospective review, we argue that reviews should be targeted where the net benefits of review are the highest. For the rule relevance and rule improvement goals, these are likely to be rules applying to industries, technologies, or scientific understanding that are changing more quickly. For those cases, the static rulemaking process may result in rules that grow increasingly mismatched to circumstances over time (Bennear & Wiener, Reference Bennear and Wiener2019a, Reference Bennear and Wiener b). For the regulatory learning goal this is likely to involve rulemakings that raise methodological issues or policy choices that apply to many rules and many agencies and are likely to result in significant improvements in future rulemakings.

2.2 Tasks of retrospective review

Retrospective review is often discussed as single analytic process. But in reality, retrospective review consists of at least eight tasks, described below, and one could imagine different institutions having different roles in each task.

1. (i) Issue instructions to do retrospective analysis. This instruction could come from within the promulgating agency itself or it could come from an outside institution such as Congress, the White House, or the Courts. Historically this task has been handled by the President (through executive orders) or by Congress. However, this instruction has been fairly broad, with discretion left to the promulgating agency, perhaps with input from stakeholders or OIRA, to determine which rules will be selected for review.

2. (ii) Implement selection criteria for which rules to review. Examples of criteria may include: (i) rules with high opportunity to reduce cost, (ii) rules with high opportunity to increase net benefits, or (iii) rules with high opportunity to learn (variation across rules, RIA methodologies). Ideally, these selection criteria are directly related to the goals of the retrospective review. If the goal is to determine if rules remain relevant, the rules that should be selected are those at risk of irrelevance. If the goal is to reduce costs (or increase net benefits), then rules should be selected that have high potential for cost reduction (or increasing net benefits). If the goal is to learn, then samples of multiple rules and RIAs should be selected that offer variation across observations in order to gain insights into policy designs, forecasting methodologies, or other features.

3. (iii) Set the scope of analysis. In theory, every review could examine the full set of criteria outlined in the prior section – relevance, costs, benefits, cost-effectiveness, ancillary impacts, social well-being, and distributional equity. In practice, some of these criteria may not be relevant for particular rules and, hence, analysis of those criteria is not a worthwhile expenditure of effort and resources. More generally, a key task in the review process is to balance the decision costs of doing retrospective analysis against the value of information for policy improvement, and determine which aspects should be examined (Wiener, Reference Wiener1998). This calls for a scoping analysis, looking at all the criteria, with a process for selecting criteria and evidence for more detailed analysis based on the results of this scoping process. A further scoping task concerns the determination of the counterfactual – what would have happened in the absence of the rule(s) (Cropper et al., Reference Cropper, Fraas and Morgenstern2017). This may also include decisions about what the relevant set of policy alternatives could have been.

4. (iv) Acquire data for analysis. Ideally, the need for these data would have been anticipated during the initial rulemaking and the data would already be collected. However, this is not always the case, and data acquisition and cleaning often consume large amounts of time and resources during a review.

5. (v) Conduct the analysis. Someone must use the data to analyze the chosen rules given the scope selected.

6. (vi) Outside review of the analysis. The analysis should be reviewed by a body other than the one conducting the review. In current practice, this role is frequently fulfilled by OIRA.

7. (vii) Make recommended policy changes. Based on the review, there may be changes deemed needed to revise the past regulation. Actually revising the rule is a task that probably has to be done by the regulatory agency with the statutory authority (hence the agency that promulgated the initial rule), or by Congress.

8. (viii) Publish and archive all aspects of review. There needs to be publication and archiving of the retrospective reviews for the purposes of transparency and learning. Prior analysis of retrospective reviews in the USA found that of the retrospective reviews that have been conducted, much of that analysis is not published, recorded or archived in ways that the public and scholars can find (Wiener & Ribeiro, Reference Wiener, Ribeiro, Bignami and Zaring2016b).

The next section explains our methodology for examining institutions for retrospective review in light of the goals and tasks framework we just established.

3.1 Methods and data

We conducted text analysis of all Presidential Executive Orders (EOs) that pertain to regulatory review since President Gerald Ford (more detail on these presidential efforts can be found in Wiener & Ribeiro, Reference Wiener and Ribeiro2016a), as well as major relevant legislation enacted by Congress. While some U.S. States and other countries, notably the EU, have also undertaken efforts at retrospective review (Golberg, Reference Golberg2018; Radaelli, Reference Radaelli2020), we focus here on efforts at the U.S. federal level. The list of prior efforts that were analyzed with a brief summary can be found in Table 1.

Table 1 Summary of prior efforts at retrospective review.

We examined each EO and statute along two dimensions. First, we determined which of the three goals – rule relevance, rule improvement, and regulatory learning – were addressed by the EO/Statute. Note that a single EO/statute may target more than one of these goals. Second, we identified what institutional mechanism was required for the review and grouped those mechanisms into two broad categories – within the same agency that promulgated the review, or across multiple agencies.

3.2 Findings

Table 2 summarizes the past executive branch efforts at retrospective review. Each executive action was categorized based up on which goal(s) were targeted and within goals which specific criteria were required to be analyzed. Each action was further categorized based on what institution was required to conduct the retrospective review. “Within Agency” means that the agency that had promulgated the initial rule was required to conduct the review. “Across Agencies” means the executive order assigned or established an institution that spans regulatory agencies that was tasked with (at least part) of the regulatory review. Detailed text-based justification of our categorization of each EO/Statute is available in the Supplementary Material.

Table 2 Analysis of past federal efforts at regulatory review.

Table 2 highlights several common features of prior executive action on retrospective review:

1. (i) Actions have largely focused on the regulatory relevance and rule improvement goals. No executive order has specifically required that reviews address the regulatory learning goal. (EO 12291, section 6(a)(5), addressed “duplicative, overlapping and conflicting rules.” EO 12866, section 5(c), authorized the Vice President to call for review of “groups of regulations of more than one agency.” EO 13563, section 3, called for “coordination across agencies” to address “redundant, inconsistent, or overlapping” rules. But none of these called for retrospective review of multiple rules or RIAs in order to learn to improve policy design or accuracy in forecasting methods.)

2. (ii) Within the rule improvement goal, past efforts have mainly focused on the criterion of reducing the costs of each regulation, with less attention to the other criteria noted above, such as benefits, ancillary impacts, social well-being, and distribution.

3. (iii) Past efforts have focused on the agency that issued the rule, asking that agency to undertake the retrospective analysis, rather than exploring other institutional options.

So far, efforts at retrospective review have been mostly through episodic lookbacks, or occasional agency efforts to update rules (Wagner et al., Reference Wagner, West, McGarity and Peters2017), with few instances of advance planning in initial rules themselves to collect data over time and then conduct a planned retrospective analysis at a future date.Footnote ²

4.1 Alternative institutional options

While prior efforts have largely placed responsibility for retrospective review with the agency that issued the initial regulation, there are actually numerous alternative institutional options for retrospective analyses. Some of these options currently exist and have been previously used, albeit sparingly, while others would be new institutions requiring executive (or legislative) action to create.

To overcome the obstacles to retrospective review within an agency, it is often proposed that retrospective regulatory analyses be conducted or at least overseen by a central administrative body. Examples given for this institutional structure include the Office of Information and Regulatory Affairs (OIRA) in the USA, and the Regulatory Scrutiny Board (RSB) in the EU. Indeed, in the USA, OIRA does have oversight responsibility for retrospective reviews.Footnote ⁴ This responsibility could be extended to include development of detailed methodological guidelines for agency review (Aldy, Reference Aldy2014). An advantage of this hybrid institutional approach is that the expertise in review methodology is likely to be stronger at OIRA, but the topical expertise and data on each rule are likely to be stronger at the promulgating agency. So providing detailed methodological guidance centrally, while continuing to rely on agencies to conduct the review, may overcome one of the downsides to the agency-only approach. However, this approach still relies on the promulgating agency to conduct the analysis, and does not overcome the restriction of that approach with respect to comparative analysis or analysis of cumulative impacts. A further extension of responsibility could have OIRA conduct the analyses itself, at least in some cases such as where its ability to compare rules across agencies would be helpful. But OIRA currently does not have the budget and staff to take on this task (indeed OIRA’s staff has declined in size over the past several decades), and would require executive and legislative action to increase its staffing and analytic capabilities.

There are several additional institutional options that could try to overcome the major obstacles to the promulgating agency conducting retrospective analyses – such as a lack of resources or expertise in evaluation methods, an inability to compare multiple rules or across agencies, and inhibitions to criticizing the agency’s own activities.

1. (i) Other expert government agencies (outside of OIRA). There are other public agencies that already conduct regulatory reviews, including the Government Accountability Office (GAO). While this institution currently conducts some retrospective reviews, the GAO does not have the broader authority to select regulations to be reviewed and conduct those reviews. In theory, GAO could be given such authority although the enabling mechanism for such authority would likely require Congressional action. While an advantage of these potential institutions is that they can convene necessary expertise and they can examine rules across multiple agencies, a downside is that if regulatory changes are recommended, these institutions cannot actually adopt those changes. If the promulgating agencies or Congress do not agree, these recommendations may not be implemented.

2. (ii) Outside experts from academia, nonprofits, or think tanks. Instead of relying on existing government agencies, retrospective review could be conducted by private entities, presumably under contract with the government. For example, the National Academies of Science (NAS) is a private nonprofit organization that is frequently asked to conduct reviews of government regulatory processes by convening panels of experts. Academics and think tanks frequently engage in regulatory analysis, occasionally under contract and sometimes for other reasons (e.g., intellectual interest). An advantage of this institutional framework is the ability to draw on expertise outside of government. This institutional framework might be particularly well-suited to more difficult types of retrospective review including those that focus on cumulative impacts and interactive effects. However, this institution would encounter the same potential limitation that any recommendations would have to be enacted by the promulgating agency or legislature.

3. (iii) One-time commission of inquiry. This is a slight variant on the outside expert institutional framework in which, rather than relying on nonprofit contracts, the government establishes the authority internally to call an expert panel to conduct regulatory review, as needed. The commission of inquiry is widely used to investigate disasters and can be commissioned either by Congress or by the President (Balleisen et al., Reference Balleisen, Bennear, Cheang, Free, Hayes, Pechar, Preston, Balleisen, Bennear, Krawiec and Wiener2017). A similar framework could be used to establish a panel of experts for particular regulatory reviews.

4. (iv) New standing commission or review board. Another concept borrowed from the institutions for disaster review is the standing review board, modeled after the National Transportation Safety Board (NTSB). The advantage of a permanent review board (as opposed to one-time commission of inquiry) is the ability of the board to develop deep expertise in review methods that span policy domains (Balleisen et al., Reference Balleisen, Bennear, Cheang, Free, Hayes, Pechar, Preston, Balleisen, Bennear, Krawiec and Wiener2017). This option shares the concern about agency buy-in with the three previous options. Going further, some observers have proposed creation of a new standing commission (a “regulatory improvement commission,” RIC) to review the stock of existing regulations and their cumulative impacts, which could be advisory to the promulgating agencies or to Congress, or which could potentially be imbued by Congress with the authority to adopt its own revisions to agencies’ past rules, or to propose groups of legislative changes to past rules that Congress would then vote up or down as a slate (Mandel & Carew, Reference Mandel and Carew2013).Footnote ⁵ A standing commission would be particularly well-suited to identify issues that come up repeatedly in multiple rules across multiple agencies.

5. (v) Interagency working group. An alternative to the completely independent review institutions discussed above would be to establish an interagency working group tasked with conducting regulatory review. The working group could have representatives from each of the major regulatory agencies as well as representatives from government organizations focused on review methodology (e.g., OIRA or RSB). This working group would preserve the benefits of having a specialized group focused on evaluations that is at least partially removed from the promulgating agency (and hence can be more critical), but because each agency has representation on the working group there is the possibility that the working group will have more data, expertise and buy-in from the issuing agencies and this may lead to higher uptake of the working group’s recommendations. One concern about this approach might be that members of the commission engage in logrolling – exchanging positive reviews of one another’s rules.

In discussing the above institutions, we have focused on the analyses themselves. Whichever institution is selected, there needs to be a well-defined role for soliciting stakeholder and public input into the review process. This may include the ability to nominate rules for review either through a formal comment process or by way of citizen suits (Bull, Reference Bull2015, p. 96), supply relevant data and expertise for review, and comment on reviews and recommendations. Congress, as representatives of these various stakeholder and public groups, may also play a role by requesting reviews or mandating them, potentially through the use of so-called “sunset provisions” whereby statuatory authority for a rule expires if the rule is not reviewed in a particular amount of time (Ranchordás, Reference Ranchordás2015).

4.2 Mapping objectives and tasks to institutional options

The objectives of retrospective analysis may influence the choice of institutional approach. Government retrospective review has often aimed at individual regulatory policies, with a view to revising those specific policies, often to reduce their costs (Aldy, Reference Aldy2014). Broader retrospective analysis would assess the full scope of important impacts of each regulation (not only costs, but also benefits and ancillary impacts, with a view not only to reducing costs, but to increasing net benefits). But this may take too much time for any one agency’s staff and/or require expertise from multiple agencies. Similarly, a learning focus may require input from multiple agencies as well as deep expertise in analytic methods that may be better found outside the original rule-promulgating agency. In both of these cases, the reliance on the promulgating agency to conduct the rule is likely to limit the ability of the analysis to meet the objectives.

Further, the different tasks for retrospective review need not all be done by the same institution. Some tasks may be best performed by the promulgating agency, at least for rules with certain objectives, while others may be better performed by an alternative institution. Just as one example, the USA has focused on asking the agency that issued the rule to select rules for review and to conduct the retrospective analysis. That agency may have the most data and expertise, and the authority to revise the rule. But it may also face high opportunity costs in staff time diverted from other priorities. Whereas agencies may be motivated to submit ex ante RIAs in order to have their new rules pass OMB/OIRA review and be promulgated, there may not be as strong a motivation for agencies to conduct ex post RIAs when the retrospective review does not have practical rewards or only threatens to change the agency’s past work. And the issuing agency may face inhibitions from publishing a candid retrospective analysis that criticizes its own past rule or analysis (Wiener & Ribeiro, Reference Wiener, Ribeiro, Bignami and Zaring2016b).

Table 3 shows the range of tasks and objectives in the rows and the set of institutional options in the columns, and offers some examples of the mapping from objectives and tasks to institutional options. Institutional relationships between objectives and tasks that currently exist are shown in bold, while those that we are suggesting may be better in the future are in italics.

Table 3 Mapping objectives and tasks to institutions.

Under past EOs, OMB/OIRA has asked each agency to select the rules to be reviewed. The main selection criterion seems to have been cost – the opportunity to reduce the costs of each rule. A suggested complement is to invite stakeholders to nominate rules to be selected for review, such as through public comments in response to an OMB/OIRA or agency call for nominations (as occurred in several administrations), or through a petition process to each agency (Bull, Reference Bull2015), or through a public input process to a commission (Mandel & Carew, Reference Mandel and Carew2013, pp. 14–19). That may draw on the practical experience of stakeholders, but may also tend to focus on the parochial interests of those stakeholders. The agency could still have the final choice of whether to select the nominated rules. Another approach would be to have a broader selection process examine many candidates and select which rules deserve review – such as by an interagency working group, an expert board, or a commission established for this purpose (Mandel & Carew, Reference Mandel and Carew2013). This broader selection exercise could be better able to identify the rules most in need of review, especially if the agency faces inhibitions, or if the criteria include broader impacts (beyond cost and target benefits, to include ancillary impacts).

The analysis itself could be undertaken by the agency and historically this is what happens in most cases. Presumably it has the best data and expertise on the topic and understands the complex operation of the rule. But it may be that agencies have not been collecting data on their rules – hence the interest in requiring a plan for such data collection from the time the rule is proposed and adopted (Miller, Reference Miller2015; Cropper et al., Reference Cropper, Fraas and Morgenstern2017; Dudley & Katzen, Reference Dudley and Katzen2019).Footnote ⁶ Further, the agency may face opportunity costs and inhibitions. Thus it may also be useful in some contexts to have the retrospective analysis undertaken by another institution, such as an interagency working group, an expert board, or a commission (Dudley & Mannix, Reference Dudley and Mannix2018, pp. 16–17). Researchers at universities and think tanks often are the ones who undertake such retrospective analyses, either on their own or as contractors to agencies.

Whether the agency or another institution undertakes the analysis, it is most likely only the agency that has the legal authority to promulgate revisions to the rule, through the steps of proposed rule, notice and comment, and final rule, as provided under the Administrative Procedure Act. OMB/OIRA would ordinarily exercise oversight of such revisions.

A commission could be established to assess multiple rules, including the combined and interacting effects of accumulated multiple rules (Mandel & Carew, Reference Mandel and Carew2013). It could be created as a one-time exercise (perhaps lasting several years) to take stock of the accumulated body of regulation and recommend revisions. This could reflect the experience of the Defense Base Closure Commission, which identified military bases for closing or repurposing, somewhat insulated from the politics of Congress (Mandel & Carew, Reference Mandel and Carew2013). And it would be analogous to the one-time commissions of inquiry created after major disasters such as the 9/11 terrorist attacks in 2001 and the BP Deepwater Horizon oil spill in 2010.

But rather than a one-time exercise, there are also advantages to establishing such a commission as a standing body, similar to the NTSB, which would have a greater depth of experience and expert staff to inform its ongoing analyses (Balleisen et al., Reference Balleisen, Bennear, Cheang, Free, Hayes, Pechar, Preston, Balleisen, Bennear, Krawiec and Wiener2017). Establishing a standing board allows for the collection of expertise in evaluation methods that can be applied to multiple rules (or collections of rules) over time. It also separates the review activity from the direct regulatory responsibility. This level of independence enables more candid reviews of potential errors or missteps in the regulatory process. One concern about the use of a nonregulatory body to conduct reviews is that the independent reviews may lead to recommendations that then the implementing agency must take seriously and promulgate, and the implementing agency may not have much incentive to do so. While this is a possibility, analysis of the recommendations made by the NTSB (which has no authority to implement recommendations made to regulatory agencies and Congress) suggests that the vast majority of these recommendations are eventually adopted (Balleisen et al., Reference Balleisen, Bennear, Cheang, Free, Hayes, Pechar, Preston, Balleisen, Bennear, Krawiec and Wiener2017, p. 507; National Transportation Safety Board, 2017). The respect for the quality of the analysis and the objectivity of the NTSB has proven to be effective at persuading the regulatory agencies and Congress to take recommendations seriously.

The objectives and tasks of learning from multiple rules – to test the actual performance of policy designs (in light of predictions), and to test and improve the accuracy of ex ante forecasting methods – seems to go beyond what one agency could undertake, unless the agency were analyzing multiple rules within its own portfolio (as it might to test performance of differing policy designs in one sector). The broader learning objectives, especially to test and improve on variation in ex ante forecasting methods, might be best served by an interagency working group, or an expert board, or a commission, with the breadth and staff expertise to compare across multiple rules.

The influence of retrospective reviews on regulatory rules could be advisory, or could have more legal authority. The outputs of analyses of individual rules, or of broader multirule and multimethods analyses, could be presented to the relevant agencies as recommendations for agency action, and to OMB/OIRA as recommendations for new guidance on methods of ex ante and ex post impact assessment. Or they could be presented to Congress as recommendations for legislative enactment. Greater authority could be conferred if Congress were to enact legislation creating a commission and delegating to that commission some authority to adopt rules changes itself, but such an approach could sacrifice the value of agency expertise and could potentially be in conflict with principles of administrative law. One proposal is to have Congress create a regulatory review commission which would then propose a set of numerous changes to Congress which Congress would vote up or down as a package without amendments (Mandel & Carew, Reference Mandel and Carew2013, p. 14).

In the past, retrospective review has often been seen as a one-time follow-up evaluation – a second look back (after ex ante RIA). Achieving more regular and effective application of retrospective analysis would be an important advance. Going further, some rules or agency programs may warrant not just one look back, but ongoing multiple periodic reviews (e.g., every 2, 5, or 10 years), toward a continuous process of adaptive updating (McCray et al., Reference McCray, Oye and Petersen2010; Ribeiro, Reference Ribeiro2018; Bennear & Wiener, Reference Bennear and Wiener2019b). Some current laws call for such periodic reviews, such as the reviews every 5 years of the National Ambient Air Quality Standards (NAAQS) under the Clean Air Act, and of policies under the Lautenberg Chemical Safety for the 21st Century Act.