Overview

In recent years, several high profile administration rules have been overturned by the courts (“net neutrality” from the Federal Communications Commission (FCC), “gainful employment” from the Department of Education) or withdrawn after a storm of public protest and comment (Centers for Medicare and Medicaid Services’ (CMS) Medicare Part D rule or the Internal Revenue Service rule for 501(c)(4) candidate-related activities). Why have agencies so regularly been unable to promulgate rules that stand the tests of judicial and public scrutiny? 

Regulations are only as good as the data on which they are based. Congress delegates its rulemaking and regulatory power to federal agencies in part so that subject matter experts can use the facts available to appropriately reflect congressional intent. It is, therefore, the obligation of federal agencies to ensure that they infuse their statutory authority for rulemaking and other regulatory activity with as great a command of the facts and issues as possible. Acquiring data and expertise and integrating it into the agencies’ processes should be a matter of design, not accident. These are critical issues in the development of an agency’s institutional culture. Of particular importance are:

• Processes to identify key research issues related to rulemaking;
• Systems to acquire data rich enough to support fact-based rules;
• Independent expert vetting processes to validate research methods and findings; and
• Investment in the expertise and professional standing of the workforce.

This analysis is devoted to outlining aspects to be considered in developing federal agency procedures to foster an effective rule-making culture.

Components of a Fact-Based Approach

The degree to which rulemaking effectively addresses policy issues in a balanced and fact-based manner hinges on identifying the following high-priority issue areas: building a skilled, credible, and research savvy workforce; acquiring needed information; building systems for internal review; and permitting effective external review. It is important to note at the outset, however, that the very effort of putting these values and systems into place will have the benefit of conveying to stakeholders the agency’s seriousness of rulemaking purpose.

Asking the Right Questions

What are the highest priority areas for rulemaking? What should be the objective of such rules? What research is developed as part of the rulemaking? Is the research based on sound methodology and data? Are the findings consistent with the data? These questions necessarily must balance the views of many stakeholders as well as maintain validity within the field that is subject to the rulemaking.

1. Strengthen Internal Review Process

Nothing gets done right the first time. Agencies need to develop processes for research review — in addition to the usual communications vetting and managerial approval. Research papers should be reviewed by members of the staff with different areas of expertise to ensure that the profession broadly accepts the quality of the work. Developing a process in which there is a second set of impartial research eyes on every product is essential. This also builds respect for constructive feedback within the agency.

2. Utilize Advisory Board Comprised of Outside Experts

An advisory board composed of stakeholders can provide the agency staff with a long-range “heads up” on emerging issues, thereby permitting agency staff the time to develop expertise and information on the issues. In addition, the advisory board would review the findings and validate underlying data and methodology that are used in developing agency research. Regular — typically twice annually — meetings provide the vehicle for developing an agency research agenda in a timely fashion. The advisory board also serves the purpose of ensuring that the agency remains focused on a fair and level playing field for all those affected by its rules.

Agency open forums are a vehicle for stakeholders to weigh in on the pressing issues at any moment. A regular (e.g. quarterly) schedule of such town halls is a complement to the advisory board in developing priorities and identifying gaps in expertise in the agency.

Outreach to industry will help staff fully consider cases where there may appear to be inefficient pricing, incomplete markets, externalities, and so forth. Industry experts may have fairly straightforward explanations of customs, history, peculiarities of the business, informal and formal self-enforcing rules, which provide a clearer picture than data sometimes can. That said, staff will need to use judgment to decide if these explanations are plausible and justified.

Nuances and particular aspects of specific policy disciplines can be easily misjudged when regulating.

Example: Researchers Examine Hurricane Charley and Building Codes

The particulars of building code standards, which are usually regulated at the local level, could be damaged if regulated from a federal level without the proper incorporation of existing codes, practices, and benefits of building standards. In a study conducted regarding regulations issued by the Federal Emergency Management Agency (FEMA) that established the National Flood Insurance Program (NFIP), researchers found that some homes in Lee County, Florida which were built under NFIP standards experienced 57 percent more total damages in the 2004 Hurricane Charley than those built using the previously instituted local building codes.

Building the Agency Workforce

An agency is only as good as its people. Obviously the key step is to recruit quality staff in research, management, and communications positions. But past that, it is important that the staff develop an ethic for fact-based and research-based decision making.

Not only is it vital to be surrounded by expert staff, but they must also be given a voice. Too often an agency may have skilled staff, but their input and findings are meant only as a rubber stamp for administration priorities.

For example, in interviews conducted with senior federal agency economists, Mercatus Institute researchers found that senior federal economists in a variety of agencies felt that their input on federal regulatory development had less impact than desired. Some of the economists interviewed also stated that the decision makers within their federal agency were not attuned to economic analysis as a main reason for decision making. These economists also felt that, instead of using the economic research to inform an agency’s policy decisions, some agencies made the decision and followed up with research.[ii]

Some practices that support these research goals are:

1. A Staff Working Paper Series

Staff-level research should be available to the public (and others in the agency) in the form of a web-published working paper series. Such publications should clearly not carry the imprimatur of the agency; they are strictly a vehicle for staff to display their command of research issues.

An effective working paper series serves several purposes. First, it is a recruiting tool, as it permits the agency to attract better talent; talent that cares to develop a personal track record of accomplishment during their time at the agency and beyond.

Second, it invites feedback by the larger research community. This deepens the overall pool of facts ultimately available inside and outside the agency when rulemaking occurs. It also assists the agency managers in identifying high-performers among the research staff. Finally, this research allows the staff to display its expertise and impartiality, enhancing the credibility of the agency as a whole during the policy process.

2. Attendance at Professional Research Meetings

Regular participation in professional research meetings by others in the research community is the norm outside of government. The professional feedback, give-and-take, and criticism improve the quality of research and develop an appreciation for other points of view, areas of uncertainty in the state of knowledge, and a culture of deciding on the basis of the evidence. Conference attendance by government employees is easy to demagogue — and sometimes abused. However for research-based agencies it should be supported.

3. Specific Goals and Scope

Outside of the working paper series and other pure research endeavors, research directly related to rulemaking must have explicit goals and scope. For example, though specific methodologies may change cost-benefit analyses, these analyses can tally up costs and benefits in such a way that nearly any outcome can be achieved. Moreover, it will necessarily require a fair number of assumptions, such that the cost-benefit result is primarily a result of assumptions, especially for complicated markets (see Cochrane 2014).[iii] One way around this is use of mixed methodologies (surveys, GE models with frictions, etc.).

Example: FTC Develops Sophisticated Review of Markets

Another is agency-wide guidance on the proper approach. For instance, post-1970s the FTC took a much more sophisticated view of market structure, and was more inclined to dismiss per se claims of a market failure in the absence of clear evidence. Starting from a model of perfect efficiency will inevitably invite intervention in every case. Internal guidance that directs a staff must carefully consider economies as they are, and not compared to a frictionless ideal, goes a long way toward generating balanced analyses.

Embedding a staff understanding of the goals for fact-based rulemaking serves as a useful counterbalance to the inevitable pressures from political appointees to pursue an administration-driven agenda that may not be supported by the data and research.

Example: EPA Pushes Regulations Benefiting Special Interests

Regulations released last year by the Environmental Protection Agency (EPA) exemplified agency pressures to bow to a specific interest. In this case, the EPA issued annual regulations regarding the increase in the use of biomass diesel – primarily created through soy bean products – by 28 percent over previous year requirements, touting the changes as a move toward energy security. The EPA estimated that the regulation would result in up to $425 million in increased costs, causing $52 million in environmental damages. However, the regulation did increase soybean oil revenues, by $1.2 billion. The agency pushed through with the regulation, announcing the changes in the heart of soy bean country, among those most receptive to the changes.[iv]

Acquiring Needed Information

Quality information is the lifeblood of the policy process. The agency should stress its willingness to meet with stakeholders and receive briefings on policy issues. Since there is a fine line between providing information and “lobbying,” the agency should cultivate a culture of informational outreach early in the process before specific decisions have been reached. Although each research endeavor and rulemaking will be different, the process must be explicit such that the research and outreach findings occur before important decisions, and further need to be an actual input in the decision making process (not merely a “checking off of the box”). Administrators may be tempted to allow research to follow a parallel track, only using research findings at the end to bolster predetermined agency decisions.

Example:  CMS Proposes Medicare Part D Rule Contradicting its Own Research Findings

In one recent case where regulators have used this parallel approach, CMS was forced to withdraw most of a Medicare Part D proposed rule that made initial claims to reduce costs and improve plan benefits. The agency based the need for key portions of the rule on internal CMS studies and complaints that the agency claimed to have received from seniors.

Once CMS’ research became public, however, independent actuaries raised questions about the methodology and patient groups complained the rule would disrupt and eliminate millions of Part D policies. Even the FTC took the unusual step of filing a Comment Letter stating that the rule would reduce competition and significantly raise costs for consumers.

Congressional oversight hearings were convened to investigate with Members from both parties expressing deep skepticism about the rule and CMS’ reasons for issuing it. It soon became clear the conclusions of these key studies misrepresented the findings so to appear as though mail service pharmacy and preferred pharmacy networks inflated Part D costs. The conclusions, however, focused only on a few costly outliers, ignoring the overwhelming evidence showing that both mail service and preferred networks actually saved money in the vast majority of cases.

CMS should be commended for withdrawing most of the proposed rule and promising to revisit some of the offending studies. Nonetheless, the entire episode could have been avoided with better research practices.

Conclusion

In recent years, several high profile administration rules have been overturned by the courts or withdrawn after a storm of public protest and comment. This suggests that federal agencies can improve their procedures and goals to ensure a strict adherence to fact-based approach as they issue and implement regulations. Agencies need institutional structures that ensure an educated workforce are given a voice, and continue to improve as experts in their field. They must also gather necessary data and information from appropriate stakeholders and have strong review procedures, both internally and externally. Federal rulemakings have widespread consequences. Without appropriate institutional policies and cultures, they run the risk of being systematically flawed.