Disaster Post-Mortem Analysis: Multi-Agency Response Evaluation

Introduction and Purpose

The purpose of this memorandum is to debrief the disaster incident of December 4, 2014, in Metropolis, State, as reported to Lessons Learned Information Sharing (LLIS.gov), the national online network of the U.S. Department of Homeland Security and Federal Emergency Management Agency. LLIS is maintained to facilitate communication of lessons learned, best practices, and innovative ideas for personnel and communities engaged in homeland security and emergency response (LLIS, 2008). While LLIS does not represent the official positions of the U.S. Department of Homeland Security, it serves as an important forum for clarifying and focusing learning within federal agencies.

Situational Analysis Framework

Post-mortem debriefing sessions are expected of incident commanders and those who conduct response field training as part of organizational missions. Ideally, debriefing should occur following the end of any prolonged incident response and within reasonably close temporal proximity to that closure. These sessions are important because they provide responders an opportunity to offer early feedback on response operations. Through this feedback loop, trainers gain critical insight for designing and improving training courses that truly serve responders.

This memorandum is organized according to the steps and objectives of a situational analysis, consideration of key success factors related to managing disasters of this magnitude and scope, identification of criteria for measuring response efficacy, and discussion of potential assessments to gauge key response success factors.

The Five Success Factors in Disaster Response

The situational analysis process covers three phases of disaster response: initial, interim, and final assessments. Data collected and analyzed must address the disaster type, imminent hazards, major problems, and available resources. Assessment must extend to life safety, available lifelines, access routes, essential facilities, and coordinated implementation of comprehensive response. Life safety is always the first priority, as emphasized by New York State Division of Homeland Security and Emergency Services: "First and always, assessment must focus on immediate emergency needs for life, safety, protection of property and essential services" (Miske, 2006; NYS-DHSES, 2013).

The objectives of situational assessment for debriefing purposes include providing comprehensive and timely reports on disaster scope and impact, accurately informing the public, assessing effectiveness and appropriateness of decision-making at all relevant levels, and adjusting provision for recovery assistance and resources (NYS-DHSES, 2013). Information systemization must address assessment and collection, analysis and collation, and dissemination—all of which must articulate to facilitate consistent, efficient communication and collaboration.

The fundamental threads running through disaster response are competency, compliance, comprehensiveness, communication, coordination, and culture. These attributes are integrated throughout disaster response management, yet each must be robustly addressed as a distinct consideration, as the six factors play pivotal roles in each phase of disaster response. These factors serve as pillars for managing large, multi-agency and multi-jurisdictional response programs.

While competency is clearly a critical factor in effective disaster response management implementation, it is important to consider how competency is developed and sustained. Detailed discussion of this factor follows in the section on designing disaster response evaluation tools and processes.

Compliance is an indicator of fidelity to existing, time-tested, proven protocols essential to effective inter-agency functioning. For example, the overarching National Incident Management System (NIMS), the National Response Framework (NRF), and the Incident Command System (ICS) provide guidelines for conducting disaster responses that serve as comprehensive communication and coordination frameworks. The ICS definition from the United States Center for Excellence in Disaster Management and Humanitarian Assistance illustrates this point: ICS is "a set of personnel, policies, procedures, facilities, and equipment, integrated into a common organizational structure designed to improve emergency response operations of all types and complexities" (ICS, 2004). A key reason for complying with the ICS framework is that it is a scalable, adaptable mechanism that references a familiar hierarchy, enabling people from diverse areas and disciplines to conduct effective responses to disaster events and foster collaborative functioning within homeland security initiatives (ICS, 2004).

Multi-agency planning is the foundation upon which effective, comprehensive implementation must proceed. A primary function of the Incident Command System is to provide an optimized structure for first-on-scene responders that promotes rapid, accurate, and comprehensive response (ICS, 2004). A key success factor in disaster response is the level at which people from multiple agencies—unaccustomed to working together on a routine basis—achieve seamless communication and operational plans (Moynihan, 2009).

An integrated communication plan is a crucial component of effective disaster response management. A sure path to failed incident management is a communication system inadequate to the response context, resulting in communication breakdown. The challenge is ensuring that voice and data communication systems are effectively integrated at levels that enable communication across personnel, agencies, and jurisdictions.

Under non-crisis conditions, coordination evolves gradually as participating organizations develop incremental mutual adaptations (Moynihan, 2008). Working relationships that emerge over time form a basis for mutual regard, trust, and reliability (Moynihan, 2008). However, under crisis conditions of disaster response, first responders must rapidly come together as a unified body to perform extraordinarily difficult tasks for which they may have limited experience (Moynihan, 2008). Pre-established roles and responsibilities mapped by federal protocols serve to expedite coordination of personnel and resources.

Moynihan (2008) cited the early sluggish response to Hurricane Katrina as evidence that coordination is more effective when agencies assume a "push" approach to disasters rather than a "pull" approach. The most apparent indicator of this shift was the change to vocal command by the Department of Defense (DOD) rather than detailed, exacting request processes. A push approach enables leaders to make use of deep organizational knowledge, to understand how organizational standard operating procedures will limit or further responsiveness, and to adjust their original assumptions about a disaster event, thereby adapting their response to better fit circumstances.

Designing and Evaluating Disaster Response Programs

Khademian (2002) argued that agency leaders cannot be expected to easily modify or establish organizational culture. However, if more than one cultural mode exists within an organization, recognition by leaders that switching cultural modes can bring about more effective response can substantially improve collaborative efforts (Weick, 2001). An example of demonstrably effective cultural switch occurred during the collaborative response to Hurricane Katrina: the DOD abandoned strict adherence to Joint Directorate of Military Support (JDOMS) procedures—designed to prevent the DOD from committing to unsuitable missions or engaging in unnecessary interagency action—and pursued a more aggressive course of action characterized by a military "can do" spirit and willingness to work around rules to achieve a mission (Moynihan, 2008, p. 6).

As Moynihan (2008) wrote: "One should not underestimate the importance or difficulty of culture switching. It requires an ability to recognize what cultural attributes exist within an organization, and when each cultural attribute is appropriate" (p. 6).

The main capability to which evaluation of disaster response must be directed is preparedness, since "the basic design of a crisis management system—mitigation, preparedness, response, and recovery—assumes a consistent, integrated approach across those functions" (Moynihan, 2008, p. 10). Evaluation of disaster preparedness must take place on many levels, assuming a constellation of significant variables will coalesce for individuals, municipalities, jurisdictions, and agencies as needed. Significant variables include provision of resources, joint planning, training, and assessments and audits. While drills, simulations, and competency tests are important, they must be undergirded by a broader perspective that considers essential factors that may be less apparent than observed performance.

As an example, after 9/11, FEMA was integrated into the new Department of Homeland Security (DHS), and in the process, lost direct access to the White House and was positioned so as to not consolidate response plans into one unified and coordinated framework (Moynihan, 2008). Moreover, reorganization of federal agencies erased the line between FEMA and state preparedness functions, with preparedness responsibility assumed by the Office of Domestic Preparedness. Notably, grants were awarded primarily to preparedness directed at terrorist events over natural disasters (Moynihan, 2008). The relationship between disaster response quality and effectiveness, and changes to policies, institutional structures, and lines of authority, is salient in this example.

Drills and tests can be designed to specifically evaluate the degree to which key success factors are present in disaster response and evident in disaster response management. Yet these evaluations must assume a broader approach by sweeping in institutional, economic, and cultural variables that have unavoidable influence on implementation. Trainings must address perceptual constraint, which can occur at individual or organizational levels. As Moynihan (2008) asserted, "response requires an array of interdependent competencies, and it is the need to rapidly integrate these competencies that give rise to and continues to provide the compelling logic" for specific interagency disaster response protocol (p. 4).

At the same time that drills and tests evaluate the effectiveness of disaster response implementation according to mutually regarded federal protocol authority, it is essential to train responders to recognize that their capacity to provide an expedient, organized response may also contribute to response constriction (Moynihan, 2009). Critical literature highlights the need for training content to include information preparing first responders to monitor their own mindsets and recognize how expeditious thinking can limit holistic perspective (Moynihan, 2009; Yechiam, et al., 2005). Training, drills, and tests must ensure participants understand that the more automatic a response is—the more it is based on past experiences or event recency—the higher the probability that critical information will be missed, whether through urgency or adopted schema (Yechiam, et al., 2005).

Conclusion

The objective of Presidential Policy Directive (PPD-8) is "to achieve a shared understanding and a common, integrated perspective across all mission areas—Prevention, Protection, Mitigation, Response, and Recovery—in order to achieve unity of effort and make the most effective use of the Nation's limited resources" (LLIS, 2014). The successful implementation of disaster response may be evaluated according to criteria of the National Disaster Recovery Framework (NDRF), which is appropriately focused on the post-event period. Assessment should address the level to which key attributes of NDRF are met. Successful disaster response must: (1) align with core recovery principles; (2) demonstrate the degree to which recovery personnel accomplish their responsibilities and fulfill their responder roles; (3) show how well communication and collaboration are conducted within a coordinating structure that includes all stakeholders; (4) reflect the degree to which lessons learned influence guidance provided to those engaged in pre-disaster and post-disaster recovery planning; and (5) demonstrate the utility of processes and resources communities may access to establish an "opportunity to rebuild stronger, smarter, and safer" (LLIS, 2014).