Health Care Introduction to Evaluation Research Evaluation

Health Care

Introduction to Evaluation Research

Evaluation Research is a vital aspect of organizational and program establishment and development. Used extensively across multiple industries, Evaluation Research encompasses dozens of methods, some of which overlap and/or encompass other methods. Though the methods vary, they ideally share the common characteristics of good basic research, systematic processes, data collection to increase knowledge, enhancement of decision making, and practical use of evaluation findings. In addition, despite the variations of methods, there are basic steps that can be followed in order to conduct an effective evaluation of an organization or program.

Evaluation Research "Snapshot"

Evaluation Research has been variously defined as a unique method of applied/action research, a social applied/action research method of evaluation, or a unique method of assessing programs (Powell, 2006). These definitions apparently depend on the focus of the definer. However, regardless of the definition, all Evaluation Research apparently has common elements: it is a systematic process; it entails data collection regarding resources, services, processes, programs and organizations; it is used to amass knowledge and enhance decision making; it is intended to have practical uses; it is supposed to use the careful and rigorous practices of good basic research (Powell, 2006). Unlike pure research, Evaluation research has a specific purpose involving the potential for some action and it involves an ongoing and dynamic dialogue among the organization, the program and users in order to continually monitor, diagnose and change or even end the program, as needed. The importance of Evaluation Research is underscored by the numerous resources provided by organizations such as the Center for Disease Control for evaluating youth-oriented programs addressing such topics as smoking, asthma, HIV, nutrition and physical activity (Center for Disease Control, 2011).

Given the broad common elements of Evaluation Research, its application to and benefits for many types of organizations are obvious. Generally, an organization uses Evaluation Research to: explain its purpose and methods; define its history; determine program goals; avoid and/or correct mistakes; give feedback to its staff; increase its efficiency; provide bases and support for its planning and decisions, including but not limited to whether the program will continue; examine new ideas; explain how it uses its resources; explain its impact; show concern for their clients and determine how well they are serving their clients; become more visible politically effective in the community; meet grant requirements (Powell, 2006). In sum, Evaluation Research ideally assists an organization in assessing and improving every aspect of a program, from start to finish.

Due to the large numbers and varied types of organizations that may use Evaluation Research, numerous methods of Evaluation Research have been developed. According to Powell, approximately 42 approaches/models are described by one expert, alone (Powell, 2006). Powell's article briefly describes some of the more common methods of Evaluation Research, including: input measurement; output measurement; impact assessment; benchmarking; standards; quantitative evaluation; qualitative evaluation; cost analysis; and organization effectiveness. The Input Measurement method of Evaluation Research is the most traditional approach, which is limited in its usefulness because it merely measures the resources of an organization (Powell, 2006). Output Measurement, also known as Performance Measurement, assesses the accomplishments of some activity, such as effectiveness in accomplishing its goals, user satisfaction, and use of facilities/equipment (Powell, 2006). Impact Assessment, also known as Outcomes Assessment, assesses the benefits of the organization's products/services on the clients who are served by the program (Powell, 2006), and is generally measured by Service Quality (the difference between the client's expectations and his/her perceptions of the actual performance), or Process Evaluation ("looking at how something happens rather than or in addition to examining outputs and outcomes" (Powell, 2006). The Benchmarking Method of Evaluation Research, one of the more recent approaches, assesses "total quality management" by analyzing an organization's points of comparison against externally established criteria set by gathering data from organizations in the same field (Powell, 2006). The Standards Method (Scriven, 1991, p. 5) assesses an organization according to guidelines developed by a group of experts in that organization's field (Powell, 2006). The Quantitative Evaluation Method (Scriven, 1991, p. 60) assesses an organization through measuring numeric variables and includes many of the methods already discussed and is considered relatively objective because it uses a number of reliable statistical techniques to measure performance (Powell, 2006); however, there are also drawbacks in using this method in that it is less valuable in evaluating new programs that might change before data can be collected/analyzed and much of the data is collected in a controlled setting and cannot be generalized (Powell, 2006). The Qualitative Method (Scriven, 1991, p. 87), which is increasingly popular, is a more holistic assessment that accounts for human behavior/experience and is best used in areas that cannot be easily quantified. While the Qualitative Method holds advantages such as a greater flexibility and awareness of time/history, it also has weaknesses such as requiring labor-intensive data collection and producing data that cannot easily be generalized (Powell, 2006). Tools are readily available for using qualitative and quantitative methods; for example, the Center for Disease Control provides tools for both qualitative and quantitative methods of evaluating youth-oriented programs (Center for Disease Control, 2011). The Cost Analysis Method itemizes operational costs to determine whether practices/programs are cost-effective in order to determine whether the practice/program is justified (Powell, 2006); (Scriven, 1991, p. 5). Finally, the Organizational Effectiveness Method of Evaluation Research is actually a goal of many other methods, using a number of criteria and "key questions" about such topics as goals, client satisfaction and employee quality to measure an organization's/program's performance (Powell, 2006). Research shows that several of these Evaluation Research methods overlap and that they are merely a handful of numerous evaluation methods (Powell, 2006).