Health Experiment Obtaining a Baseline Reading

Evaluation Plan for a Cardiovascular Disease Prevention Program

Introduction

The escalating burden of cardiovascular disease (CVD) among people of color in Oklahoma City, Oklahoma, presents a critical public health challenge. In response to this, a targeted intervention program has been developed with the aim of reducing CVD risk factors in this vulnerable population. This paper outlines the Evaluation Plan designed to assess the effectiveness of this program, beginning with a rationale for selecting a specific impact evaluation question that focuses on comparing the health outcomes of the program participants with a similar population not receiving the intervention. This is followed by a description of the evaluation design, specifically the Non-equivalent Groups Design (NEGD). It then discusses the various threats to internal validity, and outlines strategies to mitigate these threats. Data collection methods and measurement tools used in the evaluation are also described. Finally, the paper discusses how the results of the evaluation will be used.

Evaluation Question

What is the effectiveness of a cardiovascular disease prevention program among people of color in Oklahoma City, compared to a similar population in a neighboring city without the program?

Rationale for Choosing This Question

The selected impact evaluation question is important to the objectives of the health promotion program aimed at reducing cardiovascular disease among people of color in Oklahoma City. This question targets the heart of the program's intended effect — evaluating whether the intervention directly contributes to lower risk factors associated with cardiovascular diseases, such as improved dietary habits, increased physical activity, and better management of blood pressure and cholesterol levels. Through a focus on these outcomes, the evaluation aims to quantify the program's direct benefits. The rationale for choosing this question lies in its potential to provide clear, evidence-based results that can demonstrate the program's success or highlight areas needing improvement. This outcome-oriented approach can help with securing continued funding and support, influencing policy decisions, and justifying the expansion of the program to other communities with similar demographics and health challenges.

Evaluation Design

The Non-equivalent Groups Design (NEGD) is suited for this evaluation due to practical constraints that make random assignment unfeasible. In the NEGD, two groups are compared — one that receives the intervention (experimental group) and another that does not (control group). These groups are selected based on shared characteristics that are as similar as possible except for the intervention itself, reducing the likelihood of selection bias affecting the outcomes (McKenzie et al., 2022). The notation O1 X O2 | O3 O4 explained in the context of this design is as follows:

1. O1 (Pre-test observation of the experimental group): This involves collecting baseline data on the experimental group before the intervention begins. Metrics include initial health literacy levels, cardiovascular health stats, and behavioral factors related to cardiovascular risk.

2. X (Intervention): This represents the implementation of the cardiovascular disease prevention program, which include educational workshops, community health fairs, and direct healthcare services.

3. O2 (Post-test observation of the experimental group): Following the intervention, this observation assesses changes from the baseline to determine the intervention’s impact on the experimental group.

4. O3 and O4 (Pre-test and post-test observations of the control group): Similar data are collected from the control group, which does not receive the intervention, at parallel times. This group serves as a baseline to understand changes that occur without the intervention, providing a counterfactual against which the program’s effects can be measured.

Significance of the Evaluation Design

The strength of the NEGD lies in its ability to suggest causality more strongly than non-experimental designs, even though it cannot fully match the causal inference powers of randomized controlled trials (Mildner, 2013). In using pre- and post-tests in both the experimental and control groups, this design helps control for external variables that could influence the outcomes, thus enhancing the internal validity of the findings. However, this design is sensitive to how well-matched the experimental and control groups are, and any significant differences between them could introduce bias, affecting the validity of the conclusions. Therefore, careful selection and characterization of these groups are imperative to ensure that the results are as reliable and generalizable as possible within the given constraints.

Threats to Internal Validity

Selection bias presents a significant threat to the internal validity of this study. This arises when there are inherent differences between the characteristics of the intervention group in Oklahoma City and the control group in a neighboring city, which might influence the study's outcomes. Differences could include demographic disparities, baseline health statuses, or access to healthcare services, potentially leading to incorrect assumptions about the intervention's effectiveness. Additionally, maturation is another internal threat where natural changes occur in participants over time, independent of the intervention. Changes in health behavior or cardiovascular health might result from aging or other personal developments rather than the program itself, which can skew results. Finally, history effects, such as new healthcare policies, economic downturns, or public health crises occurring during the study period in either location, can independently affect participants' health behaviors and outcomes.

Mitigating Threats to Validity

To address these threats and improve the study’s internal validity, several strategic approaches can be adopted. Matching helps make sure both the intervention and control groups are comparable in terms of age, gender, socioeconomic status, and baseline health conditions (McKenzie et al., 2022). This also helps to minimize selection bias by making sure the only significant difference between the groups is the intervention itself. Repeated measurements are critical for monitoring changes attributable to maturation. Through a comparison of data from multiple time points, one can see whether changes in outcomes are due to the intervention or natural developments over time. Additionally, a contextual analysis will be conducted to consider external events that could impact the study's results and thus account for history effects.

Data Collection Methods

The data collection methods will obtain both quantitative and qualitative data. Surveys will be used to collect baseline and follow-up data on health literacy and behaviors. These will assess knowledge acquisition, attitude changes, and behavioral adjustments regarding cardiovascular health management. Health screenings will measure physiological markers such as blood pressure, cholesterol levels, and body mass index at regular intervals, providing objective data on the physical health changes of the participants. Lastly, post-intervention focus groups will gather qualitative feedback from participants, offering insights into their satisfaction with the program, perceptions of its impact, and personal narratives that may not be captured through quantitative methods. These discussions are invaluable for understanding the nuanced effects of the program and ensuring a holistic evaluation of its outcomes.

Measurement Tools and Reliability

To evaluate the impact of the cardiovascular disease prevention program, the project will use a combination of standardized measurement tools, each selected for its reliability and relevance to our study objectives.

Health Literacy and Behavior Survey

For assessing health literacy and behavior changes, the Newest Vital Sign (NVS) tool will be used; it is a widely recognized and standardized tool designed to measure health literacy. The NVS involves a quick assessment where participants review a nutritional label and answer questions that gauge their ability to interpret and use the information correctly. This tool is particularly suitable for our program as it directly relates to the participants' ability to manage their health, especially in making informed dietary choices which are crucial for cardiovascular health (Grabeel et al., 2023).

Biometric Measurements

To measure the physical health outcomes of participants, clinically validated tools to assess biometric data will be used. These include devices for measuring blood pressure and cholesterol levels, among other relevant health indicators. These measurements will provide concrete data on the physiological impacts of the program, such as improvements in blood pressure or cholesterol levels that are critical markers for cardiovascular health. The tools chosen for these measurements are approved for clinical use, so that they meet high standards of accuracy and reliability necessary for the evaluation.

Focus Group Protocols

To complement the quantitative data with qualitative insights, focus groups will be used with structured discussion guides. The structured nature of these guides helps will help to make it so that discussions are focused and relevant, while also allowing flexibility for participants to express their views and experiences freely. This method will help us gather in-depth insights into how the program affects participants' attitudes and behaviors towards managing their cardiovascular health.

Use of Evaluation Results

Refining Program Components

The data collected and analyzed from both the quantitative and qualitative measurement tools will inform the ongoing refinement of program components. It will thus be possible to identify which aspects are most beneficial and which may require modification. For example, if data indicate that certain educational components of the program are not effectively improving health literacy, these can be redesigned to be more engaging or informative. This iterative process of refinement is helpful in driving the overall quality of the program.