New Brunswick Extra-Mural Program Evaluation Proposal

Introduction and Program Overview

The New Brunswick Extra-Mural Program (EMP) operates under the aegis and supervision of Regional Health Authorities (RHAs) to bring health services directly to people's homes. It is open to residents regardless of age, as long as they meet the eligibility criteria, and focuses on the client and family. The EMP works by bringing together all parties — including health care providers, physicians, clients, and patients' families — in a coordinated manner. The program fulfills its mandate through the provision of services that include acute, palliative, chronic, rehabilitative, and supportive care. All EMP clients can access medical, physiotherapy, occupational therapy, respiratory therapy, clinical dietetics, social work, pharmacy, speech-language pathology, and nursing care services, all available on a 24/7 basis (New Nouveau Brunswick, n.d.).

This evaluation proposal's goal is to determine program effectiveness. The evaluation will examine program components that function optimally and ought to be replicated and expanded into other initiatives in the future. Program evaluation aims to: demonstrate the program's effectiveness to financiers; improve program effectiveness and implementation; document program accomplishments; manage limited resources more effectively; justify current funding; and support the case for increased funding levels.

Evaluation will also serve to ensure that scientific bases are used for decision-making; that services are result-oriented and socially equitable; that service agencies perform effectively; and that provinces and provincial agencies are accountable for their respective services. This can be achieved through clear plans, feedback systems, and inclusive partnerships that enable continuous improvement and learning. Evaluation effectiveness is facilitated by adherence to pre-set standards. Stakeholder engagement promotes the involvement, input, and power-sharing of organizations and individuals invested in evaluation findings. This approach ensures stakeholders' unique views are heard and that the evaluation addresses program goals, functions, and results. It enhances the likelihood of evaluation efficacy, improves credibility, safeguards human subjects, fosters cultural competence, and helps circumvent real or perceived conflicts of interest.

Program Profile and Organizational Description

The lead program evaluator for the New Brunswick EMP was selected on the basis of relevant expertise. The evaluation team's members bring different areas of expertise and are assigned evaluation responsibilities accordingly. Accepting this position means committing to fulfilling all requisite obligations. The program evaluator's role involves partnering with the organization and working to improve the EMP (Module 3: The Role of the Program Evaluator, n.d.). This requires interacting with concerned parties — including consultants, collaborators interested in program success, and other stakeholders — to plan and carry out evaluations. The evaluator must develop an evaluation plan, establish goals and a budget, organize data collection, present results, and hold discussions with consultants (U.S. Department of Health and Human Services, 2005).

Throughout these dealings, ethical standards must be maintained — for example, treating people considerately and fairly. The evaluator is ethically and professionally bound to report results honestly and fully. Human subject protection is equally important: evaluators must ensure that subjects are unharmed, that participation is voluntary, and that subject confidentiality and privacy are protected. One key obligation is clarifying what can and cannot be done — including how far program services quality and outcome utility can be clearly judged, how far the project can assume responsibility for changes that occur, and the limitations on generalizing results to future conditions (Managing the Evaluator: Roles, Responsibilities and Maintaining the Relationship, n.d.).

Considerable personal bias may arise if a program evaluator becomes an advocate for the program, since making statements unfavorable to it will become difficult. Managers are generally not entrusted to assess their own programs because of conflicting interests. Similarly, individuals involved in program development may be biased and inclined to have their own suggestions implemented, potentially prioritizing their own input over objective assessment (Bowen, n.d.).

The EMP in New Brunswick serves patients in their communities and homes and does not discriminate on the basis of age. Managed by the Regional Health Authorities, it offers quality home health care services to all eligible residents whose needs can be safely met in the community. The program offers various options to patients, including:

The EMP recognizes that each patient and their family have unique needs and circumstances, and therefore strives to develop a targeted plan of care to fulfill each patient's particular requirements (Extra-Mural Program, n.d.).

The organization, based in New Brunswick, offers a range of services in homes and communities with professionalism. It not only works to enhance the health of patients, but also endeavors to prevent illness and help people regain and maintain good health — all within home and community settings. In palliative care, the organization strives to provide quality of life to the satisfaction of the patient (Vitalite Health Network, 2011).

The focus of this evaluation is on how the program can advance institutional priorities using both the distinctive and joint influence of the program. In evaluating the program sequentially, the focus remains on program objectives and ongoing enhancements. All assessments aim to inform decisions. The key purposes of examining the program are: finding out how much progress has been made toward achieving the goals; and finding ways of enhancing the program's performance.

The purpose of carrying out this evaluation is to determine the extent to which the project has achieved its strategically set objectives, as reflected in accountability reports. This evaluation seeks to confirm the results of the EMP and to establish the suitability of the measures in place for inspecting and assessing such projects. In the area of health, objective achievement is assessed and feedback is given in the form of results and outputs. Long-term outcomes include the impact on clients, families, and communities in terms of health. Short-term outputs can be limited to measures of program reach, such as the number of patients and communities served.

This proposal consists of the research and analysis to be carried out, presented in outline form, along with the work plan, the reporting schedule, and deliverables. Researchers and consultants should be included in the project to enable it to be carried out effectively and successfully (Appalachian Regional Commission, 2012).

Research Questions and Ethical Considerations

Although evaluation team members typically brainstorm and generate several possible questions, the following have been prioritized as key program aspects to be studied at this time.

For ascertaining whether program implementation is proceeding as planned: Have appropriate personnel been recruited? Have they been properly trained? What challenges was the program tasked with tackling? How were these problems addressed? What were the program's long-term goals? Were they achieved? Is the program working? How does it relate to long-term goals? What are the characteristics of program beneficiaries? What contributed to achieving program objectives? What measures accurately reveal the benefits the program has produced and its long-term impact on the community? (Appalachian Regional Commission, 2012).

For determining whether program objectives are met: Has proper testing and treatment been conducted for more patients? Are patients complying with treatment — that is, not lost to care and follow-up?

The program manager and team can better understand the program's structure by examining its objectives and modus operandi to improve clients' lives. Clear objectives and work procedure requirements will help personnel remain goal-oriented and focused on achieving program outcomes. There is a broader call for accountability regarding resources allocated to these programs, as they rely on public and other funds. Those who use and participate in these services — including workers and volunteers — need to know whether the programs are effective. Measuring outcomes thus serves both accountability and improvement: learning achieved through outcome assessment can be fed back into the system and incorporated into program activities (Yung et al., 2012).

Before data collection begins, ethical considerations must be carefully addressed. Strategies to protect the rights and dignity of evaluation participants will be incorporated into the project design and implementation. The evaluation can benefit participants and others through changes made at the program or agency level — for instance, it can guide strategies for improving the program's impact, resulting in more positive outcomes for current or future participants. However, associated risks must also be considered. Every effort will be made to identify potential harms and take steps to reduce them. Participation in the evaluation will always be voluntary. Participants will have the right to: choose whether or not to participate without penalty; withdraw at any time, even after initially agreeing to participate; and refuse to complete any section of the evaluation, including declining to answer any question (Evaluation Ethics, 2009).

It may not always be possible to conduct evaluations anonymously, as identifying information such as the participant's name may need to be collected. Nevertheless, all information gathered will be treated as confidential and will not be shared with others. Where concerns about the safety of participants arise, the evaluation will take care to protect all participants as much as possible. It is important that the evaluation be designed to be both high quality and ethical (Evaluation Ethics, 2009). Ensuring informed consent implies:

Ethical safeguards will help participants feel at ease and provide full information, knowing they are protected. To guarantee confidentiality, the study will incorporate the following strategies: