The Effectiveness of Marylands Homeless Solutions Program Outcome Evaluation

HOMELESS SOLUTIONS PROGRAM

Outcome Evaluation on the Effectiveness of Maryland’s Homeless Solutions Program

NAME

DEPARTMENT

INSTRUCTOR NAME

March 9th 2023

Methods Description

This an outcome-based evaluation seeking to assess Maryland’s Homeless Solutions Program’s (HSP) effectiveness in realizing its intended objectives (Schalock, 2013). The HSP’s concept map indicates that the program seeks to reduce the number of homeless people in the state, increase participants’ awareness about financial literacy, and increase participants’ access to mental health professionals. This evaluation pursues three objectives: to assess whether the HSP has effectively contributed to reduced homelessness in the state of Maryland, to determine whether participation in the program increases knowledge about financial literacy, and to determine whether participation results in improved mental health for participants. The variables of interest for the first objective are homelessness rates and the number of HSP program beneficiaries who have managed to secure and maintain stable housing. The variables of interest in objective two are knowledge about budgeting, borrowing, saving, and investing (Dewi et al., 2020). More specifically, the evaluation seeks to determine whether the number of homeless people in Maryland has declined as a result of the HSP, whether the participants report improved mental health, and whether program participants are more knowledgeable about budgeting, borrowing, saving, and investing as a result of taking part in the program.

To realize its objectives, the evaluation adopts a pre-post interventional research design, which involves measuring variables of interest before and after an intervention in the same group of participants (Aggarwal & Ranganathan, 2019). The basic premise of the pre and post-test design is that a variable of interest is measured before an intervention is administered and again after, allowing the evaluator to take note of trends and measure the extent of change at different points in time (Aggarwal & Ranganathan, 2019). The pre-post design is more appropriate for studying changes in knowledge and mental health status as it provides a means to look at where a participant starts and comparing it to where they end. The pre-post-test design is the most appropriate for this particular evaluation as it provides a means to assess how participants’ mental health status as well as knowledge and attitudes about financial literacy change over time as a result of taking part in the HSP. Further, the design provides a means to compare homelessness levels before the start of the HSP and after to determine whether the program has been effective in reducing homelessness in the state.

The HSP commits to reducing homelessness by providing funding and other technical support to continua of care to support homeless service programs and shelters in local communities. To assess the program’s effectiveness, the evaluators will select three homeless people crisis shelter homes that have benefited from HSP funding over the past three years to take part in the study. To ensure similarity among shelter homes, the evaluators will only consider homes that have been in existence for between five and ten years, are of comparable size in terms of beneficiary numbers, and have the HSP as their only source of funding. This will eliminate the effect of other contextual and organizational factors that could affect how participants fair in the home. The three homes will be selected conveniently based on their proximity to the evaluator to cut down on costs.

Upon obtaining the relevant approvals to carry out the study, the evaluator will select ten participants from each home who were admitted into the facility over the past one month to take part in the program. To ensure gender balance, five of the participants will be male and five will be female. The participants will be selected randomly from the enrolment registers that will be provided by the homes. They will be engaged in a meeting to explain the objectives of the evaluation and informed that their participation is voluntary and they could withdraw at any time. No coercion will be given to participants, who will be required to indicate their consent to participate by signing consent forms. To be eligible, participants will be aged above eighteen and be able to communicate in English as the evaluation will only be conducted in English.

The evaluation will collect both qualitative and quantitative data. Quantitative data will be used to analyse the first and third objectives, while qualitative data will be used for the second objective. For the first objective, which assesses how well the HSP helps participants secure and manage stable housing, quantitative data will be collected through document review. At the state level, the program will be evaluated by comparing homelessness rates before and after 2017, when the HSP began. The HSP began to be implemented in Maryland in 2017 following the passage of legislation that consolidated the homelessness programs across the state into a single program under the Department of Housing and Community Development (Luell, 2020). Data on homelessness rates will be obtained through reviewing annual reports on homelessness that are prepared every year by the Maryland Interagency Council on Homelessness for the governor and state general assembly.

At the individual shelter house level, the requirement that eligible shelter houses should have been in existence for between five and ten years will provide a means to make comparisons using 2017 as the base year. Annual data on the proportion of beneficiaries who have been able to secure and manage stable homes before HSP funding began in 2017 will be obtained and compared with data after 2017. Data from 2017 and earlier years will be organized into one panel, while the post-2017 data will be organized into another panel and the two compared to determine whether there are any significant differences. Analysis of variance (ANOVA) will be used to determine whether there are significant differences in the means of the two groups/panels of data, after which post-hoc tests will be conducted to identify where the differences lie (Shim & Siegel, 2008). The data will be obtained by reviewing enrolment records, annual reports, and individual beneficiary files. Data before 2017 will be regarded as pre-intervention data, while data after 2017 will be treated as post-test data.

Quantitative data will also be used to evaluate the third objective, which assesses how well the HSP contributes to improved mental health among homeless persons in funded shelters. To evaluate this objective, data on participants’ mental health status will be collected at the start of the first month, after three months, and at the end of the sixth month to assess the change in mental health status. Data will be collected using the survey technique by administering the Patient Health Questionnaire (PHQ-9), a self-assessment instrument used to screen for and measure the severity of depressive symptoms. The PHQ-9 will be distributed to each of the ten participants selected from the three shelters (Kroenke et al., 2001). PHQ-9 scores at baseline will be compared with scores recorded after three and six months of taking part in the HSP mental health support program.

Scores of 5 on the PHQ-9 represent mild depressive symptoms, 10 represents moderate depression, 15 represents moderately severe depression, and 20 represents severe depressive symptoms (Kroenke et al., 2001). Participants’ scores at baseline, as well as after three and six months will be arranged in three different panels and analyzed using descriptive statistics, chi-square, and ANOVA tests to determine the extent of the change in mental health status. An improvement in participants’ PHQ-9 scores will imply improving depressive symptoms and hence, that the HSP program has been effective in improving participants’ mental health. At the same time, worsening PHQ-9 scores or insignificant results will indicate that the program has little effectiveness in improving participants’ mental health.

Finally, qualitative data will be used to study the second objective, which assesses the HSP’s effectiveness in increasing knowledge and improving attitudes about financial literacy. The outcome of interest is knowledge about budgeting, borrowing, saving, and investing. Five interviewers will be engaged to conduct the interviews with the thirty HSP participants (10 from each of the three shelters). The face-to-face interviews will be guided by an interview guide and take between 25 and 30 minutes to complete for each participant. The guide will contain seven questions spanning around the process of developing budgets, the benefits of budgeting, strategies for proper saving, and investment options available when one has little capital.