A cost-effectiveness analysis of a housing and recovery program for people with severe and persistent mental illnesses and precarious housing

ABSTRACT

The Doorway program supports people with Serious and Persistent Mental Illnesses (SPMIs) receiving care within Victoria’s public mental health system and who are precariously housed. This paper reports on its cost-effectiveness. The study design was difference-in-differences involving a Treatment-As-Usual comparison group (TAU) using deidentified client records. The primary outcome measure was the proportion of days of secure housing per participant during the study and pre-study periods. Similarly, health service usage was also extracted from participants’ records. The cost of public mental health service usage was then estimated, including using governmental published data. The program’s Incremental Cost Effectiveness Ratio (ICER) and Cost-effectiveness Acceptability Curve were estimated using the bootstrap method. An ICER threshold of Australian $50,000 per additional person housed (hereafter A$) at 100% was set to determine cost-effectiveness and the probability of cost effectiveness at A$50,000. Mean total costs (secure housing and mental health service usage) were greater for the Doorway group compared to the TAU group (A$22,987). Mean percentage of days securely housed was also greater for the Doorway group (by 45.4%). The ICER value was A$50,613, the probability of this being 83.1%. The Doorway intervention was both more costly and more effective. Its cost-effectiveness approximated the pre-specified ICER threshold.

KEYWORDS:

• Cost-effectiveness
• housing and recovery program
• precarious housing
• serious and persistent mental illnesses

Introduction

The Doorway program is a Housing First (HF) program that provides support to people with Serious and Persistent Mental Illnesses (SPMIs) receiving care within Victoria’s public mental health system and who are precariously housed. By “precariously housed” is meant housing where there is immanent risk of the person becoming homeless, in contrast to the person being absolute homeless. This paper reports its cost-effectiveness including uncertainty estimates around the ICERs. It is one of very few papers to do this, the exceptions being the study of cost-effectiveness of the Chez Soi and the Chez Soi Aboard programs (Latimer et al., 2019; Latimer et al., 2020; Lemoine et al., 2021).

The Doorway program differs from the programs, referenced above, which have very high levels of absolute homelessness, referred from the homeless persons sector (from the streets or emergency housing shelters). Only 5% of people in receipt of the Doorway program are absolutely homeless, referred from the streets and emergency homeless shelters Whereas 83.6% Chez Soi Intensive Care Management study (Stergiopoulos et al., 2015) and 69.7% in the Chez Soi D’Abord (Tinland et al., 2020) were in this absolute homeless category.

It is important to study this precarious housing group as they have not previously been the focus of study. This is because they have usually been combined in populations mainly consisting of people who are absolutely homeless (Baxter et al., 2019).

They are also an important, policy-relevant group as they involve large numbers of people, including within the public mental health sector. In Australia, (population, 23.8 million in 2015), it was estimated there were 59,800 adults (0.4% of the adult population) with a Severe and Persistent Mental Illness (SPMI) who had complex, multi-agency needs including housing. Eleven per cent (n = 6578) of people in this complex needs group could be identified as being precariously homeless (Whiteford et al., 2017). This number exceeds the number of people with SPMIs who were currently experiencing homelessness (estimated in this study as being 5.2% or 3109 for SPMI with psychoses, no-figure for SPMIs without psychoses, but unlikely to be greater than a further 5% or 2950).

A previous paper published by this group, demonstrated that Doorway housed 89 (57%) of 157 accepted participants during the period 2015–2018 (Dunt et al., 2022). The 157 Doorway participants overall were housed for significantly more days (119.4 extra days per participant) than TAU participants. There was a significant and positive Doorway effect on health outcomes as assessed by the Health of the Nation Outcomes Scales (HoNOS) (overall scale as well as one dimension of the scale). Doorway participants had significantly reduced length of stay during acute and community hospital admissions (7.4 fewer days per participant) compared with the TAU group. Nevertheless, while these results are favorable both with regard to improved health outcomes and reduced service use, the overall cost-effectiveness of the Doorway program remains unknown.

Cost-effectiveness analysis (CEA) provides relevant information for governments to justify the funding of these programs in a real world situation. It is important though that this CEA is best practice and its reporting is in conformity with the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement (Husereau et al., 2013). The Chez Soi and the Chez Soi Aboard programs do this, specifically using Cost-effectiveness Planes and Cost Effectiveness Acceptability Curves (CEACs) (Latimer et al., 2019; 2020; Lemoine et al., 2021; Nixon et al., 2009). Earlier economic studies of homelessness programs, by contrast, have focused more frequently on costs of these programs and their cost offsets alone (Gilmer et al., 2014; Ly & Latimer, 2015; Stergiopoulos et al., 2015). The aim of this paper then is to present findings relating to the cost-effectiveness of the Doorway program.

The Doorway program

Doorway extends the Housing First (HF) model which was pioneered in the early 1990s by Tsemberis and his Pathways to Housing program in New York City (Tsemberis et al., 2004). HF is an evidence-based intervention model that involves the immediate provision of permanent housing and wrap-around supports to individuals who are homeless and living with serious mental illness, rather than traditional “treatment then housing” approaches.

The Doorway program is an integrated housing and recovery support program delivered by Wellways, a non-governmental, not-for-profit agency in Victoria, Australia. It provides care for people with SPMIs and mainly have precarious housing, on referral from the Victorian public mental health sector (Nous group, 2013; Dunt et al., 2017; Dunt et al., 2021). Doorway demonstrates fidelity to almost all of the characteristics of the Housing First model using a scale developed to assess this (Gilmer et al., 2014; 2010). A limitation of the Doorway program, as an HF program is that it offers accommodation up to 18 months only, not ongoing secure housing as in other HF programs.

The Doorway model, like a number of HF programs e.g. Chez Soi, provides support to participants to source their property of choice (Stergiopoulos et al., 2015) They do this in Doorway, using inexpensive properties available on the open private rental market. Housing and Recovery Workers (HRWs) at Doorway are responsible for implementing the model on a day-to-day basis, on behalf of an integrated team. The HRWs role is non-clinical.

Doorway also provides rental subsidies, advocacy and brokerage support, varying between A$464 and A$6038 per participant across clinical partner regions. This wide difference in rental subsidies reflects the differences in rents in disadvantaged, rural areas and high-demand, inner metropolitan. Their role combines housing and (non-clinical aspects) of mental health recovery work. HRWs sit, co-located at a partnered clinical service with a team that was already providing ongoing clinical support. Their dual roles are unusual as typically their activities have been divided between a Home-Based Mental Health Worker, and a Housing Worker. More information about Doorway and the HRW role is available (Dunt et al., 2022).

The control program offers TAU available for all people in receipt of public mental health care in Victoria. It does not offer the subsidies and services only available under the Doorway program.

Methods

Ethics approval for the evaluation was granted by the St Vincent’s Hospital Melbourne (SERP HREC reference number HREC/16/SVHM/241) on behalf of all participating hospitals. Site Specific Approvals were also received from all participating health services. Doorway participants had to provide informed (verbal) consent for the collection of their deidentified health service usage, as they formed the study population for an interview study that does not form part of this study (81 of 157 providing consent). The control participants did not provide informed consent, as they were not interviewed and only included for the collection of de-identified usual care data. Deidentified data for both groups is available from the authors on request.

Original study design. The original study had a difference-in-differences design, with Group being either the Doorway population (study group) or a Treatment As Usual (TAU) (control group). Time periods were 12 months pre-entry and post-entry up to Doorway program end at 18 months. The primary outcome measure was the proportion of days of secure housing per participant during the study period. Costs included secure housing and mental health service utilization, both described in detail below. To minimize bias and confounding, all analyses controlled for ten potential confounders including housing status at start date [proxy for homelessness], age group, gender, employment status, education status, pension, country of birth, indigenous status, preferred language, SPMI diagnosis, days on the program. Analyses for continuous outcome variables were based on multivariate GLM modeling. The Group*Time Period (or Group*Time Point) interaction term was used to measure the Doorway effect using indicator levels at pre-/at entry and post-entry. A full description of Methods, other than housing and mental health service usage described here, is available (Dunt et al., 2022). The design is simple difference-in-difference as the dataset without incorporation of changes in trends pre- and post-entry. This was because there were insufficient data points to analyze and assess the effects of changes (six points defining 5 periods – see below).

Secure housing status and mental health service usage data was extracted from participant’s electronic hospital and Doorway records in deidentified, non-reidentifiable form (Department of Health. Victoria, 2015). Where possible, data from all Doorway Partners (or equivalent in control group) were collected for 12-months pre-entry, entry and then 6-monthly throughout the Doorway period and finally at exit. Hospital records are the same statewide in Victoria and thus do not differ between Doorway and control group.

To be eligible for entry to the study, individuals needed to be aged between 18 and 65 years; to have an SPMI; to be homeless/at risk of being homeless whose housing was precarious and at immanent risk of becoming homeless as identified by the clinical partner health services for the Doorway group and equivalent for the control group; to be on referral from, and in current receipt of public mental health care from Doorway’s clinical partners or equivalent in control group; and to be case managed. Operationally, they needed to have had contact with their health service after 30 June 2014 with some of that contact being between 1 July 2015 and 30 June 2018 (the Doorway study period).

Study groups. The Doorway group (N = 157) was drawn from four clinical Mental Health Services, three in metropolitan Melbourne – St Vincent’s Hospital, the Alfred Hospital and the Peninsula Hospital and one in regional Victoria – the Latrobe Regional Hospital. Eligible participants entered the study on their acceptance by the program. The control group (N = 80) was drawn mainly from Austin Health, a metropolitan mental health service geographically adjacent to that of St. Vincent’s (n = 63). A further 17 controls were drawn from the Latrobe Regional Hospital (completely separate from the drawing of Doorway participants).

Outcome measures. The primary outcome measure was the proportion of days of secure housing per participant during the study period. Secure housing was defined as private rental units organized through the Doorway program (or equivalent agencies in the catchment area of the Latrobe Hospital for the control group) or, in a few cases paid for by the person themself. Non-secure housing included all other arrangements such as rooming houses, supported residential services, caravan parks, living with family (typically under duress), couch surfing, etc. It also included Prevention and Recovery Care (PARC) (short-term up to 30 days) and Community Care Units (CCUs) (long-term – up to 2 years, though sometimes more), both of which are forms of bed-based mental health care, not secure housing. There were no participants in social or community housing.

Cost measures – housing. Housing costs were based on the Doorway contribution to housing costs (Personnel and Housing) – see Table 1 (Wellways, undated). Housed Doorway participants were charged Housing and Personnel costs. Unhoused Doorway participants were charged Personnel costs only.

Table 1. Doorway costs 2015–2018 (A$ nominal and A$ 2015–2016).

Costs	1-yearly	Multiplier	3-yearly	3 Year per capita costs based on 157^% participants
Housing Costs (excluding personnel)
Participant rental in advance	61,320	X1	61,320
Rental subsidy	133,120	X3	399,360
Landlord insurance	8500	X3	25,500
Surety fund	75,000	X1	75,000
Brokerage	75,000	X1	75,000
Lease, property licensing costs	120,984	X3	362,952
Corporate costs^A$	107,250	X3	321,750
Housing Sub-total	581,174		1,320,882	8,413
Personnel support costs
HRWs^%%	690,300	X3	2,070,900
Leading staff	234,992	X3	704,976
Corporate costs	35,750	X3	107,250
Personnel subtotal	961,042	X3	2,883,126	18,364
TOTAL	1,542,216		4,204,008	26,777
In 2015–2016 Dollars^#	1,52,6543		4,161,284	26,505

Notes:

^% Not all of the participants were housed, or in fact housed for the full 18 months period, 89 were housed generating 30,809 housed days (8966 non-housed days) and 68 were not housed at all (7734 days not housed).

^A$ Admin (corporate) costs of A$429,000 over 3 years allocated 75% to Housing (non-personnel) 25% to Personnel i.e. A$321,750 and A$107,250 respectively.

^%% HRW costs are A$17,000 per staff member. There were 5.9 FTE staff members including 0.6FTE funded by Wellways.

# Calculated using the General Government Final Consumption Expenditure (GGFCE) price deflator.

Other housing costs were not included in the cost and cost-effectiveness analysis as the perspective for this study is limited societal, including both health and costs of secure housing to the Victorian state government. Other housing costs included Living with family (typically under duress), Rooming houses, Couchsurfing and Sharehouse with costs for these options reported in Appendix 1.1. It is noted that these housing options almost universally provided insecure housing for the TAU participants.

Support payments from the Australian, not the Victorian government, were used by both the Doorway and TAU participants. These include the Disability Support Pension (including additional Commonwealth Rent Assistance) or New Start (unemployment) benefits support rental costs for TAU participants. The Disability Support Pensions or the similar, though less generous, New Start government benefits also supported Doorway participants (30% of their pension benefit). These long-term payments were not included in the cost and cost-effectiveness analyses due to an inability to consistently access this data. They were also unlikely to change due to entry into the Doorway program, that was not long-term.

Cost measures – health service utilization: Costs included mental health service utilization and this included.

• Acute mental health hospital admissions (National Hospital Cost Data Collection Report, 2019; Report on government services 2018a, n.d.a),

• Community mental health hospital admissions (Report on government services 2018a, n.d.a),

• Admitted and non-admitted ED presentations (Report on government services 2018b, n.d.b) as well as

• Community health services utilization (Report on government services 2018c, n.d.c).

They do not include GP visits, pharmaceutical use, ambulance call-outs or use of the legal system. Unit costs for mental health service usage were derived from published government sources.

Data analysis

Part A descripion of Doorway population, costs and effectiveness

Mental health service usage data was available for up to 12 months pre-entry (Doorway and equivalent for TAU) and therefore could be analyzed pre- and post-entry. To aid the analysis, a Set date (date of intake for Doorway group and first service usage for the TAU group) as well as 6-monthly time periods and 6-monthly time points in relation to the Set date were established for each participant. The time periods used in the model consisted of two pre-entry 6-monthly time periods and three post-entry time-periods (five in total) defined by six data points – beginning12-months pre-entry, 6-months pre-entry, entry, 6-months post-entry, 12-months post-entry. Where there are two assessments within a 6-month period, the value used in the analysis is the mean value of the two assessments. Housing data for participants was only available for post-entry to Doorway (and equivalent in the TAU group) and could only be analyzed post entry. Operationally, participants needed to have had contact with their health service after 30 June 2014 with some of that contact being between 1 July 2015 and 30 June 2018 (the Doorway study period).

As noted, analyses for continuous outcome variables were based on multivariate Generalised Linear Models (GLM) modeling (Gaussian family type) incorporating ten potential confounding variables. These were deleted sequentially, in individual analyses, where variables with an F value of < = 0.200, as it was found that removal of these variables had little effect on the Rsq value of the model and slightly improved the Adjusted Rsq value (IBM Corp., 2020). Dichotomous outcome variables were analyzed using logistic regression. Odds Ratios were used to estimate Effect Size. All analyses were done using SPSS (Version 27).

Part B cost-effectiveness analysis

As noted, the study is reported in conformity with the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement (Husereau et al., 2013). The evaluation perspective was limited societal, including both health and secure housing costs incurred by the Victorian state government, and the time horizon was the time spent on the Doorway or TAU programs (Sanders et al., 2016). As the time horizon (availability of Doorway program to participants) is limited to a maximum of 18 months, no discounts for cost or effectiveness measures were made. Since housing data was only available post-entry, analysis involving both housing and mental health service usage could only be analyzed post-entry for the economic evaluation.

A health economic plan was developed a priori. The incremental difference in total cost was divided by the incremental difference in the percentage of days housed, to determine the incremental cost-effectiveness ratio (ICER) between the Doorway cohort and the TAU cohort. The analysis utilized the bootstrapping method (5,000 repetitions) of individual ICERs to determine the point estimate as well as the 50%, 75% and 95% confidence ellipses (Nixon et al., 2009). Results are visually represented in a cost-effectiveness plane. An ICER threshold of A$50,000 per additional person housed at 100% was set to determine cost-effectiveness.

The results are also presented in a cost-effectiveness acceptability curve (CEAC) using the same bootstrap method. The “cost-effectiveness threshold” represents the willingness to pay for each additional person in the Doorway intervention who achieves 100% housing. The “probability of cost-effectiveness” represents the probability that the Doorway intervention will be cost-effective at each of the given willingness to pay thresholds, with the probability of cost effectiveness at A$50,000 reported (Nixon et al., 2009). A sensitivity analysis was conducted to test the robustness of the findings with respect to the probability of being cost effective within the A$50,000 willingness-to-pay threshold. Should the Doorway program cost less, 25% will be added to the cost; should the Doorway program be more effective, a 25% reduction will be applied to the effect.

Results

Part A brief description of Doorway population, costs and effectiveness

There were significant differences for a few of the ten socio-demographic characteristics of the Doorway and TAU groups that were measured with adjustments for up to ten of these variables being made during analysis (Dunt et al., 2022).

On intake into Doorway, there were 36 (22.9%) participants living/staying with family; 29 (18.5%) couch surfing; 23 (14.6%) supported residential accommodation; 22 (14%) boarding house, hostel or rooming house; 15 (9.6%) house or flat; 14 (8.9%) crisis accommodation; 4 (2.5%) caravan, garage, motel or building; 2 (1.3%) itinerant. Twenty-one of the 36 participants (58.3%) who were living/staying with family spoke of “family conflict and difficulties”; 20 (55.6%) had Alcohol or Drug issues.

Doorway participants stayed for different periods of time (median 38 weeks) in the program. Seventy six participants exited the program early (median 14 weeks). Among those housed, the median length of time to find appropriate housing was 11 weeks, (range 0–43).

There were a limited number of significant differences across the ten socio-demographic characteristics in the Doorway and TAU groups. The only major difference in these ten characteristics was that, compared to the control group, lower proportions of the Doorway group were in the 18–20 year age group and the 50 + age groups (p = 0.016). There were major differences in Education status and Pension status between the two groups, however the amount of missing data in the control group (37.5% and 36.3%) made it difficult to draw any firm conclusions. There was also a major difference in the type of Housing occupied at Start date with (i) lower proportions of the Doorway group being of no fixed address or living in a house or flat, and (ii) higher proportions of Doorway participants couch surfing or living/staying with family. All of these were adjusted for during the GLIM multivariate analysis (Dunt et al., 2022).

Primary outcome. The average percentage of days securely housed in the Doorway group was 53.0% (SD 36.1%) and for the TAU group, 7.6% (SD 19.8%), with a mean difference of 45.4% (95% CI 36.0% to 54.8%), with a greater effect favoring the Doorway cohort.

Doorway housing costs. Total nominal costs across the three-year period were A$4.2M consisting of A$1.32 m for housing (non-personnel) costs and A$2.88M for personnel costs (mainly for the HRWs). The total cost of A$4,161,284 (2015–2016 A$) for the 157 participants (registered for 47,509 days) was A$26,505 per participant.

Mental health admissions and costs – Acute and Community Length of Stay (LOS). After adjusting for independent variables, GLM analysis indicated that the Doorway effect on Length of Stay was significant with the Doorway Group having a greater reduction in LOS of 7.4 days per participant (F = 3.178, p = 0.013). (F Value and p-value, respectively, test the null hypothesis that the model does not explain the variance of our response variable. The F Value is computed as MS_Model / MS_Error, and under the null hypothesis, F Value follows a central F-distribution with numerator DF = DF_Model and denominator DF = DF_Error).

Costs for acute admissions were estimated at A$1238.6 per day (and for community admissions, at A$510.5) per day in 2015–2016 dollars. The cost difference in the two admission types combined, over the study period was A$6335 per participant in favor of the Doorway group – Table 2. This method of calculation introduces a conservative bias as the pre-period is 12 months, and the post-period is 18 months. Changes over time in admission costs were examined for a Doorway effect, using GLM and this was statistically significant, favoring Doorway (F = 2.549, p = 0.038 95% CI 337-12,333).

Table 2. Costs for Hospital admissions (2015–2016 A$) for whole Doorway program.

Type of admission	CONTROL (N = 80)			DOORWAY (N = 157)
Six monthly period	Number of admissions	Length of stay	Total cost (A$)	Number of admissions	Length of stay	Total cost (A$)
COMMUNITY
Period 1	5	86	43,901	13	432	220,527
Period 2	1	1	510	15	1061	541,619
Period 3*	3	68	34,713	11	547	279,233
Period 4	1	17	8,678	8	272	138,851
Period 5			–	3	32	16,335
Community Total	10	172	87,803	50	2344	1,196,565
ACUTE
Period 1	66	1402	1,272,029	22	466	577,181
Period 2	24	658	341,850	19	446	552,410
Period 3*	25	576	403,779	7	191	236,570
Period 4	16	236	308,408	7	90	111,473
Period 5	30	704	460,754	6	120	148,630
Acute Total	161	3575	2,786,820	61	1313	1,626,265
TOTAL
Period 1	71	1488	1,315,930	35	898	797,709
Period 2	25	659	342,360	34	1507	1,094,029
Period 3^%	28	644	438,492	18	738	515,803
Period 4	17	253	317,086	15	362	250,323
Period 5	30	704	460,754	9	152	164,966
TOTAL	171	3747	2,874,623	111	3657	2,822,830
Cost pre-Doorway (Periods 1,2)			1,658,290			1,891,738
Costs Intake to Exit (Periods 3,4,5)			1,216,333			931,092
Pre-post costs (savings)			441,958			960,646
Savings per participant			5524			11,860
Savings difference between Control and Doorway per participant						6335

Note: *First six months in the program for Doorway participants.

ED presentations and costs: Average cost per admitted acute ED presentation 2015–2016 was A$907 in Victoria. For non-admitted cases, it was A$408. The Doorway group had 0.28 fewer presentations per participant. After adjusting for independent variables, GLM analysis indicated that the Doorway effect was not a significant predictor of the number of presentations after Holm–Bonferroni correction (F = 2.667, p = 0.031) (Haynes, 2013). The cost difference for combined admitted and non-admitted cases was A$145 less for each Doorway participant – see Appendix 1.2. After adjusting for independent variables, changes over time in the cost of ED presentations were analysed using GLM. The Doorway effect was significant favoring Doorway (F = 2.844, p = 0.023 95% CI 19.7-270.3).

Community health services contacts and costs. Doorway had an added saving of 0.05 days per participant. After adjusting for independent variables, GLM analysis indicated that the Doorway effect was not significant. Average cost per community contact was A$384 Changes over time in the cost of community health services contacts were analysed, after adjusting for independent variables using GLM, with the Doorway effect showing not to be significant (F = 1.953, p = 0.100).

Part B Cost-effectiveness analysis

Mean total costs (secure housing and health service utilization) were greater for the Doorway group compared to the TAU group (MD A$22,987; 95% CI A$10,775–A$35,200); with Doorway costs at A$43,648 (SD A$36,561) and TAU costs at A$20,661 (SD A$41,760).

Mean percentage of days securely housed was also greater for the Doorway group compared to the TAU group (MD 45.4%; 95% CI 36.0%−54.9%); with Doorway proportion at 53.0% (SD 38.1%) and TAU proportion at 7.6% (SD 19.8%). The ICER value was A$50,613 (A$22,987/45.4%) for each additional person 100% securely housed in the Doorway program over an average 1 year time horizon. The probability of cost effectiveness at A$50,000 was 83.1% (Figure 1).

Figure 1. Cost-effectiveness acceptability curve for the Doorway compared with Treatment As usual.

The Doorway intervention was not considered a dominant intervention as it was both more costly and more effective, and this is seen with the point estimate in the top right quadrant (Figure 2). It is noted that these results have a high degree of certainty with the 50%, 75% and 95% confidence ellipses all within the upper right quadrant.

Figure 2. Point estimate and confidence ellipses for the Doorway compared with Treatment As Usual.

Since the Doorway program cost more rather than less, but was more effective, the sensitivity analysis consisted of reducing the effect (secure additional days housed) by 25%. In this scenario, the previous funding decision was upheld with ICER value of A$71,465 (A$22,987/32.2%) for each additional person 100% securely housed in the Doorway program over an average 1 year time horizon. The point estimate and the 50%, 75% and 95% confidence ellipses all remained within the upper right quadrant representing an intervention that was more costly and more effective (Appendix 1.3). The probability of cost effectiveness at A$50,000 was 12.2%.

Discussion

Based on a 24-months study period, Lemoine et al. (2021) in the French Un Chez Soi d’Abord RCT, estimated that there was an incremental cost of €5.3 per additional day in secure housing for their HF group using a lifetime horizon. The acceptability curve showed that the HF intervention was cost-effective in 42% and 63% of simulations at a threshold (willingness-to-pay ceiling value) of €30 and €60 per additional day spent in independent housing respectively.

Latimer et al. reported the cost-effectiveness findings for the Canadian At Home/Chez Soi Trial. The authors did this for both Assertive Community Treatment (ACT) and Intensive Case Management (ICM) (Latimer et al., 2019; Latimer et al., 2020). ICM supports people with more moderate needs than people requiring ACT. At CanA$60 per extra day, there was 80% chance that ACT HF was cost-effective. At A$67 per extra day, there was 80% chance that ICM HF was cost-effective.

Incremental Cost-Effectiveness Ratio for the Doorway program approximated A$50,000 for each additional person 100% housed in the Doorway cohort. With an average length of stay in the Doorway program of 38 weeks, this was just over A$1300 per week. At a threshold (community willingness-to-pay ceiling value) of A$50,000, the probability of Doorway being cost-effective was high (83.1%).

The cost-effectiveness outcomes are not directly comparable to Chez Soi or Chez Soi D’Abord which provide services to the homeless who are “rough sleepers.” Doorway provides support to people with SPMIs with precarious housing who are at risk of becoming “rough sleepers.”

Funding for housing is significantly greater than average rental housing costs as it includes the intensive support provided by the HRWs. The wraparound support provided by Doorway compares favorably with A$3,575 per week for Community Care Unit, the most appropriate alternative community arrangements existing in Victoria. CCUs, partly funded by a personal contribution for rent and utilities, provide mental health support services (Report on government services, 2018c, n.d.c). Unlike CCUs, Doorway has no major capital costs either to government or renters, these being costed to the private owners of the flats/apartments.

It should also be noted that Doorway has significant positive effects on clinical outcomes (HoNOS). Some services not included in this study may demonstrate further cost offsets for Doorway (Baxter et al., 2019)

The study has some strengths and some weaknesses. To our knowledge, it is the first cost-effectiveness study of a program for people with SPMIs, at risk of homelessness, rather than with absolute homelessness. It is controlled both in terms of there being an external control group and having pre- and post-entry comparisons but is not a randomized control, which was not feasible. It also uses data for Doorway and TAU participants based on routinely collected mental health service use, including prior to entry, rather than relying on the index person’s availability and their memory of service use. It suffers some missing data associated with this use of treatment records but less than relying on a person’s availability and their memory. The design is a simple, rather than full difference-in-difference study type. This is acknowledged as a limitation as it is unable to detect changes other than the introduction of the Doorway program. These include for example change in other barriers to access care.

As noted, the Doorway program only offers accommodation up to 18 months, not ongoing secure housing as in other HF programs. Its cost-effectiveness analysis necessarily has this same time horizon and is a limitation.

Policy implications. Doorway is clearly a successful program and model – a model based on subsidizing and supporting people experiencing precarious housing, to find accommodation at the cheaper end of the private rental market. It can be readily be extended to other regions in Victoria and elsewhere.

It would be advantageous to reduce waiting times to enter housing, now averaging between 12 and 16 weeks (depending on Region) for Doorway participants. This could be addressed if Doorway, entered a strategic alliance with other homelessness services that provide more immediate housing.

Conclusion

The results of this study are broadly in line with the cost-effectiveness analyses of Chez Soi and Chez Soi D’Abord (Latimer et al., 2019; 2020; Lemoine et al., 2021). The study also adds to the generalizability of their results outside North America and France. Doorway also demonstrates more clear-cut and positive costs and benefits than other appropriate alternatives available in the Victorian community. The Doorway model with its focus on the cheaper end of the private rental market, with support of a rental subsidy and an HRW, highlights new and substantial opportunities to house people with SPMIs and precarious housing, and in so doing, further develops the HF model.

Acknowledgements

The authors would like to acknowledge and thank the following people and organizations for their help in preparing the evaluation report: Doorway staff and nonDoorway stakeholders who provided information about the operation of Doorway; Doorway staff who were instrumental in providing the quantitative data for the evaluation from Wellways databases, undertaking the cleaning of the data, providing extra information as requested, and working with the health services to provide hospital data; Staff at the Alfred Hospital, Latrobe Regional Hospital, Peninsula Health and St Vincent’s Health Services who provided the requested hospital data for Doorway participants; and staff at the Latrobe Regional Hospital and Austin Health who provided the data on the control group participants. The authors would particularly like to acknowledge the input of Ms Kate Wendt and Mr Spiros Papadopoulos of the Austin Hospital, and Dr Stuart Lee of the Monash Alfred Psychiatry Research Centre for their work in accessing data in their respective hospitals. The authors would also like to thank Ms Beth Fogerty and Mr Josh Oates of the Doorway program.

Disclosure statement

The author(s) declare the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Additional information

Funding

The first and second author received funding to undertake this study from Wellways, the organization delivering Doorway. This funding in turn was 21 derived from the Victorian Government Department of Health and Human Services as part of its funding of Wellways for the Doorways programme.

Notes on contributors

David Dunt

Emeritus David Dunt is a medically-trained public health specialist and epidemiologist with major interests in health services research and program evaluation. He has a particular interest in the evaluation of complex government health care programs. He is a pioneer in both academic public health, academic health services research and health program evaluation in Australia. He has 185 refereed publication as well as 165 other publications, mainly government reports in 1991, he established and directed for 17 years, the then NHMRC National Centre for Health Program Evaluation, now known as the Centre for Health Policy in the Melbourne School of Population and Global Health. He has had a major impact on government policy including the conduct of two Ministerial reviews – the Review of Mental Health Care in the Australian Defence Forces and the Independent Study into Suicide in the Ex-Service Community. His recommendations led to major reorganization of mental health services in both the ADF and the DVA. He has an international reputation in research relating to the delivery of after-hours general practice after hours. His study of theoretical and methodological issues concerning the measurement of patient satisfaction is regarded as a landmark article on this topic.

Susan Day

Susan Day is an experienced social scientist, program evaluator and data analyst. She was first trained as a social worker and later in program evaluation. She has worked with David Dunt since the mid-1990s. Her research interests have covered such diverse subjects as dementia programs, primary medical after care, mental health service, 22 funding of rural health services, coordination of health services, osteoarthritis and assessing techniques of health utility.

Natasha Brusco

Natasha Brusco is an experienced physiotherapist and health economist, with 20 years of health service experience in clinical, management and leadership roles across multiple public and private services. Dr Brusco is currently the Rehabilitation Stream Lead for the Rehabilitation, Ageing and Independent Living (RAIL) Research Centre, Monash University. She has a background as a physiotherapist in neurological rehabilitation and she is highly skilled in health economics and health service research through large scale pragmatic trials. Over the past 5 years, Dr Brusco has been consulting for a number of disability organizations as a health economist and clinical resercher. In addition, she has been commissioned by the Victorian Department of Health and Human Services to provide workshops and industry support in the areas of health economics, including two economic evaluations for state-wide health service scaling projects for the Victorian Department of Health and Human Services. She has authored 42 refereed journal articles including reports of 5 clinical trials and 5 systematic reviews. Dr Brusco’s research and expertise focuses on health economics across many areas including cost-effective models of rehabilitation, the development of workplace health and safety programs, stroke care and the disability sector.

Appendices

Appendix 1.1. Non-Doorway housing costs

• Living with family. It is estimated that one quarter of parents spend between A$51 and A$100 to support their adult children who live at home, while slightly more than that spend between A$101 and A$200 (including internet, transportation and power) (Mozo, 2018).

• Rooming houses range from A$160 to 220 per week (excluding outliers) in Victoria (Gumtree, 2021; Trovit, 2021).³

• Couchsurfing This is estimated to be around A$50 per week at zero or nominal cost plus gifts to the host (Couchsurfing, undated).

• Sharehouse/flatmate range from A$220–320 per week (Flatmate Finders, undated).

Appendix 1.2. Costs of admitted/non-admitted ED presentations (2015–2016 A$) for whole Doorway program.

Table

Period presentation type	CONTROL (N = 80)			DOORWAY (N = 157)
	Number of participants	Number of presentations	Total cost A$	Number of participants	Number of presentations	Total cost A$
PERIOD 1
Admitted		25	22,675		20	18,140
Missing		40	16,320		14	5712
Not admitted					11	4488
Sub-Total 1	80	65	38,995	74	45	28,340
PERIOD 2
Admitted		6	5442		14	12,698
Missing		17	6936		12	4896
Not admitted					7	2856
Sub-Total 2	68	23	12,378	74	33	20,450
PERIOD 3
Admitted		12	10,884		10	9070
Missing		23	9384		7	2856
Not admitted					5	2040
Sub-Total 3	61	35	20,268	74	22	13,966
PERIOD 4
Admitted		3	2721		6	5442
Missing		13	5304		9	3672
Not admitted					3	1224
Sub-Total 4	54	16	8025	60	18	10,338
PERIOD 5
Admitted		5	4535		2	1814
Missing		22	8976		2	816
Not admitted					1	408
Sub-Total 5	38	27	13,511	41	5	3038
TOTAL		166	93,177		123	76,132
Cost pre Doorway (Periods 1,2)			51,373			48,790
Costs Intake to Exit (Periods 3,4,5)			41,804			27,342
Pre-Post Costs (Savings)			9569			21,448
Savings per participant			120			265
Savings difference in favor of Doorway						145

Appendix 1.3. Point estimate and confidence ellipses (a) and cost-effectiveness acceptability curve (b) for the Doorway compared with Treatment As Usual when the effect for the Doorway program was reduced by 25%