Sheffield Hallam University has now produced its final evaluation of the Welsh Houses to Homes initiative, highlighting its successes and offering recommendations to build on the experience to date.
Headline figures
The headline news from the Welsh goverment perspective has been that its target of bringing 5000 long-term homes back into use during the term of the Parliament has already been exceeded long before the deadline of March 2016. The triumphant press release reported that 7,560 homes had been returned to use as a result of the scheme.
The evaluation report does not make much of this figure, because its focus is primarily on the Houses to Homes loans scheme, into which the Welsh Government has pumped £20million since 2012. According to the Sheffield Hallam figures, only 144 homes had been returned to use direcctly via the loans scheme at the point when the data was collected for the report. The huge difference between this and the 7560 claimed by the Welsh Government is explained by the two organisations meaning different things when they talk about "Houses to Homes", with the government figures referring to all the long-term empties brought back into use by local authorities via any mechanism since the inception of the scheme, rather than simply via the loans scheme itself.
There is a relationship between the two figures, however, as the Evaluation Report amply documents: the loans scheme provided renewed impetus to the work to tackle empty homes across the principality. In practice this meant greater outreach to empty homes owners, more resources devoted to empty homes work, more training for practitioners, and the development of regional organisations to share good practice and push the initiative along.
Taking a closer look
The Evaluation Report contains an excellent summary which should be required reading for engaged practitioners and which makes it unnecessary to rehearse all the salient points here. But some aspects require further analysis and interpretation.
Returning to the figures mentioned above, the Evaluation Report does identify a significant increase in successful outcomes arising from local authority interventions since the initiative was launched, with figures for the four years as follows:
2011-12 is the baseline, before the scheme was launched; most of 2012-13 was taken up with marketing the loans scheme, building organisational structures such as the regional forums, and recruiting and training staff, so that only in 2013-14 were the positive results flowing through.
The figures have to be regarded with a degree of caution because they rely on a poorly defined performance measure (PR04) which lacks the clear guidelines developed by EHN for the comparable indicator once used in England. Nevertheless, taken alongside strong anecdotal evidence from local authority officers, it seems reasonable to regard the improved outcomes as genuine and to attribute them to the Houses to Homes initiative in the broadest sense.
It is noteworthy that there was a significant reduction in the number of direct actions in 2014-15 compared with the previous year, although the number of successful outcomes was higher. Regrettably, based on the anecdotal evidence in the report, the reduction in direct actions by local authorities reflects the diminishing momentum of the scheme, another all-too-familiar stop-go story. One might expect to see a reduction in the number of successful outcomes in 2015-16 as a result. Fortunately, the continued availability of funding should ensure that this is a relative decline from a peak to a somewhat lower plateau rather than a total collapse. We can expect to see continuing benefits from the scheme.
The distance between the higher and the lower figures also helps highlight the value of developing a broad empty homes strategy rather than focusing solely on one measure like loans. The loans scheme provided the backbone for the overall initiative, but the positive outcomes (as in Kent) reflect a wider range of interventions. It is worth quoting the summary at some length on this subject as it is critical to understanding what it means to have a "strategy":
1.16 The majority of local authority empty homes officers attributed the increase in properties brought back into use each year since the launch of Houses into Homes to the introduction of the scheme. In total, 20 out of 22 officers reported that the scheme had resulted in an increase in the number of long-term empty homes brought back into use in 2014/15 as a result of direct actions by the local authority. In addition, 15 out of 22 officers reported that Houses into Homes had resulted in an increase in the total number of empty non-residential properties brought back into use as accommodation.
1.17 Regional coordinators and local authority officers were unequivocal in their view that Houses into Homes brought numerous benefits in its wake, which helped to increase the number of empty properties brought back into use. These were reported to include increased corporate commitment to tackling the problem of empty homes, increased staffing (particularly in the first two years of the scheme), an improvement in the effectiveness of local authority efforts to bring properties back into use, and support for efforts to address other strategic priorities (for example, increasing the supply of affordable housing, supporting regeneration initiatives and generating employment).
Rome not re-occupied in a day
The surprisingly low figure of 140 dwellings re-occupied since the scheme was launched is another indicator of the time lag between initiation and successful outcome. The report documents a much larger number of units where work was in progress. 360 (or 361 - the figure seems to vary) loans had been approved and another 82 were being processed by the end of March 2015. The lesson to be learned is that schemes such as this deserve plenty of time to bed in and produce results. Fortunately the Welsh Government has been clear about the merits of what it has been doing and has made further funding available (though on different terms). This more strategic approach compares favourably with the half-hearted stab at a strategy (if one can even dignifiy it with the term) in England over 2012-15.
One reason why the outcomes would take longer to arrive is that many of the projects involved properties that had been empty for a considerable period of time:
This said, the report does provide a fair amount of anecdotal evidence from both practitioners and owners/developers that some of loan procedures were too long-winded or staff insufficiently experienced or knowledgeable to deal with some scenarios that arose: there seems to be scope for improvements.
Flawed cost-benefit analysis
Among the more disappointing aspects of the Evaluation Report is the flawed cost-benefit analysis of the loan scheme. It discounts many of the successes based on the responses of the owners who said they would have proceeded anyway. This is probably fair enough. But it also ignores all the future benefits of the scheme through the recycling of the loans. As far as the cost-benefit analysis is concerned the loans are effectively grants. The report notes:
Finally the analysis has not discounted [sic] future benefits due to uncertainty about when benefits will accrue.[1]
Elsewhere in the report, the authors were prepared to extrapolate from one set figures to arrive at another, and there is ample evidence from Kent about, for example, the level of bad debts that could have been used to make projections for the Welsh scheme. This could have been on the basis that money was to be recycled into further loans, producing far more on the "benefits" side. As a bare minimum alternative, it would have been possible to estimate that x% of the loans would be repaid, allowing a net cost rather than a gross cost to be factored into the analysis, thereby significantly reducing the "cost" side of the equation.
In the absence of either of these calculations the cost-benefit analysis becomes largely meaningless in the real world of recycling loans. It would seem advisable for the Welsh Government to commission the small amount of extra research needed to produce more realistic figures that would take loan repayments and/or recycling of the loan fund into account, and in the process do the scheme justice.
You can access the full report via our information library here.
Some apects of the scheme are also discussed in EHN news stories about previous evaluation reports eg:
2011-12 is the baseline, before the scheme was launched; most of 2012-13 was taken up with marketing the loans scheme, building organisational structures such as the regional forums, and recruiting and training staff, so that only in 2013-14 were the positive results flowing through.
The figures have to be regarded with a degree of caution because they rely on a poorly defined performance measure (PR04) which lacks the clear guidelines developed by EHN for the comparable indicator once used in England. Nevertheless, taken alongside strong anecdotal evidence from local authority officers, it seems reasonable to regard the improved outcomes as genuine and to attribute them to the Houses to Homes initiative in the broadest sense.
It is noteworthy that there was a significant reduction in the number of direct actions in 2014-15 compared with the previous year, although the number of successful outcomes was higher. Regrettably, based on the anecdotal evidence in the report, the reduction in direct actions by local authorities reflects the diminishing momentum of the scheme, another all-too-familiar stop-go story. One might expect to see a reduction in the number of successful outcomes in 2015-16 as a result. Fortunately, the continued availability of funding should ensure that this is a relative decline from a peak to a somewhat lower plateau rather than a total collapse. We can expect to see continuing benefits from the scheme.
The distance between the higher and the lower figures also helps highlight the value of developing a broad empty homes strategy rather than focusing solely on one measure like loans. The loans scheme provided the backbone for the overall initiative, but the positive outcomes (as in Kent) reflect a wider range of interventions. It is worth quoting the summary at some length on this subject as it is critical to understanding what it means to have a "strategy":
1.16 The majority of local authority empty homes officers attributed the increase in properties brought back into use each year since the launch of Houses into Homes to the introduction of the scheme. In total, 20 out of 22 officers reported that the scheme had resulted in an increase in the number of long-term empty homes brought back into use in 2014/15 as a result of direct actions by the local authority. In addition, 15 out of 22 officers reported that Houses into Homes had resulted in an increase in the total number of empty non-residential properties brought back into use as accommodation.
1.17 Regional coordinators and local authority officers were unequivocal in their view that Houses into Homes brought numerous benefits in its wake, which helped to increase the number of empty properties brought back into use. These were reported to include increased corporate commitment to tackling the problem of empty homes, increased staffing (particularly in the first two years of the scheme), an improvement in the effectiveness of local authority efforts to bring properties back into use, and support for efforts to address other strategic priorities (for example, increasing the supply of affordable housing, supporting regeneration initiatives and generating employment).
Rome not re-occupied in a day
The surprisingly low figure of 140 dwellings re-occupied since the scheme was launched is another indicator of the time lag between initiation and successful outcome. The report documents a much larger number of units where work was in progress. 360 (or 361 - the figure seems to vary) loans had been approved and another 82 were being processed by the end of March 2015. The lesson to be learned is that schemes such as this deserve plenty of time to bed in and produce results. Fortunately the Welsh Government has been clear about the merits of what it has been doing and has made further funding available (though on different terms). This more strategic approach compares favourably with the half-hearted stab at a strategy (if one can even dignifiy it with the term) in England over 2012-15.
One reason why the outcomes would take longer to arrive is that many of the projects involved properties that had been empty for a considerable period of time:
This said, the report does provide a fair amount of anecdotal evidence from both practitioners and owners/developers that some of loan procedures were too long-winded or staff insufficiently experienced or knowledgeable to deal with some scenarios that arose: there seems to be scope for improvements.
Flawed cost-benefit analysis
Among the more disappointing aspects of the Evaluation Report is the flawed cost-benefit analysis of the loan scheme. It discounts many of the successes based on the responses of the owners who said they would have proceeded anyway. This is probably fair enough. But it also ignores all the future benefits of the scheme through the recycling of the loans. As far as the cost-benefit analysis is concerned the loans are effectively grants. The report notes:
Finally the analysis has not discounted [sic] future benefits due to uncertainty about when benefits will accrue.[1]
Elsewhere in the report, the authors were prepared to extrapolate from one set figures to arrive at another, and there is ample evidence from Kent about, for example, the level of bad debts that could have been used to make projections for the Welsh scheme. This could have been on the basis that money was to be recycled into further loans, producing far more on the "benefits" side. As a bare minimum alternative, it would have been possible to estimate that x% of the loans would be repaid, allowing a net cost rather than a gross cost to be factored into the analysis, thereby significantly reducing the "cost" side of the equation.
In the absence of either of these calculations the cost-benefit analysis becomes largely meaningless in the real world of recycling loans. It would seem advisable for the Welsh Government to commission the small amount of extra research needed to produce more realistic figures that would take loan repayments and/or recycling of the loan fund into account, and in the process do the scheme justice.
You can access the full report via our information library here.
Some apects of the scheme are also discussed in EHN news stories about previous evaluation reports eg:
- http://www.ehnetwork.org.uk/newsitem/houses-homes-gives-impetus-welsh-authorities
- http://www.ehnetwork.org.uk/newsitem/welsh-houses-homes-initiative-latest-update
- http://www.ehnetwork.org.uk/newsitem/wales-loan-scheme-forges-ahead
- http://www.ehnetwork.org.uk/newsitem/wales-leads-way-national-empty-homes-initiative