Missing the Mark: Mortgage Valuation Accuracy and Credit Modeling

Abstract

In 2008, the US mortgage market collapsed under a lack of transparency, incorrect pricing, and underestimated risk. Price indexes, however, could help investment managers monitor assets that are heterogeneous or infrequently traded. Responding to needs for better valuation approaches, we created new localized house price indexes and evaluated their ability to predict transaction prices and mortgage performance. We show where and when valuation errors occur and how to avoid them. Our work has a broader application than mortgage valuation for analysts or investors valuing alternative assets—namely, using the most granular indexes yields positive but diminishing modeling gains when submarket trends exist.

Disclosure: The authors report no conflicts of interest.

Editor’s Note

This article was externally reviewed using our double-blind peer-review process. When the article was accepted for publication, the authors thanked the reviewers in their acknowledgments. Jonatan Groba and Dan Scholz, CFA, were the reviewers for this article.

Submitted 25 November 2017

Accepted 26 July 2018 by Stephen J. Brown

Acknowledgments

The authors are grateful for useful comments provided by Financial Analysts Journal editors, Fred Graham, Jonatan Groba, Andy Leventis, Dan Scholz, CFA, Scott Smith, and participants at conferences held by the American Enterprise Institute, American Real Estate and Urban Economics Association, George Washington University, Homer Hoyt Institute, and the National Association of Realtors. The analysis and conclusions are those of the authors alone and should not be represented or interpreted as conveying an official FHFA analysis, opinion, or endorsement. An earlier version of this paper, FHFA Working Paper 16-04, was selected in 2016 as best paper in real estate market analysis by the American Real Estate Society.

Notes

¹ As a referee pointed out, alternative assets may have other properties (being perishable and collectible and having liquidity issues and storage costs) that make drawing a perfect comparison of them with real estate difficult. In both types of assets, however, the commonalities of heterogeneity and infrequent trading can be overcome by pairing up repeat-sales of the same good to track a constant-quality price appreciation. Further distinctions between properties can usually be addressed with a control variable or sample stratification, which is what we explore in this article.

² The repeat-sales statistical method has been a standard technique (for several decades) to create aggregated house price indexes. The real estate sector offers a practical opportunity to study how a price index’s degree of aggregation affects its ability to predict an individual asset’s observed price, return, and performance. Price dynamics can exhibit dramatic temporal and regional variations that are masked when aggregate indicators are used. Consequently, aggregation bias may have an enormous impact on individual properties purchased by developers and on pools of mortgages where the value and performance affect cash flows paid out to investors of asset-backed securities. The contrast between valuing an individual unit and valuing several units as a group is not unique to the field of real estate; it also exists in equity markets, where prices update frequently and investments may be in individual companies or collections of companies aggregated into a fund.

³ Submarket stratification and location-specific prices exist in virtually all sectors with transportation costs, including energy, minerals, and agricultural production. The same contrast could have been done by Dimson et al. (2015) because even in geographically proximate areas, such as Sonoma Valley and Napa, similar wine grapes and maturities can still have different valuations and risks. For that reason, wineries often have estates in both regions and may blend grapes from the regions.

⁴ Perhaps counter to some expectations, the most granular index is not always the best choice. Broadly, we find the optimal degree of granularity depends on a market’s size and heterogeneity. For smaller markets, which typically have more homogeneous products (houses), a less granular index would suffice. For larger markets, where heterogeneity across classes (neighborhoods) and product types (differences in housing stock) leads to distinct trends, a more granular index is warranted. This is one of the first papers to show such results across the entire nation and to raise concerns about the within-city geographic distribution of mortgage assets. Both aspects have implications for strategic asset allocation decisions to appropriately balance a portfolio, whether it be heavily concentrated in housing or if it is one of several allocation types.

⁵ We provide statistical exercises to show how one might evaluate whether increased data availability could be helpful for financial analysis (we raise granularity concerns, but the importance of longer and reliable international series was voiced years ago by Stevenson 2000). Through discussions with market participants and our own work, we are aware that domestic and international investors (e.g., asset managers, hedge funds, insurance companies, banks, sovereign funds, and REITs) take similar decision-making approaches when investigating the gains from additional data layers.

⁶ Ours is certainly not the first study to investigate whether local real estate measures could offer better model fit. For example, Nagaraja, Brown, and Wachter (2014) included ZIP code random effects in city-level statistical estimations and found that they significantly reduce out-of-sample model error. These random effects effectively represent intercept shifts. They do not allow for differential growth rates across ZIP codes but within cities. The benefit of those controls confirms that real estate markets are often extremely local in nature. As another example, Campbell, Giglio, and Pathak (2011) showed that forced sales can have different impacts on different neighborhoods (they depressed the values of nearby homes in lower-income areas but not in higher-income locations). This result shows that, again, an intercept shift for areas within cities is not enough for optimizing model fit because appreciation rates or changes may not follow the same patterns throughout a housing cycle. In our article, instead of introducing a time-invariant control, we actually model at the local levels to allow both intercepts and growth rates to vary.

⁷ For example, assume an index has a value of 110 in 1998, 121 in 1999, and 190 in 2007. The cumulative price appreciation from 1998 to 2007 is [(190 – 110)/110]100% = 73%, with an annualized rate over those nine years of [(1 + 0.73)^(1/9) – 1]100% = 6.3%. The immediate average annual price change for that market from 1998 to 1999 is [(121 – 110)/110]100% = 10%. A series of frequently asked questions and answers is online at www.fhfa.gov/PolicyProgramsResearch/Research/PaperDocuments/bdl_faqs_local_hpis.pdf. Additional information covers the indexes, data source, missing values, interpreting index values, why they start in different periods, large and small values, adjusting to real terms, computing price appreciation with the indexes, smoothing, transaction dates, merging with demographic data, and an interactive mapping tool.

⁸ The label ZIP3 (3-digit ZIP code) refers to the first three numbers of the postal code, and ZIP5, to the five-digit code. For example, 32308 would belong to the ZIP3 of 323. Historically, the first digit has usually identified a group of states, and the next two digits represent a subregion within the group. The CBSA includes both metropolitan statistical areas and micropolitan statistical areas and is defined by the US Office of Management and Budget on the basis of data from the US Census Bureau.

⁹ www.fhfa.gov/DataTools/Downloads/Pages/House-Price-Index-Datasets.aspx .

¹⁰ The underlying data cover about 70% of the housing market and go back to the 1970s, forming probably the most comprehensive mortgage database available, albeit not entirely for public use. In 2013, Fannie Mae and Freddie Mac released loan-level data to track originations and ongoing monthly performance activity. Our house price indexes were coupled with those sources to update marked-to-market asset pricing and perform portfolio analyses. The indexes have been released for free as a public good, and we aspire to update them each calendar year. Already, the data have been mentioned in work on the geographic flow of bank funding, access to credit, affordability, urban renewal, and health care, as well as used for a Gallup prediction poll during the last presidential election. We realize that the Financial Analysts Journal has a large investor- and/or finance-focused audience, but our framework and data also offer ways (many of which we did not anticipate) to advance research and policy across other academic disciplines.

¹¹ A “half-pair” is an alternative count measure for a repeat-sales index. As opposed to paired properties, a half-pair is the transaction count where either the first or the second sale occurs in the given period.

¹² Recall that the mean square error is the sum of the estimator’s bias and variance. We expected the first term to be greater for large areas (where estimates of individual properties might be measured with less precision because of the greater degree of appreciation), and we expected the second term to be greater in small areas (where fewer observations lead to more noise and volatility in estimates).

¹³ F-tests with a null hypothesis of equality between the ZIP5 index and the next best index were rejected at the 0.01 level of significance for all holding periods ranging from 1.4 million observations in Year 1 to 350,000 in Year 8.

¹⁴ The CBD was calculated as the maximum value within the CBSA of the inverse of the standardized land area plus the share of housing units in 20+-unit structures. Land area data are from the TIGER/Line shapefiles. The TIGER/Line shapefiles and related database files are an extract of selected geographic and cartographic information from the US Census Bureau’s Master Address File/Topologically Integrated Geographic Encoding and Referencing Database. Structure type is from the 1990 Decennial Census, the earliest census for which ZIP code data are available. Distance to the CBD was calculated “as the crow flies,” or as straight-line distance. An area, such as a ZIP code, belonged to a particular region if its centroid was located in that concentric circle.

¹⁵ By arbitrage, we mean that certain areas of cities have observable and distinctive gains that can accumulate over time. Our indexes are able to point to where appreciation rates are greatest, which could allow for targeted asset investment. As one of the anonymous reviewers pointed out, arbitrage can also occur when an asset is potentially under-/overpriced, and our indexes could test whether mispricing occurs systematically for certain groups of borrowers and what the implications are for asset performance—as, for example, in prepayment speeds or default likelihoods. This latter interpretation is left for future research.

¹⁶ The differences in RMSEs may seem small in magnitude, but our models were calibrated on a nationwide portfolio; the economic significance may not always be negligible. As an example, consider a security with a reference pool of around 150,000 loans with an average principal balance of $240,000 and an LTV ratio of 75; using a ZIP code–level index (as opposed to a city-level index) could lead to a difference in aggregate valuation of approximately $19 million.

¹⁷ This kind of factor analysis extends beyond real estate; it has become the vogue recently in smart alpha investment strategies.

¹⁸ Public loan origination and performance data are becoming increasingly available. Both Freddie Mac and Fannie Mae provide single-family loan-level data going back to 2000 for more than 40 million loans. Freddie Mac data are available at www.freddiemac.com/research/datasets/sf_loanlevel_dataset.html. Fannie Mae data can be found at www.fanniemae.com/portal/funding-the-market/data/loan-performance-data.html. The public datasets do not have enough information, however, to reliably identify the property location of individual loans. We obtained a proprietary identifier from Freddie Mac that allowed us to match mortgages by physical address and map information on property price transactions onto loan performance data.

¹⁹ Default is defined as D180, meaning that a loan had a payment delinquency for at least 180 days.

²⁰ A larger difference between actual and predicted rates in center-city regions of large cities should not be surprising to practitioners. In such places, price gradients are steeper and increase at a faster rate than in areas farther from the CBD. When a single index (such as one at a national level) is used, the estimates will tend to underestimate price levels in center-cities and overestimate levels in outskirts. More granular indexes reduce these errors. Borrowers will realize, too, that an asset’s valuation is much higher (or CLTV is lower) and will be less likely to exercise the put option and default than may be predicted by a national index. Similarly, because center-city areas appreciate at faster rates, refinancing may be more prevalent as borrowers repay to “lock in” favorable rates when leverage is in their favor.

²¹ An odds ratio estimate greater than 1.0 indicates that the probability of outcome i occurring (i.e., prepayment or default) increases with a one-unit change in the predictor variable.

²² The statistic is the ratio of the RMSE of the national index to the RMSE for another level of index granularity. Recall that a chi-squared distribution is the sum of squares of statistically independent Gaussian variables and that the ratio of two chi-squared distributions is an F-distribution. We used the sample size of each group as the degree of freedom, and critical p-values were calculated separately for each panel in Figure 3 but were the same for each choice of index granularity within a panel. For example, the critical values for ratios in the center-city were 1.0375 for the 10% level of significance, 1.0484 for the 5% level, and 1.0691 for the 1% level.