7. Appraisal-Based Methods
- Statistical Office of the European Communities;International Labour Office;International Monetary Fund;Organization for Economic Co-operation and Development;United Nations;World Bank
- Published Date:
- May 2013
7.1 As was mentioned in previous chapters, the matched model methodology to construct price indices, where prices of identical items are compared over time, cannot be applied in the housing context. One of the reasons is the low incidence of re-sales and the resulting change in the composition of the properties sold. The repeat sales method, which was discussed in Chapter 6, attempts to deal with the quality mix problem by looking at properties that were sold more than once over the sample period. However, using only repeat-sales data could be very inefficient since all single sales observations are “thrown out” and could also lead to sample selection bias.
7.2 In several countries information on assessed values or appraisals of properties is available, which might be useful as proxies for selling prices or, more generally, market values. In countries where they have been collected for tax purposes, appraisals will typically be available for all properties at a particular reference period. In a number of studies assessed values were used in addition to sale prices in a repeat sales framework to reduce the problem of inefficiency and the potential problem of sample selection bias. For example, Gatzlaff and Ling (1994) used sale prices as the first measure and appraisals as the second measure in a repeat “sales” regression. Clapp and Giaccotto (1998) did the reverse and used appraisals as the first and selling prices as the second measure. Both studies found that these methods produced price indices similar to a standard repeat sales index.
7.3 The above assessed-values repeat sales methods are based on pseudo price relatives in which the appraised values may be derived from different periods. But when assessed values for all properties are available that do relate to a single valuation period or reference date, then it will be possible to use the standard matched model methodology. For each property sold in some comparison period for which we have a sale price, a base period “price” - the assessed value - is now available also. Price relatives with a common base period - the valuation period - can then be constructed, and these sale price appraisal ratios can be aggregated using a standard index number formula, though some re-scaling may be required.
7.4 The use of a conventional matched model index number formula simplifies the computation of the index because there is no need to use econometric techniques to estimate the index or to adjust for compositional change, as is the case with hedonic and repeat sales methods; see Chapters 5 and 6. Another feature of the sale price appraisal ratio method (SPAR) method discussed in the present chapter is that it is free from revisions because there is no modeling and pooling of data involved. Thus, in contrast to the repeat sales method and the multiperiod time dummy hedonic method, previously computed price indices are not re-estimated when new sales data become available.
7.5 The SPAR method has been used in New Zealand since the early 1960s and is currently also used in several European countries, notably in Denmark, the Netherlands and Sweden. Given that a few countries around the world are actually using the SPAR method, it is not surprising that there is only a small though expanding literature available. It would appear that Bourassa, Hoesli and Sun (2006) were the first to publish a paper on this method. According to them, “the advantages and the relatively limited drawbacks of the SPAR method make it an ideal candidate for use by government agencies in developing house price indices”. Rossini and Kershaw (2006) found that the SPAR method outperformed several other methods in terms of reduced volatility of weekly index numbers. De Vries et al. (2009) reported a higher precision of monthly SPAR indices for the Netherlands compared with monthly repeat sales indices. Shi, Young and Hargreaves (2009) compared SPAR and repeat sales indices for New Zealand and found a rather low correlation on a monthly basis.
7.6 When the properties are reassessed and new appraisal data become available, the SPAR index can, and probably should be, rebased. A long-term index series is obtained by “splicing” the existing and new series. Properties in the Netherlands are currently being re-valued each year, which makes it possible to construct an annually chained RPPI, where the valuation period (which is January) serves as the link month. Shi, Young and Hargreaves (2009) argued that bias could arise from frequent reassessments. De Vries et al. (2009) did not find any chain-link bias but observed that the standard error of the chained SPAR index increases each time new appraisals are introduced because an additional source of sampling error is added.
The SPAR Method in Detail
7.7 Suppose that we have samples of properties sold at our disposal for the starting or base period 0 and for comparison periods t (t = 1,…,T). As in earlier chapters, the samples will be denoted by S(0) and S(t). In each period we know the sale prices of all sampled properties; the price of property n in period t is represented by ptn. As mentioned before, houses that were sold in period t were generally not sold in period 0, so there is a lack of matching. However, suppose that assessed values or appraisals are available for all properties in the housing stock, and that they relate to a single valuation period. The valuation period will serve as the base period, and the appraisal for property n will be denoted by a0n. Thus, for each property belonging to the period t sample S(t) we know both the period t selling price ptn and the base period assessed value a0n. In other words, for all n ∈ S(t) we can establish a price relative - a sale price appraisal ratio - ptn / a0n, which can be used in a matched model framework to compute an RPPI.
7.8 Although it would be possible to construct geometric appraisal-based indices, we will focus here on arithmetic indices as these seem to be more appropriate in the housing context. The arithmetic appraisal-based index can be defined as
Expression (7.1) describes a Paasche-type index because we are using the comparison period sample S(t) in both the numerator and the denominator. The quantities are equal to 1 as every property is basically a unique good. The construction of a Laspeyres-type price index would be problematic or even impossible: period t price information for dwelling units belonging to the base period sample S(0) is only available for those few units, if any, that were resold in period t. This means that the construction of a Fisher-type index will not be feasible either. As shown by the second expression, (7.1) can be written as a value-weighted average of the sale price appraisal ratios ptn/ a0n, where the weights
7.9 The appraisal-based Paasche-type index,
7.10 The appraisal-based price index (7.1) does not make use of the observed selling prices in the base period. As a result, the index will differ from 1 in the base period, which is problematic. However, this problem can easily be resolved by normalizing the indices by dividing them by the base period value. We then obtain the following arithmetic SPAR index:
where N(0) and N(t) denote the number of properties sold in periods 0 and t (the respective sample sizes).
7.11 The second expression on the right-hand side of (7.2) writes the SPAR index as the product of the ratio of sample means and a bracketed factor. Since the SPAR method is a matched model method (with respect to periods 0 and t), the bracketed factor adjusts the ratio of sample means for compositional changes occurring between each period t and the base period 0. So, while short-term volatility is likely to be present due to period to period mix changes, the SPAR method is expected to exhibit much less volatility than the ratio of sample means.
7.12 The arithmetic SPAR index can be interpreted as a proxy for a sales based Paasche RPPI. (1) But many countries, including EU member states, are typically aiming at a Laspeyres index rather than a Paasche index. Stratification could be used as a means to approximate this target index while using the SPAR method. The SPAR (Paasche) indices at the stratum level will then be aggregated using base period expenditure share weights to obtain the overall “Laspeyres-type” index. The RPPI in the Netherlands is an example of such a stratified SPAR approach, where region and type of house are used as stratification variables. The index is compiled monthly and published jointly with the Dutch Land Registry Office. Stratification might also help to account for any systematic differences between appraisals and market values across regions or different types of houses (de Vries et al., 2009; de Haan, van der Wal and de Vries, 2009).
7.13 The SPAR index can alternatively be interpreted as a sample estimator of a stock RPPI. If in each period the properties sold are viewed as random samples from the base period housing stock, then the SPAR index is an estimator of the Laspeyres stock RPPI. Properties sold that were added to the stock after the base period should in this case be excluded. (2) As mentioned in earlier chapters, the sample of houses sold may not be representative of the total stock so that sample selection bias could arise. Stratification will again be a helpful tool to mitigate this problem.
Methodological and Practical Issues
7.14 Since the appraisals relate to the base period, in general the properties will have been valued at their base period characteristics. But for the SPAR index (7.1) to be a constant quality price index, the appraisals should be evaluated at characteristics of the comparison period. Thus, if housing characteristics change over time, the SPAR method will not adjust for those changes, similar to the repeat sales method. This is an important drawback.
7.15 Yet in practice there could be some implicit adjustment for quality changes. In the case of the New Zealand SPAR index, Bourassa, Hoesli and Sun (2006) note: “the base appraisal is adjusted for subsequent improvements to the property that require a building permit”. If this is done in real time, adjustments for major quality improvements will indeed be made. However, apart from the fact that not all property improvements require a building permit, it is unlikely that these adjustments adequately deal with the net effect of improvements and depreciation of the structures.
7.16 In the Netherlands there may also be some implicit quality adjustment in the SPAR index. The assessments are typically carried out some time after the appraisal reference month and may take into account major improvements to the properties. Furthermore, as mentioned above, the assessments are nowadays performed every year. Annual chaining by itself could alleviate the problem of quality change if the updated appraisals properly account for changes in the characteristics. Of course this will depend on the exact way the properties are valued, which may not be known to the index compilers.
Quality of the Assessment Information
7.17 Abstracting now from quality adjustment issues, the SPAR method is obviously dependent on the quality of the assessment information. There are three broad ways in which assessments of (non-traded) properties can be carried out: by using hedonic regression, by comparing them to similar traded properties, and by expert judgment. The methods used differ among countries and sometimes even within a particular country. In various countries, private companies are engaged in mass appraisal. Although the details of the methods used are often not publicly available, some of those companies appear to combine hedonic regression with local market information or expert judgment.
7.18Bourassa, Hoesli and Sun (2006) noted that the appraisals in New Zealand are derived from hedonic regressions, but unfortunately they did not present the exact method. In Chapter 5 it was explained that there are different hedonic approaches and that the predicted prices - in this case the appraisals - depend on the type of data used and the number of observations, the specified functional form, the variables included and other choices made. Thus, even though hedonic regression is the least arbitrary of the three assessment methods mentioned above, there can still be a lot of uncertainty and error involved, which has an unknown impact on the sale price appraisal ratios and the resulting SPAR index.
7.19 The use of comparable properties seems to be widespread. Chinloy, Cho and Megbolugbe (1997) compared a sample of U.S. private sector appraisals to selling prices. They suspected that the reliance on a relatively small number of comparable houses leads to more volatility than can be observed in market-wide selling prices. More importantly perhaps, they found that appraisals exceeded sale prices in approximately 60 percent of the cases, leading to an average upward bias of two percent.
7.20 In countries where official assessments are designed for property taxation purposes, like in the Netherlands, the assessed values may not to be too far off the mark since the government has an incentive to make the assessments as large as possible in order to maximize tax revenue while taxpayers have the opposite incentive to have the assessments as small as possible. In the Netherlands the municipalities are responsible for making the assessments. The methods used differ across the municipalities. Some of them, for example the capital city of Amsterdam, use the comparable house method whereas others apparently use some kind of hedonic regression method. De Vries et al. (2009) argued that Dutch authorities may in fact have an incentive to make the assessments not too high to avoid court procedures because households who feel the appraised value is too high can lodge an appeal.
7.21 The advantage of the SPAR method as compared to hedonic regression methods is that information on only a few property characteristics is needed: assessed values (relating to a common reference period), possibly some stratification variables, and addresses to merge the data files if the selling prices and appraisals come from different sources. In the Netherlands, for example, transaction prices and a limited number of stratification variables are recorded by the Land Registry whereas the appraisals are from a second administrative data source. It is well known that merging data files by address can be difficult, although in the Netherlands this does not seem to be a major issue.
7.22 Data cleaning is another important practical issue. The SPAR method is dependent on the quality of the appraisals. Some of the sale price appraisal ratios might be found implausible, perhaps because the appraisals are deemed “wrong”, and deleted from the data set. (3) Deleting erroneous observations, such as obvious entry errors, is good practice. A cautious approach is called for, however, as deleting price relatives can lead to biased results. At least a rule for deleting outliers should be explicitly formulated to inform users.
A Regression-Based Imputation Interpretation
7.23 In this section we will show that the SPAR method is essentially an imputations approach in which the “missing” base period prices are estimated from a linear regression of selling prices on appraisals. Recall first that the base period prices of the properties belonging to the period t sample S(t) cannot be observed directly since those properties were generally not traded in period 0. We can try to estimate the “missing” prices to obtain the imputation Paasche price index
7.24 The imputed value
7.25 Consider the following two-variable regression model for the base period:
Equation (7.4) is a simple descriptive model where selling prices are regressed on appraisals. We assume that this model is estimated by Ordinary Least Squares (OLS) on the data of the base period sample S(0). The predicted prices for n ∈ S(0) are
7.26 For convenience we first rewrite (7.3) as
In the second step of (7.6) we have used
Notice that if
7.27 The third assumption is that equation (7.5) holds for n ∈ S(t): the linear relationship between base period selling prices and appraisals postulated and estimated for the properties actually sold during the valuation or base period 0 (for n ∈ S(0)) is assumed to hold also for properties that were not sold. But this is a very restrictive assumption. While the linear relation can be tested for n ∈ S(0), (5) it would be difficult if not impossible to test it for n ∈ S(t) as the selling prices are “missing”. The presence of appraisal bias, in the sense that the appraisals over- or underestimate the unknown market values (the prices at which the properties would have been sold), can bias the SPAR index. Bias in the SPAR index will particularly arise if the “true” value of β1 for n ∈ S(t) would be very different from β1 for n ∈ S(0).
7.28 In this section we focused on the SPAR index as a sales RPPI. A related approach, where the appraisals serve as auxiliary information in a “generalized regression” (GREG) framework in order to estimate a stock based RPPI, was described by de Haan (2010b). The GREG method uses population information on the appraisals instead of sample information. He showed that the SPAR index is a straightforward estimator of the GREG stock based index which, when applied to Dutch data, turned out to be almost as efficient.
Main Advantages and Disadvantages
7.29 The merits of the SPAR method are listed below. The main advantages are:
The SPAR method is essentially based on the standard matched model methodology and links up with traditional index number theory.
The method is computationally simple.
Information on housing characteristics is not required in order to implement this method; the only information required is data on sale prices and appraisals. In some countries the data is available from administrative sources such as the land registry, and usually covers all transactions (for resold properties).
This method uses much more data than the repeat sales method and hence there are fewer problems due to sparse data. In particular, sample selection bias is likely to be smaller. Also, the SPAR method does not suffer from revision of previously calculated figures when new data becomes available.
Conditional on the data cleaning rules, the SPAR method is reproducible.
7.30 The main disadvantages of the SPAR method are:
The method cannot deal adequately with quality changes (major repairs or renovations and depreciation) of the dwelling units. (6)
The SPAR method is dependent on the quality of the base period assessment information. The exact way the valuations are carried out may not always be clear and has an unknown impact on the results.
The method cannot decompose the overall property price index into land and structures components. (7)
An Example on Data for the Town of “A”
7.31 Using the data set for the town of “A”, which was described in Chapter 4, a SPAR index was computed. Recall that this data set contained sales of detached houses for 14 quarters, starting in the first quarter of 2005 and ending in the second quarter of 2008. After some data cleaning –in particular deleting houses that were older than 50 years at the time of sale were - a total of 2289 sales remained.
7.32 To compute SPAR index numbers we also need assessed values for the properties sold. Our appraisal data relate to the first quarter (i.e., January) of 2005. Matching the sales data set and the appraisal data set was quite successful; 99.3 % of the selling prices could be matched with the corresponding appraisals; i.e. for only 15 observations we could not find an appraisal, so these were deleted. The resulting SPAR index, PSPAR, is plotted in Figure 7.1 and listed in Table 7.1, along with the hedonic imputation Fisher index, PHIF, described in Chapter 5, and the repeat sales index, PRS, estimated in Chapter 6. The trend of PSPAR* is very similar to that of PHIF, but PSPAR is slightly more volatile.
Figure 7.1.SPAR Index, Hedonic Imputation Fisher Price Index and Repeat Sales Index
Source: Authors’ calculations based on data from the Dutch Land Registry
7.33 A potential drawback of the SPAR method is that is entirely dependent on the accuracy of the appraisal data. An inspection of the distribution of the sale price appraisal ratios indicated a number of big outliers. Specifically, there were several observations with very high sale price appraisal ratios (up to 10.5), in most instances as a result of unusually low appraised values. It is most likely that a significant proportion of these outliers were recording errors. Hence, we decided to delete the biggest outliers. Following Statistics Netherlands data cleaning methods at the time, based on the distribution of the natural logarithm of the sale price appraisal ratios, 26 observations were removed for which the log of price ratio differed more than 5 standard deviations from the mean. (8) We ended up with 2248 observations.
7.34 The improved SPAR index, labeled PSPAR*, computed on the cleaned data set is also shown in Figure 7.1 and Table 7.1. As can be seen, cleaning of the data had a substantial impact on the result: PSPAR* is much less volatile than the index PSPAR that was computed on the initial data set. The trend was also affected: PSPA*, is generally lower than PSRAR due to the fact that most of the deleted observations had unusually high sale price appraisal ratios. Figure 7.1 confirms that - using a relatively small data set which covers a short time period - the SPAR method generates more credible results than the standard repeat sales method, especially after cleaning the data.
7.35 A comparison of PSPAR*, with the hedonic imputation Fisher index PHIF reveals that in several periods, for example in the last four quarters, the price changes according to the two methods are in opposite directions. Also, PSPAR* is generally lower than PHIF;; at the end of the sample period, in quarter 14, the difference amounts to 0.026 index points. At first sight this seems to suggest that PSPAR*, has a downward bias. However, a difference of the same magnitude (0.027 points) is already found in quarter 2. So if we had normalized both series to equal 1 in quarter 2, the two methods would have produced approximately the same index value in quarter 14. This is an illustration of a general starting problem encountered when comparing volatile time series: the choice of starting or base period affects the average difference during the sample period.
Administrative data sets, particularly those from the land registry, typically contain all sales (excluding newly-built properties) in each period. From a sales point of view there is no sampling involved. In this interpretation, the SPAR index has no sampling error, but it does have error due to the use of appraisals, which are estimates of the “true” market values.
It may seem that properties which are new to the stock cannot even be used because the necessary appraisals are lacking. However, this depends on the appraisal system. If former rental houses have been sold and are thus added to the stock of owner occupied housing, then they will have a base period appraisal value if rental houses are also assessed. Moreover, if property taxes are uniformly based on period 0 valuations for a number of years, then the authorities would need those values for newly built houses as well. The difficulty is of course that the authorities would have to “invent”an assessed value for a new house in period 0, even if it did not exist in that period. Such assessments might be problematic and hence should probably be excluded from the computation of the index.
The example for the town of “A” at the end of this chapter shows that the removal of a relatively low number of outliers can have a substantial effect on the SPAR index
If the selling prices would be used as official valuations, then of course the values 0 and 1 would exactly hold and we would find a perfect fit of (7.4) to the period 0 data.
Van der Wal, ter Steege and Kroese (2006) and de Vries et al. (2009) compared Dutch government appraisals to selling prices. In the latter study the linear relationship (7.4) was explicitly tested (for the properties traded in the valuation month) for various valuation months. It turned out that the constant term was indeed very small and that the slope coefficient did not significantly differ from 1
In countries where the assessments provide separate information on the value of the structures and the value of the land, the SPAR index could in principle be adjusted by using exogenous information on the net depreciation of houses of the type being considered
If fresh property assessment information appeared every month or quarter, this information could be used to form separate price indices for both land and structures, provided that the assessments decomposed the total assessed value of the property into land and structures components. Unfortunately, official assessments generally are made only once a year or once every few years. This low frequency information could however be used to check the land and structures price indices generated by hedonic regression methods.
As a first step in the data cleaning procedure, Statistics Netherlands removed all properties with selling prices or appraisals below 10 000 or above 5 000 000 Euros. In our data set, however, there were no such properties. Note that Statistics Netherlands recently changed the outlier detection and removal procedures.