- Statistical Office of the European Communities;International Labour Office;International Monetary Fund;Organization for Economic Co-operation and Development;United Nations;World Bank
- Published Date:
- May 2013
12.1 This handbook provides detailed and comprehensive information on the compilation of residential property price indices (RPPIs). It provides an overview of the conceptual and theoretical issues that arise, explains the different user needs for such indices and gives advice on how to deal with the practical problems that statistical offices are confronted with in the construction of such indices. Earlier chapters cover all relevant topics including: a description of the different practices currently in use; advice on the alternative methodologies available to the compiler; and the advantages and disadvantages of each alternative. The purpose of this chapter is to draw together all this information and make recommendations on best practice for compiling residential property price indices, including how to improve international comparability. The recommendations necessarily take into account the different situations countries are confronted with in terms of data availability and therefore cannot be too prescriptive.
12.2 Users of RPPIs are also catered for. The handbook provides information not only on the different methods that are and can be deployed in compiling such indices, but also on the statistical limitations of what is being measured. Users will want to bear the latter in mind so that the results of an index can be interpreted correctly. Any set of recommendations has to start with an understanding of the basic concept underlying the target index, in other words what a residential property price index is trying to measure. This will, of course, depend on user needs and the purpose of the index.
12.3 The recommendations given below follow the same order as Chapters 3 to 8. Chapter 3 describes the main elements of a conceptual framework for RPPIs, and Chapters 4 to 8 describe the main statistical methods that can be used in constructing such indices. The different methods essentially relate to alternative solutions to the problem of quality change, that is, how to adjust an RPPI for changes in the quality mix of the properties sold and for quality changes (the net effect of renovations, extensions and depreciation) of the individual dwellings.
Target and Conceptual Basis
12.4 In principle, the target index, in other words the type of index to be compiled, will depend on its purpose. The System of National Accounts 2008 should be used as the conceptual framework for RPPIs.
12.5 A price index which is required to measure the wealth associated with the ownership of residential property should be stock-weighted. A stock-weighted index is also appropriate for a financial stability indicator, in particular for an index which is being used to identify property price bubbles.
12.6 A price index which is required for measuring the real output of the residential real estate construction industry should be sales-weighted. A sales-weighted index is also appropriate for a consumer price index (CPI) that follows an acquisitions approach.
12.7 A price index which is required to measure the wealth associated with the ownership of residential property should cover all residential property, that is, both existing properties and properties which have been recently built. (1) This is also the case for an index used as a financial stability indicator.
12.8 A price index which is required for measuring real investment in the residential real estate industry should cover sales of new property. (2) The construction part of new housing produced is part of gross investment. The cost of the land, apart from the value of any improvements made to this element, should be excluded for this purpose. However, as was explained in Chapter 3, a price index for the sales of both new and existing houses is required in order to construct real output measures for the activities of real estate agents in selling new and existing houses to purchasers. The scope of the index for this application should cover both the structure and land values of the residential property sales.
12.9 A price index restricted to new properties is also appropriate when a residential property price index is an input into a CPI for the measurement of owner-occupier housing costs on a net-acquisition cost basis, that is, where the CPI covers the cost of acquiring properties which are new to the owner-occupier housing market. This approach, one of a number of alternatives as was explained in Chapter 3, treats the purchase of a dwelling exactly like the purchases of any other consumption good. (3)
12.10 Regardless of the different uses of the index, the purpose of a residential property price index is to compare the values of the sales or of the stock of residential property between two time periods after allowing for changes in the attributes of the properties. For this purpose it is necessary to decompose price changes into those associated with changes in attributes and the residual which relates to the underlying “pure price” change.
12.11 A constant quality price index is appropriate for both a stock and sales-weighted price index. There are a number of practical methodologies which can be used to construct such an index. Recommendations on which of the available methods should be used in which circumstances are provided below.
Decomposition between the Building and Land Components
12.12 A decomposition of the RPPI in structures and land components may be required, particularly if a country’s balance sheet estimates of national wealth in the National Accounts make this distinction. Such decomposition may also be necessary when a residential property price index is an input into a CPI for the measurement of owner-occupier housing using the net-acquisition approach.
Statistical Methods for Compiling Constant Quality Indices
12.13 The methods adopted by statistical agencies to construct constant-quality RPPIs vary among countries and are dictated in large part by the availability of data generated by the processes involved in buying and selling a property. The challenges of compiling constant-quality residential price indices can be summarized by the following three factors:
Residential properties are notoriously heterogeneous. No two properties are identical.
Prices are often negotiated. The (asking) price of a property is not fixed and can change throughout the transaction process until the price is finalised. This means that a property’s market value can only be known with certainty after it has been sold. (4)
Property sales are infrequent. In many countries, less than ten per cent of the housing stock changes hands every year, which means that a particular house is likely to be resold approximately once every ten years.
12.14 The different methods of index construction used by a statistical agency reflect the differing solutions used to meet the above challenges. Four methods have been studied in depth in this handbook: stratification or “mix-adjustment”, hedonic regression methods, repeat sales, and appraisal-based methods (i.e., the SPAR method). Below, recommendations are made on each. Each method attempts to adjust for the change in the “quality mix” of the houses whose prices are observed and combined to construct the index. Some methods, however, are unable to adjust for quality changes of the individual houses, i.e. for the net effect of depreciation of the structures and renovations and extensions. Where data from the administrative processes for buying and selling a residential property are used in the construction of the index, the price will usually relate either to the offer price or to the selling price - these can differ from one another.
12.15 The recommendations do not address the challenge of computing an RPPI in countries where a significant proportion of the housing stock relates to informal or traditional dwellings. An example of computing an RPPI under the latter circumstances is given in Chapter 10 and draws on the experience of South Africa. In such circumstances it is not possible to be very prescriptive in terms of recommendations since the situation will vary considerably among countries and there is no ideal solution that will deliver a residential property price index which is conceptually pure and does not generate practical difficulties. Rather, the compiler will need to draw on the best available sources of information and will no doubt have to make conceptual and methodological compromises in computing an index. In these circumstances it is particularly important that statistical agencies provide evaluations of the resulting price indices and guide users on their uses.
Stratification or mix-adjustment
12.16 Stratification or mix-adjustment is the most straightforward way to control for changes in the composition or quality mix of the properties sold. It also addresses any user need for sub-indices relating to different housing market segments. The effectiveness of stratification will depend upon the stratification variables used because a mix-adjusted measure only controls for compositional change across the various groups - a mix-adjusted index does not account for changes in the mix of properties sold within each subgroup or stratum.
12.17 In theory, the more detailed the stratification, the more the index controls for changes in the characteristics of the properties covered by the index. However, increasing the number of strata reduces the average number of price observations per stratum and in fact can quickly lead to empty strata. Strata or cells which are empty then lead in turn to a lack of matching when the average price and quantity data in each cell are compared across two time periods. A very detailed stratification might also raise the standard error of the overall index. In addition, it may be difficult to identify the most important price-determining characteristics in the way that a method using hedonic regression can do (see next section).
12.18 The main advantages of stratification/mix-adjustment are:
Depending on the choice of stratification variables, the method adjusts for compositional change amongst the dwellings.
The method is reproducible, conditional on an agreed list of stratification variables.
It is not subject to revision.
Price indices can be constructed for different types and locations of housing.
The method is relatively easy to apply and to explain to users.
12.19 The main disadvantages of stratification/mix-adjustment are:
It cannot deal adequately with depreciation of the houses unless the age of the structure is a stratification variable. The latter can result in problems associated with cells with small numbers of price observations.
The method cannot deal adequately with houses which have undergone major repairs or renovations (unless information on renovations is available).
It requires information on housing characteristics that are included in the strata so that the sales can be allocated to the correct strata.
If the stratification scheme is very coarse, compositional changes will affect the indices.
If the stratification scheme is very fine, the cells can be subject to considerable sampling variability due to small sample sizes or some cells may simply be empty for some periods causing index number difficulties.
The value of land cannot be separated out using this method.
12.20 Stratification/mix-adjustment is an appropriate method where
an appropriate level of detail is chosen for the cells and can be applied in practice;
the age group of the structure is one of the stratification variables;
a decomposition of the index into structure and land components is not required.
12.21 Stratification/mix-adjustment is recommended where the volume of sales is large enough and information on housing characteristics detailed enough to support a detailed classification of properties,(5)
12.22 The application of hedonic techniques for quality adjustment and for computing price indices has made a significant contribution to the methodological development of price indices in recent years and is rapidly becoming a preferred method for compiling constant-quality residential property price indices. (6) There is no uniformity in the practical application of hedonic regression, but the idea underlying hedonics is rather simple. Hedonic regression is a statistical technique that measures the relationship between the observable characteristics of a good or service and its price or value. In the context of residential property price indices, the “best” form of the hedonic function may be linear rather than log-linear to reflect the fact that the value of a property is generally equal to the sum of the price of the structure and the price of the land.
12.23 There are basically two alternative methods of application of hedonics to residential property:
The time dummy variables method. This method generally uses a single regression, with time dummies and fixed characteristics coefficients, which covers all periods and which is re-run each time the price index is compiled. The (exponentials of the) time dummy coefficients are taken to represent the period-to-period price changes excluding quality (mix) changes. This method has the benefit of simplicity. One of the drawbacks is that it raises the issue of “revisability” of the index because the time dummy coefficients will be updated each time new periods are added and the regression is run.
However, there is a variant of the time dummy method, called the rolling window time dummy method, which can work well in practice and solves the revisability problem. A hedonic regression is run using the data for the last N periods and the last time dummy is used as a chain link factor for updating the index for the previous period. For references to the literature on this method and an example, see chapter 5.
The hedonic imputation method. A separate hedonic regression is performed in each time period and the “missing” current period prices for the properties sold in the base period are imputed using the predicted prices from the estimated hedonic equation. A symmetric approach is possible by also imputing the “missing” base period prices for the properties sold in the current period and then taking the geometric mean of both hedonic imputation indices.
12.24 Both hedonic regression methods can potentially suffer from omitted variable bias if some important price determining characteristic is omitted from the regression equation. Multicollinearity can be a practical problem, particularly when a decomposition of the index into structures and land components is required. The time dummy variable method has been frequently used by academics, in part due to its simplicity, but the hedonic imputation method is more flexible - it allows characteristics prices to change independently over time whereas the time dummy method forces characteristics prices to move in a proportional manner - and is essentially similar to the traditional matched-model methodology to compute price indices.
12.25 Hedonic regression methods can be used in conjunction with stratification to deal with any residual quality-mix change that remains within the strata. This has the added advantage of dealing with the fact that different model specifications maybe needed for different segments of the housing market or that the “value” of some characteristics will vary across different market segments.
12.26 The main advantages of hedonics are:
If the list of property characteristics is sufficiently detailed, the method adjusts for both sample mix changes and quality changes (depreciation and renovation) of the individual houses.
Price indices can be constructed for different types of dwellings and locations through stratification and the application of hedonics to each individual stratum.
Stratified price indices based on hedonic regressions to control for quality mix changes within strata allow for relative values of the stock of housing to be used to weight the quality-mix adjusted strata indices (in a stock-weighted RPPI).
The method maximizes the use of the available data.
It can in principle be used to decompose the overall price index into land and structures components, subject to the availability of data.
12.27 The main disadvantages of hedonic regression are:
The method is often regarded as being data intensive, especially in terms of the housing characteristics to be used as explanatory variables. (7)
It may be difficult to control sufficiently for location if property prices and price trends differ across detailed regions.
The method can be sensitive to the variables used in the regression and the functional form for the model.
The method is not particularly easy to explain to users and from their perspective may lack transparency.
12.28Subject to data being available on salient housing characteristics, the hedonic regression method is generally the best technique for constructing a constant quality residential property price index. The imputations approach to hedonic quality (mix) adjustment has advantages over the time dummy approach. Stratified hedonic indices are preferred over a straightforward application of hedonic regression to the whole data set.
12.29 The repeat sales method observes the price development of a specific house over a period of time by reference to the selling price each time it is sold. The price change of a selection of houses during overlapping time periods can then be observed to estimate, using a dummy variable regression model, the general trend in residential property prices. Measuring the average price changes in repeat sales on the same properties ensures a “like for like” comparison (ignoring the fact that depreciation and renovations on the structure between the periods of sale can change the property).
12.30 The main advantages of the repeat sales method are:
In its basic form, it requires no information on characteristics of the dwelling units other than the addresses of the properties that are traded. Source data are often available from administrative records.
It follows a matched-model methodology, under the assumption that depreciation and renovations have not changed the dwelling unit over the time period between subsequent sales.
Many locational and other price determining characteristics that are difficult to measure are likely to be automatically included.
Standard repeat sales regressions are easy to run and the resulting price indices are easy to construct.
No imputations are involved. By construction, location is automatically controlled for.
The results are, in principle, reproducible.
12.31 The main disadvantages of the repeat sales method are:
The method does not use all of the available selling prices; it uses information only on those properties that have sold more than once during the sample period.
The standard version of the method ignores (net) depreciation of the dwelling unit.
Sample selection bias can arise from the restriction to properties that have been sold more than once during the sample period.
The method cannot generate separate price indices for structures and for land.
The reliance on repeat sales means that there may not be enough data points to compute monthly residential property price indices for smaller categories of property.
The sample is updated as new transaction information becomes available. This means that the repeat sales property price index could be subject to retrospective revisions over a long time period. (8)
Since a house must be sold at least twice in a repeat sales index, newly built dwelling units are excluded from such an index.
12.32Although a natural starting point for constructing an index, the repeat sales method is not preferred over the (stratified) hedonic method for constructing a constant quality residential property price index. However, it can offer a solution where there is limited or no information on housing characteristics and there are a relatively large number of repeat transactions to provide enough data points for the required types of residences and where sample selection bias is not considered a problem. It is not recommended when a distinction needs to be made between the price of the structure and the price of the land.
12.33 Appraisal-based methods use “assessed” values, such as valuations for taxation purposes or valuations from specially commissioned surveys using estate agents, often done by reference to similar properties that have been sold, to overcome the two main problems associated with the repeat sales methodology - the relatively small number of price observations which are generated and the susceptibility to sample selection bias. Where the valuations all refer to a standard reference period, the matched model methodology which underlies appraisal-based methods also has the advantage that it can be applied in a straightforward way and with no necessity to use econometrics to adjust for compositional changes. However, like the repeat sales methodology, appraisal-based methods generally cannot deal adequately with quality changes to individual houses. Also, they generally rely on expert judgment on how much a property would sell for rather than on an actual transaction price. Thus, it can be argued, at the extreme, that appraisal-based methods are influenced by judgments or opinions, albeit authoritative and objective.
12.34 The Sale Price Appraisal Ratio (or SPAR) method uses appraisals with a common reference period as base period prices in a standard matched-model framework (though the results are normalized to obtain an index that equals 1 (or 100) in the base period). The experiences of the few countries that have computed a SPAR index (9) are generally positive although some researchers have reported a bias arising from frequent re-assessments and reduced precision over time arising from new appraisals.
12.35 The main advantages of the SPAR method are:
Being based on the standard matched model methodology, it is consistent with traditional index number theory.
It is straightforward to compute.
The method benefits from many more observations than the repeat sales method and is therefore less susceptible to problems arising from having a relatively small number of price observations.
It is less susceptible to sample selection bias than the repeat sales method.
It does not suffer from revisions to previously computed figures.
It is reproducible.
12.36 The main disadvantages of the SPAR method are:
It cannot deal adequately with quality changes (depreciation and renovations) of the dwelling units. (10)
Data on value assessments at the address level must be available for all properties.
The method is dependent on the quality of the assessments.
It cannot be used to decompose the overall property price index into land and structures components. (11)
12.37The SPAR methodology addresses some of the weaknesses of the repeat sales methodology and is to be preferred to the latter methodology if assessment data of sufficient quality are available and if selectivity bias is considered to be a serious feature of the application of the repeat sales methodology. The SPAR methodology does have its drawbacks but is recommended when the use of hedonics is not possible. The results from the SPAR method are improved if it is used in conjunction with stratification.
12.38 If the initial house price series indicates that some seasonal fluctuations occur, then normal seasonal adjustment techniques can be used in order to seasonally adjust the initial series. However, if the hedonic imputation or the stratification method is used to construct the initial index, some more specific recommendations are made below.
12.39 If the stratification method is used to construct the initial index and it exhibits seasonality, then the rolling year method explained in Chapter 5 can be applied to seasonally adjust the series without relying on econometric methods.
12.40 If the hedonic imputation method is used to construct the initial price index and it exhibits seasonality, then in order to obtain a seasonally adjusted series, it may be useful to construct year-over-year monthly or quarterly series as an initial step. These initial series can then be aggregated using the rolling year method into a smoothened seasonally adjusted series.
Regardless of the form used, expressed either in terms of values or indices, the per cent change will be the same.
Of course the particular circumstances will dictate the extent of the data cleaning. If the principal user is also managing the collection of information, then the survey will be tailored to his or her needs and the extent of the cleaning will likely be less extensive.
Since the number of transactions will likely vary from period to period, the number of price observations in the sample for each period will also vary.
Note that most central tendency measures of house prices when published do not typically include indicators of statistical quality such as the coefficient of variation or standard deviation.
Note that the required data is obtained for calculating either the median or mean prices; the steps involved are quite simple. Most statistical software packages can do the entire exercise quite rapidly with little intervention from the compiler.
Skewness is a measure of the asymmetry of a distribution. When the degree of skewness is zero this means that the distribution is symmetric around its mean. A positive skew means that a relatively high number of observations from the sample is concentrated on the left of the centre point and vice versa.
With these particular data, the mean was always greater than the corresponding median. This result need not always hold, particularly with very small samples.
Typically, the mean price will be higher than the corresponding median price. However, when mean and median indices are formed, there is no presumption that the mean index will increase more rapidly than the median index.
This example is loosely based on an example in McDonald and Smith (2009).
This example uses the neighbourhood as the sub-stratum but in reality it can be any geographical area for which the compiler is confident that a sufficiently large enough sample of transactions is available today and in the future to generate a reliable representative price.
This is assuming that the compiler is using sales as the basis for the weighting.
The period 0 values for all of the indices defined in this section are set equal to 1.
Fisher (1922; 466) showed that P0 defined by (11.5) is equal to the Laspeyres index PL defined by (11.1). Fisher also attributed the index P1 defined by (11.6) to Palgrave.
It can be verified that the geometric mean of PGL and PGp is exactly equal to PT. Thus if PGL is below PT then PGp will necessarily be above PT.
Although most hedonic regressions on house prices in the literature will often use many more explanatory variables, some studies and the examples in Chapter 5 show that reliable hedonic price indices can be obtained with as few as four independent variables.
There is nothing to prevent however the use of a semi-log or log functional form. Both can be used with this hedonic approach.
Note that the coefficients for the number of bedrooms are somewhat volatile between both years. This is to be expected because hedonic regressions are often characterized by the presence of multicollinearity between these two predictor variables. It should be stressed however that multicollinearity does not in itself affect the accuracy of the overall index. This phenomenon is only an issue if an accurate monetary value is needed for the value of an additional bedroom and/or for an additional bathroom, such as would be the case with a property assessment exercise. It should also be added that for the purpose of this simplified exercise, the sample size is relatively small. This can also explain why sometimes the results are not quite as robust as is often the case with larger samples.
One assumption is that the quality of the house has not changed over the period between the two sales. If information about the features of the property is available to the compiler, then it is possible to exclude from the calculation those observations that have undergone significant changes over time and that are likely to affect the price and thus distort the index. Furthermore, given that high turnover is often a sign that certain undesirable features for that particular property may be at play so that these observations can also be excluded from the calculation. It should also be mentioned that repeat-sales indices are not always strictly constant quality price indices since houses are often subject to some loss in value over time as a result of depreciation. Consequently, repeat-sales price indices typically underestimate true house price inflation, unless some corrective adjustment is made to the estimates. If the purpose of the index is to act as a short- to medium-term indicator of house prices, then the issue of depreciation which the repeat-sales approach does not handle adequately can perhaps be set aside.
The example is partially drawn from the Canadian Teranet-National Bank® repeat sales price index documentation: http://www.housepriceindex.ca/Default.aspx.
Geltner and Pollakowski (2006) use the term “round trip”.
There are very few observations so no meaningful conclusions should be drawn from this simplified example. It should only be used for illustrative purposes.
This includes conversions of existing property, for example where a warehouse has been converted into flats or an existing property has been sub-divided.
Renovations to existing dwelling units are also part of residential construction investment.
The argument in favour of the net acquisition approach is that it is the closest to the “acquisition” approach which has traditionally been adopted for other parts of a CPI and is most appropriate for a CPI being used as a general indicator of current economic conditions. But the method can draw criticism from those who require a CPI as a compensation index, as neither the weight nor the price indicator properly reflect the shelter costs of owner-occupiers. For instance, a rise in interest rates would not be reflected in a net acquisition index. See CPI Manual (2004) and the Practical Guide to Producing Consumer Price Indices (United Nations, 2009).
In some cases even the selling prices may not reflect the “true” market values, for example when they relate to distressed sales arising from divorce etc.
A coarse stratification by, say, major city and house type, where the latter is simply in terms of “newly-built” or “existing”, is not recommended.
If we look at the harmonised house price indices produced by the European National Statistical Institutes, as of 2011 more than half were using hedonics for quality adjustment. For more details, see Marola et al. (2012).
However, as was seen in previous chapters, in some cases satisfactory results can be obtained with hedonic regression methods using only three or four housing characteristics.
In practice, the link factor for the last two periods in the current repeat sales regression can be used to update the ongoing index.
In Europe, Denmark, Sweden and the Netherlands are using the SPAR method.
As with the repeat sales method, the price index generated by the SPAR method can in principle be adjusted by using exogenous information on the net depreciation of properties of the type being considered.
Where official decompositions of the total assessed value of the property into land and structures components are available, these could be used to check the land and structures price indices that are generated by hedonic regression methods.