A. Hedonic Imputation (HI) Indexes

Hedonic imputation (hereafter—HI) indexes take a number of forms: (i) as either equally-weighted or weighted indexes; (ii) depending on the functional form of the aggregator, say a geometric aggregator as against an arithmetic one; (iii) with regard to which period’s characteristic set is held constant; and (iv) as direct binary comparisons between periods 0 and t, or as chained indexes. For chained indexes the individual links are calculated between periods 0 and 1, 1 and 2,…, t – 1 and t, and the results combined by successive multiplication.

We consider in this section, as equations (2) and (3) respectively, hedonic Laspeyres and Paasche indexes—weighted, arithmetic, constant base (Laspeyres), and current period (Paasche), aggregators for binary comparisons—and then focus on a generalized hedonic Törnqvist index, given by equation (4). The Törnqvist index is a weighted, geometric aggregator which makes symmetric use of base and current information in binary comparisons. It is a superlative index and, thus, has highly desirable properties. An index number is defined as exact when it equals the true cost of living index for a consumer whose preferences are represented by a particular functional form. A superlative index is defined as an index that is exact for a flexible functional form that can provide a second-order approximation to other twice-differentiable functions around the same point. Superlative indexes are generally symmetrical with respect to their use of information from the two time periods (see Diewert, 2004). Fisher and Walsh index formulas are also superlative indexes and closely approximate the Törnqvist index. The Fisher index is the preferred target index in the international CPI Manual (Diewert, 2004, chapters 15–18).

We start by outlining the hedonic formulations of the well-known Laspeyres and Paasche indexes. Consider the hedonic function ${\widehat{p}}_i^0=h^0\left(z_i^0\right)$ from the semi-logarithmic form of (1), estimated in period 0 with a vector of K quality characteristics $z_i^0=z_{i1}^0,\dots \dots ,z_{iK}^0$ and N⁰observations and similarly for period 1. Let quantities sold in periods 0 and 1 be $q_i^0$ and $q_i^1$ respectively.

A hedonic Laspeyres index for matched and unmatched period 0 models is given by:

and a hedonic Paasche index for matched and unmatched period 1 models by:

It is apparent from equations (2) and (3) that a hedonic Laspeyres index holds characteristics constant in the base period and a hedonic Paasche index holds the characteristics constant in the current period. Thus the differences between the hedonic valuations in Laspeyres and Paasche are dictated by the extent to which the characteristics change over time; that is, $(z_i^1-z_i^0)$. The farther the z_i values differ over time, say due to greater technological change, the less justifiable is the use of an individual estimate and the less faith there is in a compromise geometric mean of the two indexes—a Fisher index. Note that new (unmatched) models available in period 1, but not in period 0, are excluded from equation (2) and old (unmatched) models available in period 0, but not in period 1, are excluded from equation (3). Laspeyres and Paasche HI indices suffer from a sample selectivity bias.

Let i ϵ S^t (t = 0,1) be the set of models available in period t. Let i ϵ S^M ≡ S⁰ ∩ S¹ be the set of matched models with common characteristics $z_i^m=z_i^0=z_i^1$ in both periods 0 and 1.

Unmatched new models present in period 1, but not in period 0 are given by i ϵ S¹ (1¬0); and unmatched old models present in period 0, but not in period 1, by i ϵ S⁰(0¬1). Let the number of models in these respective sets be denoted by N^M, N⁰ (0¬1) and N¹ (1¬0). A hedonic Törnqvist index, for matched models only, is given by the first term on the right-hand-side of equation (4). The hedonic Törnqvist index is generalized to include disappearing and new models by the respective inclusion of the second and third terms on the right-hand-side of equation (4). An alternative and equivalent formulation to equation (4) would be to include only these last two terms, but with the products taken over iϵ S⁰ and i ϵ S¹ respectively. However, we use equation (4) as it provides a more detailed, and analytically useful, decomposition of the price changes of the different sets of models.

A generalized hedonic Törnqvist index is given by:

where relative expenditure shares for model i in period t are given by $s_i^t=p_i^t{q}_i^t/{\displaystyle {\sum }_j{p}_j^t{q}_j^t}$ for t=0,1 and expenditure shares for matched models m are an average of those in periods 0 and 1, that is, ${\tilde{s}}_i^m=(s_i^0+s_i^1)/2$ for i ϵ S^M. Note that ${\tilde{s}}_i^m$ (for i ϵ S^M) plus $s_i^0/2$ (for i ϵ S⁰ (0-1) ) plus $s_i^1/2$ (for iϵ S¹ (1-0)) sum to unity. For illustration, if the expenditure shares of matched models were 0.6 and 0.7 in periods 0 and 1, respectively; of unmatched old models in period 0, 0.4; and unmatched new models in period 1, 0.3; then ${\displaystyle {\sum }_i{\tilde{s}}_i^m=0.65}$, ${\displaystyle {\sum }_i{s}_i^0/2=0.2}$ and ${\displaystyle {\sum }_i{s}_i^1/2}=0.15$.

Of note is that estimated prices are used for matched models; a good case can be made for using actual prices for matched models when available (Haan, 2004, p. 2). Equation (4) is a (superlative) Törnqvist HI index generalized to include new and disappearing models. In Section II. B. a dummy time hedonic index will be identified as an alternative approach to estimating a generalized Törnqvist hedonic index. The issue addressed by the paper is to identify an expression for the differences between the hedonic imputation and dummy time hedonic approaches. As will be seen, an econometric device is useful in this respect which requires we work with predicted, rather than actual, prices for matched models, although this paper is not alone in this (Pakes, 2003).

B. Dummy Time Hedonic (DTH) Indexes

Dummy time hedonic (hereafter—DTH) indexes are a second approach to estimating price changes that use hedonic regressions to control for the different quality mix of new and disappearing models. As with HI indexes, DTH indexes do not require a matched sample. In this section we show how the generalized Törnqvist hedonic index in (4) can be estimated as a DTH index. The DTH formulation is similar to equation (1) except that a single regression is estimated on the data in the two time periods, 0 and 1 compared, i ∈ S^t for t = 0,1. The prices, $p_i^t$, for each model i, are regressed on a dummy variable $D_{0i}^t$ which is equal to 1 in period 1 and zero otherwise, and $z_{ki}^t=1,\dots .,K$, price-determining characteristics in a regression with well-behaved residuals ${\epsilon }_i^t$:

The exponent of the estimated coefficient ${\delta }_1^{*}$ is an estimate of the quality-adjusted price change between period 0 and period 1 regardless of the reference quality vector. Consider for simplicity the case of only matched models where there is no need for the quality characteristics $z_{ki}^t$ in (5). Then ${\delta }_1^{*}=1/n{\displaystyle {\sum }_{i=1}^n(\mathit{ln}{p}_i^1-\mathit{ln}{p}_i^0)}$ and exp $\left({\delta }_1^{*}\right)$ is the geometric mean of $p_i^1/p_i^0$ — with an adjustment, as detailed in van Garderen and Shah (2002). Also note that for unmatched models, since ${\beta }_k^0={\beta }_k^1={\beta }_k$ in (5), the value of ${\delta }_1^{*}$ is invariant to the level of $z_{ki}^t$— the lines of the functions of ln $p_i^t$ on $z_{ki}^t$ for periods t = 0,1 are parallel and the shift intercept constant.

It may at first be although that weighted indexes such as the target Törnqvist index cannot be compared with DTH indexes, as in (5), since the latter are unweighted (equally weighted). However, Diewert (2002 and 2005) shows that if a weighted least squares (WLS) estimator is applied to (5), the resulting estimate of price change will correspond to a weighted index number formula. More particularly, the formulation of the weights for the WLS estimator dictates which index number formula the DTH estimate corresponds to. A WLS estimator is equivalent to an OLS estimator applied to data which have been repeated in line with their weight, akin to repeated sampling. A DTH price change estimate based on a WLS estimator, with weights ${\tilde{s}}_i^m=(s_i^0+s_i^1)/2$ for matched models and $s_i^0/2$ or $s_i^1/2$ for the unmatched old and new models respectively, corresponds to a generalized Törnqvist index (Diewert, 2005). In section 3.1 use is made of this weighting structure to derive and compare generalized DTH and HI Törnqvist index estimates.

The regression equation (5) constrains each of the β_k coefficients to be the same across the two periods compared. In restricting the slopes to be the same, the (log of the) price change between periods 0 and 1 can be measured at any value of z, as illustrated by the difference between the dashed lines in Figure 1. For convenience it is first evaluated at the origin as ${\delta }_1^{*}$ Bear in mind that the HI indexes outlined above estimate the differences between price surfaces with different slopes. As such, the estimates have to be conditioned on particular values of z, which gives rise to the two estimates (whose arithmetic equivalents are) considered in (2) and (3): the base HI using z⁰ and the current period HI using z¹ as shown in Figure 1. The very core of the DTH method is to constrain the slope coefficients to be the same, so there is no need to condition on particular values of z. The DTH estimates implicitly and usefully make symmetric use of base and current period data. As with hedonic imputation indexes, DTH indexes can take fixed and chained base forms, although they can also take a fully constrained form whereby a single constrained regression is estimated for say January to December with dummy variables for each month, although this is impractical in real time since it requires data on future observations.

• Download Figure
• Download figure as PowerPoint slide

View Full Size

• Download Figure
• Download figure as PowerPoint slide

• Download Figure
• Download figure as PowerPoint slide

A. Algebraic Differences: A Reformulation of the Hedonic Indexes

There has been little analytical work undertaken on the factors governing differences between the two approaches. To compare the HI approach to the DTH approach we first need to reformulate the HI indexes. We note that the HI approach relies on two estimated hedonic equations, $h^1(z_i^1)$ and $h^0(z_i^0)$ for periods 0 and 1 respectively:

We assume that the errors in each equation are similarly distributed, then phrase the two equations as a single hedonic regression equation with dummy time intercept and slope variables:

where $D_{0i}^t=1$ if observations are in period 1 and 0 otherwise, ${\gamma }_1=({\gamma }_0^1-{\gamma }_0^0)$, $D_{ki}^t=z_{ki}^1$ if observations are in period 1 and 0 otherwise, and ${\beta }_k=({\beta }_k^1-{\beta }_k^0)$. The estimated ${\gamma }_1^{*}$ is an estimate of the change in the intercepts of the two hedonic price equations and is thus an HI index evaluated at a particular value of $z_{ki}^t,i\in S^t$, i ϵ S^t and t = 0,1; let this value be denoted by ${\tilde{z}}_k^t$ which is equal to zero at the intercept. An HI index evaluated at ${\tilde{z}}_k^t=0$ has no economic meaning.

For our phrasing of a HI index in (10) to correspond to the generalized hedonic Törnqvist index in (4) two things are required. First, a weighted least squares (WLS) estimator should be used to estimate γ₁ from equation (8) with weights ${\tilde{s}}_i^m=(s_i^0+s_i^1)/2$ for matched models and $s_i^0/2$ or $s_i^1/2$ for unmatched old and new models respectively (Diewert, 2002 and 2005). Second, the estimate of γ₁ in (8) is at the intercept, while the generalized HI Törnqvist index (4) requires it be evaluated at the mean value of $z_{ki}^t$ implicit in the generalized Törnqvist HI index of (4):

For a Törnqvist HI index the ${\gamma }_1^{*}$ estimate is evaluated at ${\tilde{z}}_k^t={\overline{z}}_{k\mathit{\text{Törn}}}^t$. This requires an adjustment to the ${\gamma }_1^{*}$estimate.

The required generalized HI Törnqvist index is given by the exponent of:

where ${\beta }_k^{*}$ is a WLS estimate of $({\beta }_k^1-{\beta }_k^0)$. Annex 1 demonstrates, by way of Figure 2, that, for a single, k = 1 variable, z, ${\displaystyle \sum _{k=1}^k{\beta }_k^{*}{\overline{z}}_{k\mathit{\text{Törn}}}^t}$ is the required adjustment.

Consider now the DTH index in (5) which constrains ${\beta }_k=({\beta }_k^1-{\beta }_k^0)=0$ in (8) and thus ${\displaystyle {\sum }_{k=1}^k{\beta }_k{D}_{ki}}$ to be zero. The DTH index in (5) corresponds to a generalized DTH Törnqvist index if estimated using WLS where the weights are those outlined after equation (5) above. A natural question is how does the estimated DTH index ${\delta }_1^{*}$ in (5), which is invariant to values of $z_{ki}^t$, differ from the HI index evaluated at the means ${\overline{z}}_{k\mathit{\text{Törn}}}^t$ in (10)?

B. How Does a Törnqvist HI Index Differ from a Törnqvist DTH Index?

This difference is first considered by comparing ${\gamma }_1^{*}$ from (8), the HI index, and ${\delta }_1^{*}$ from (5), the DTH index. We are interested in the difference in these two estimated intercept shifts, where; ${\tilde{z}}_k^t=0$; between the estimated constant-quality shift parameters from the constrained (DTH) and unconstrained (HI) regression equation (5) and equation (8) respectively. Being intercept shifts, the difference between these two indexes will be determined at the origin, where ${\tilde{z}}_k^t=0$. This is useful as a first stage in the derivation. However, we then extend the analysis to examine how the expressions differ at, more usefully, the mean ${\overline{z}}_{k\mathit{\text{Törn}}}^t$ from (9). We now turn to a consideration of the difference $({\delta }_1^{*}-{\gamma }_1^{*})$ between these two dummy variable parameter estimates at the origin, as ‘omitted variable bias’ due to the omission of ${\displaystyle {\sum }_{k=1}^k{\beta }_k{D}_{ki}}$ in (8).

Expressions for the bias in estimated regression parameters due to the omission of relevant variables are well established (see Davidson and McKinnon, 1993). The bias for a, for example, parameter estimate of β₁ in a regression equation: y = β₀ + β₁ x₁ + β₂ x₂ + u from a regression that excludes x₂ is equal to the coefficient on the excluded variable, β₂, multiplied by the coefficient on the included variable, α₁, from an auxiliary regression of the excluded on the included variable, i.e. x₂ = α₀ + α₁x₁ + ω. Consider a simplified case of (8) of a single k = 1 characteristic and two time periods, the principles being readily extended. The auxiliary regression is the slope dummy variable, $D_{1i}^t(=z_{1i}^1$ if period 1 and 0 otherwise, in (8)), regressed on the remaining right-hand-side variables in (8), the intercept dummy $D_{0i}^t$ and the $z_{1i}^t$ characteristic with an error term ${\omega }_i^t$:

Omitted variable bias is the product of the estimated coefficient on the omitted variable, ${\beta }_1^{*}$ (for k = 1 in (8)), and the estimated coefficient ${\lambda }_1^{*}$ from the above regression (11)— Davidson and McKinnon (1993). Thus the difference (before taking exponents) DTH minus HI, at the intercept is $({\delta }_1^{*}-{\gamma }_1^{*})={\beta }_1^{*}\times {\lambda }_1^{*}$.

Our next concern is to derive this difference at ${\overline{z}}_{1\mathit{\text{Törn}}}^t$ rather than at ${\tilde{z}}_k^t=0$. Since the DTH method holds the parameter estimates constant through any value of $z_k^{\mathit{t}}$, the (log of the) DTH index is thus given by ${\delta }_1^{*}={\beta }_1^{*}\times {\lambda }_1^{*}+{\gamma }_1^{*}$.

However, the (log of the) Törnqvist HI index at ${\overline{z}}_{1\mathit{\text{Törn}}}^t$ from (8) for one variable is estimated as:

Thus the ratio of the DTH and HI indexes at the intercept is $\mathit{exp}({\delta }_1^{*})/\mathit{exp}({\gamma }_1^{*})=\mathit{exp}({\beta }_1^{*}\times {\lambda }_1^{*})$ and the DTH index is thus given by $\mathit{exp}({\delta }_1^{*})=\mathit{exp}({\beta }_1^{*}\times {\lambda }_1^{*})\times \mathit{exp}({\gamma }_1^{*})$. The Törnqvist HI index at ${\overline{z}}_{1\mathit{\text{Törn}}}^t$ from (8) for one variable is estimated as: ${exp}({\gamma }_1^{*}+{\beta }_1^{*}{\overline{z}}_{1\mathit{\text{Törn}}}^t)$. Thus the ratio of the Törnqvist DTH index to the Törnqvist HI index at ${\overline{z}}_{1\mathit{\text{Törn}}}^t$ is:

where ${\beta }_1^{*1}$ and ${\beta }_1^{*0}$ are WLS estimates.

If either of the two terms making up the product on the right-hand-side is close to zero then there will be little difference between the indexes. Neither parameter instability nor a change in the mean characteristic is sufficient in itself to lead to a difference between the formulas. The ${\beta }_1^{*}=({\beta }_k^{*1}-{\beta }_k^{*0})$ from (5) is the estimated marginal valuation of the characteristic between periods 0 and 1, which can be positive or negative, but may be more generally althought to be negative to represent diminishing marginal utility/cost of the characteristic.

C. Interpretation of $({\lambda }_1^{*}-{\overline{z}}_{1\text{Törn}}^t)$

Equation (13) shows us that the change in the coefficients is one factor determining difference between the two methods. The second expression, $({\lambda }_1^{*}-{\overline{z}}_{1\text{Törn}}^t)$, is more difficult to interpret and we consider it here. Bear in mind that the left-hand-side of the regression in equation (11), $D_{1i}^t$ is 0 in period 0 and $z_{1i}^1$ in period 1 and that $D_{0i}^t$ on the right-hand-side is 1 in period 1 and zero in period 0. If we assume quality characteristics are positive, λ₁ will always be positive as the change from 0 in period 0 to their values in period 1. Consider the weighted (Törnqvist) mean ${\overline{z}}_{1\mathit{\text{Törn}}}^0$ for i ϵ S^M and i є S⁰(0¬t) (matched period 0 and unmatched old period 0) and ${\overline{z}}_{1\text{Törn}}^1$ for i ϵ S ^M and i ϵ S^t (1¬0) (matched period 1 and unmatched new models in period 1).

If we assume for simplicity that ${\overline{z}}_{1\text{Törn}}^0={\overline{z}}_{1\text{Törn}}^1$, then ${\lambda }_1^{*}\approx {\overline{z}}_{1\text{Törn}}^1$ since ${\lambda }_1^{*}$ is an estimate of the change in $D_{1i}^t$ in (11) arising from changing from period 0 to period 1, where it is ${\overline{z}}_{1\text{Törn}}^1$ and has an expectation of ${\overline{z}}_{1\text{Törn}}^1$. The ${\lambda }_1^{*}$ estimate is conditioned in (11) on ${\overline{z}}_{1\text{Törn}}^t$, the change from ${\overline{z}}_{1\text{Törn}}^0$ to ${\overline{z}}_{1\text{Törn}}^1$, but since we assume these two have not changed, our estimate of ${\lambda }_1^{*}\approx {\overline{z}}_{1\text{Törn}}^1$ holds true. Thus the second part of the difference expression in (13), $({\lambda }_1^{*}-{\overline{z}}_{1\mathit{\text{Törn}}}^t)$, is simply $({\overline{z}}_{1\text{Törn}}^1-{\overline{z}}_{1\text{Törn}}^t)$ which, given our assumption of ${\overline{z}}_{1\text{Törn}}^0={\overline{z}}_{1\text{Törn}}^1$, is equal to 0. It follows from the right-hand side of (13), that for samples with negligible change in the mean values of the characteristics, the DTH and HI will be similar irrespective of any parameter instability. Diewert (2002) and Aizcorbe (2003) have shown that the DTH and HI indexes will be the same for matched models and this analysis gives support to their finding. However, we find first, that it is not matching per se that dictates the relationship; for unmatched models all that is required is that ${\overline{z}}_{1\text{Törn}}^1={\overline{z}}_{1\text{Törn}}^t$ which may occur without matching—it simply requires the means of the characteristics not to change. Second, that even when the means change the two approaches will be equal if ${\beta }_k^{*1}-{\beta }_k^{*0}=0$ i.e. there is parameter stability. Finally, it follows that if either of the two right-hand side expressions in (13) are large, the differences between the indexes will be compounded.

But what if ${\overline{z}}_{1\mathit{\text{Törn}}}^0\ne {\overline{z}}_{1\text{Törn}}^1$ ? Since the estimated coefficient on x₁ of a regression of y on x₁ and x₂ is given by: $\frac{{\displaystyle \sum y{x}_1}{\displaystyle \sum x_2^2}-{\displaystyle \sum y{x}_2}{\displaystyle \sum x_1{x}_2}}{{\displaystyle \sum x_1^2}{\displaystyle \sum x_2^2}-{({\displaystyle \sum x_1{x}_2})}^2}$, the estimated coefficient ${\lambda }_1^{*}$ from (11) is given

by:

for an unweighted regression where N¹ and N are the respective number of observations in period 1 and both periods 0 and 1, ${\sigma }_z^2$ is the variance of z and$\text{cov}(D_{1i}^t,z_{1i}^t)={\displaystyle \sum {\left(z^t\right)}^2-N^1{\overline{z}}^1\overline{z}}$ is the covariance of $D_{1i}^t$ and $z_{1i}^t$ from (11). Readers are reminded that from (13):

First, as noted, if there is either negligible parameter instability or a negligible change in the mean of the characteristic, then there will be little difference between the formulas. However, as parameter instability increases and the change in the mean characteristic increases, the multiplicative effect on the difference between the indexes is compounded. The likely direction and magnitude of any difference is not immediately obvious. Assume diminishing marginal valuations of characteristics, so that $({\beta }_1^{*1}-{\beta }_1^{*0})\langle 0$. Second, even assuming a positive technological advance, $({\overline{z}}_1^1-{\overline{z}}_1^t)\rangle 0$ and given (N – N¹), ${cov}(D_{1i}^t,z_{1i}^t),{\overline{z}}_1^1$ and ${\sigma }_z^2$ are positive, it remains difficult to establish from (14) the effect on ${\lambda }_1^{*}$ of changes in its constituent parts. However, third, as N¹ becomes an increasing share of N, and at the limit if N¹ takes up all of N (i.e. (N – N¹) → 0), then ${cov}(D_{1i}^t,z_{1i}^t)\to {\sigma }_z^2$ and importantly, ${\overline{z}}_1^1\to {\overline{z}}_1^t$ and the difference between the formulas tends to zero. Note that (14) is based on an OLS estimator and for a WLS Törnqvist estimator similar principles apply, although the determining factor for a DTH index to exceed a HI index is for the weights of new models to be increasing, a much more reasonable scenario. Thus the nature and extent of any differences between the two indexes will, aside from the parameter change, also depend on (i) changes in the mean quality of models $({\overline{z}}_1^1-{\overline{z}}_1^t)$, (ii) the relative number of models in each period, (N – N¹), (iii) the dispersion in z, ${\sigma }_z^2$ (iv) the mean of the characteristics in period 1,${\overline{z}}_1^1$ ,(v) the cov$(D_{1i}^t,z_{1i}^t)$ which as (N – N¹) → 0, i.e. N¹ takes up all of N, then $\text{cov}(D_{1i}^t,z_{1i}^t)\to {\sigma }_z^2$ and ${\overline{z}}_1^1={\overline{z}}_1$ and ${\lambda }_1^{*}=0$.

D. Treatment of Unmatched Observations

Diewert (2002) and Aizcorbe (2003) show that while the DTH and HI indexes will be the same for matched models, they differ in their treatment of unmatched data. Consider hedonic functions $h_i^1(z_i^1)$ and $h_i^0\left(z_i^0\right)$ for periods 1 and 0 respectively, as in (2) and (3), and a (constrained) time dummy regression equation (5). Consider further an unmatched observation only available in period 1. A base period HI index such as (2) would exclude it, while a current period HI index, such as (3), would include it, and a geometric mean of the two would give it half the weight in the calculation of that of a matched observation. A Törnqvist hedonic index, (4), would also give an unmatched model half the weight of a matched one. For a DTH index, such as (5), an unmatched period 1 model would appear only once in period 1, in the estimation of constrained parameters, as opposed to twice for matched data. We would therefore expect superlative HI indexes, such as (4), to be closer to DTH indexes than their constituent elements, (2) and (3), because they make symmetric use of the data.

E. Observations With Undue Influence

HI indexes, such as (2) and (3), explicitly incorporate weights. Silver (2002) has shown that weights are implicitly incorporated in DTH indexes by means of the OLS or WLS estimator used. Silver (2002) has further shown, for DTH indexes, that the manner in which the estimator incorporates the weights may not fully represent the weights, due to adverse influence and leverage effects generated by observations with unusual characteristics and above average residuals.

F. Chaining

Chained base HI indexes are preferred to fixed base ones, especially when matched samples degrade rapidly. In such a case, their use reduces the spread between Laspeyres and Paasche indexes. However, caution is advised in the use of chained monthly series when prices may oscillate around a trend (i.e. ‘bounce’) and as a result, chained indexes can ‘drift’ (Forsyth and Fowler, 1981 and Szulc, 1983).