Chapter

20. Elementary Indices

Author(s):
International Monetary Fund
Published Date:
September 2004
Share
  • ShareShare
Show Summary Details

A. Introduction

20.1 In all countries, the calculation of an output PPI proceeds in two (or more) stages. In the first stage of calculation, elementary price indices are estimated for the elementary aggregates of a PPI. In the second and higher stages of aggregation, these elementary price indices are combined to obtain higher-level indices using information on the net output on each elementary aggregate as weights. An elementary aggregate consists of the revenue from a small and relatively homogeneous set of commodities defined within the industrial classification used in the PPI. Samples of prices are collected within each elementary aggregate, so that elementary aggregates serve as strata for sampling purposes.

20.2 Data on the revenues, or quantities, of different goods and services are typically not available within an elementary aggregate. Since there are no quantity or revenue weights, most of the index number theory outlined from Chapter 15 to 19 is not directly applicable. As was noted in Chapter 1, an elementary price index is a more primitive concept that often relies on price data only.

20.3 The question of which is the most appropriate formula to use to estimate an elementary price index is considered in this chapter. The quality of a PPI depends heavily on the quality of the elementary indices, which are the basic building blocks from which PPIs are constructed.

20.4 As was explained in Chapter 6, compilers have to select representative commodities within an elementary aggregate and then collect a sample of prices for each of the representative commodities, usually from a sample of different establishments. The individual commodities whose prices actually are collected are described as the sampled commodities. Their prices are collected over a succession of time periods. An elementary price index is therefore typically calculated from two sets of matched price observations. It is assumed in this chapter that there are no missing observations and no changes in the quality of the commodities sampled, so that the two sets of prices are perfectly matched. The treatment of new and disappearing commodities, and of quality change, is a separate and complex issue that is discussed in detail in Chapters 7, 8, and 21 of the Manual.

20.5 Even though quantity or revenue weights are usually not available to weight the individual elementary price quotes, it is useful to consider an ideal framework where such information is available. This is done in Section B. The problems involved in aggregating narrowly defined price quotes over time also are discussed in this section. Thus, the discussion in Section B provides a theoretical target for practical elementary price indices constructed using only information on prices.

20.6Section C introduces the main elementary index formulas used in practice, and Section D develops some numerical relationships between the various indices. Chapters 15 to 17 developed the various approaches to index number theory when information on both prices and quantities was available. It also is possible to develop axiomatic, economic, or sampling approaches to elementary indices, and these three approaches are discussed below in Sections E, F, and G. Section H develops a simple statistical approach to elementary indices that resembles a highly simplified hedonic regression model. Section I concludes with an overview of the various results.1

B. Ideal Elementary Indices

20.7 The aggregates covered by a CPI or a PPI usually are arranged in the form of a tree-like hierarchy, such as COICOP or NACE. Any aggregate is a set of economic transactions pertaining to a set of commodities over a specified time period. Every economic transaction relates to the change of ownership of a specific, well-defined product (good or service) at a particular place and date, and comes with a quantity and a price. The price index for an aggregate is calculated as a weighted average of the price indices for the subaggregates, the (net output) weights and type of average being determined by the index formula. One can descend in such a hierarchy as far as available information allows the weights to be decomposed. The lowestlevel aggregates are called elementary aggregates. They are basically of two types:

  • (i) Those for which all detailed price and quantity information is available, and

  • (ii) Those for which the statistician, considering the operational cost and the response burden of getting detailed price and quantity information about all the transactions, decides to make use of a representative sample of commodities or respondents.

The practical relevance of studying this topic is large. Since the elementary aggregates form the building blocks of a CPI or a PPI, the choice of an inappropriate formula at this level can have a tremendous impact on the overall index.

20.8 In this section, it will be assumed that detailed price and quantity information is available for all transactions pertaining to the elementary aggregate for the two time periods under consideration. This assumption allows us to define an ideal elementary aggregate. Subsequent sections will relax this strong assumption about the availability of detailed price and quantity data on transactions, but it is necessary to have a theoretically ideal target for the practical elementary index.

20.9 The detailed price and quantity data, although perhaps not available to the statistician, are, in principle, available in the outside world. It is frequently the case that at the respondent level (that is, at the firm level), some aggregation of the individual transactions information has been executed, usually in a form that suits the respondent’s financial or management information system. This respondent determined level of information could be called the basic information level. This is, however, not necessarily the finest level of information that could be made available to the price statistician. One could always ask the respondent to provide more disaggregated information. For instance, instead of monthly data, one could ask for weekly data; or, whenever appropriate, one could ask for regional instead of global data; or, one could ask for data according to a finer product classification. The only natural barrier to further disaggregation is the individual transaction level.2

20.10 It is now necessary to discuss a problem that arises when detailed data on individual transactions are available. This may occur at the individual establishment level, or even for individual production runs. Recall that in Chapter 15, the price and quantity indexes, P(p0, p1, q0, q1) and Q(p0, p1, q0, q1), were introduced. These (bilateral) price and quantity indices decomposed the value ratio V1/V0 into a price change part P(p0, p1, q0, q1) and a quantity change part Q(p0, p1, q0, q1). In this framework, it was taken for granted that the period t price and quantity for product i, Pit and qit, were well defined. However, these definitions are not straightforward, since individual purchasers may buy the same product during period t at different prices. Similarly, consider the sales of a particular establishment, when the same product may sell at very different prices during the course of the period. Hence before a traditional bilateral price index of the form P(p0, p1, q0, q1) considered in previous chapters of this Manual can be applied, there is a nontrivial time aggregation problem to obtain the basic prices Pit and qit that are the components of the price vectors p0 and p1 and the quantity vectors q0 and q1. Walsh3 (1901, 1921a) and Davies (1924, 1932) suggested a solution in a CPI context to this time aggregation problem: the appropriate quantity at this very first stage of aggregation is the total quantity purchased of the narrowly defined product, and the corresponding price is the value of purchases of this product divided by the total amount purchased, which is a narrowly defined unit value. The appropriate unit value for a PPI context is the value of revenue divided by the total amount sold. In more recent times, other researchers have adopted the Walsh and Davies solution to the time aggregation problem.4 Note that this solution to the time aggregation problem has the following advantages:

  • (i) The quantity aggregate is intuitively plausible, being the total quantity of the narrowly defined products sold by establishments during the time period under consideration, and

  • (ii) The product of the price times quantity equals the total revenue or value sold by the establishment during the time period under consideration.

This solution will be adopted to the time aggregation problem as a valid concept for the price and quantity at this first stage of aggregation.

20.11 Having decided on an appropriate theoretical definition of price and quantity for a product at the very lowest level of aggregation (that is, a narrowly defined unit value and the total quantity sold of that product by the individual establishment), it is now necessary to consider how to aggregate these narrowly defined elementary prices and quantities into an overall elementary aggregate. Suppose that there are M lowest-level items, or specific products, in this chosen elementary category. Denote the period t quantity of product m by qmt and the corresponding time aggregated unit value by pmt for t = 0,1 and for products m = 1,2,…,M. Define the period t quantity and price vectors as qt ≡ [q1t, q2t,…, qMt] and pt ≡ [p1t, p2t,…, pMt] for t = 0,1. It is now necessary to choose a theoretically ideal index number formula P(p0, p1, q0, q1) that will aggregate the individual product prices into an overall aggregate price relative for the M products in the chosen elementary aggregate. However, this problem of choosing a functional form for P(p0, p1, q0, q1) is identical to the overall index number problem that was addressed in Chapters 15 to 17. In these chapters, four different approaches to index number theory were studied that led to specific index number formulas as being best from each perspective. From the viewpoint of fixed-basket approaches, it was found that the Fisher (1922) and Walsh (1901) price indexes, PF and PW, appeared to be best. From the viewpoint of the test approach, the Fisher index appeared to be best. From the viewpoint of the stochastic approach to index number theory, the Törnqvist- Theil (Theil, 1967) index number formula PT emerged as being best. Finally, from the viewpoint of the economic approach to index number theory, the Walsh price index PW, the Fisher ideal index PF, and the Törnqvist-Theil index number formula PT were all regarded as being equally desirable. It also was shown that the same three index number formulas numerically approximate each other very closely, so it will not matter very much which of these alternative indexes is chosen.5 Hence, the theoretically ideal elementary index number formula is taken to be one of the three formulas PF(p0, p1, q0, q1), PW(p0, p1, q0, q1), or PT(p0, p1, q0, q1), where the period t quantity of product m, qmt, is the total quantity of that narrowly defined product produced by the establishment during period t, and the corresponding price for product m is pmt, the time aggregated unit value for t = 0,1 and for products m = 1,....,M.

20.12 In the following section, various practical elementary price indices will be defined. These indices do not have quantity weights and thus are functions only of the price vectors p0 and p1, which contain time aggregated unit values for the M products in the elementary aggregate for periods 0 and 1. Thus, when a practical elementary index number formula, say, PE(p0, p1), is compared with an ideal elementary price index, say, the Fisher price index PF(p0, p1, q0, q1), then obviously PE will differ from PF because the prices are not weighted according to their economic importance in the practical elementary formula. Call this difference between the two index number formulas formula approximation error.

20.13 Practical elementary indices are subject to two other types of error:

  • The statistical agency may not be able to collect information on all M prices in the elementary aggregate; that is, only a sample of the M prices may be collected. Call the resulting divergence between the incomplete elementary aggregate and the theoretically ideal elementary index the sampling error.

  • Even if a price for a narrowly defined product is collected by the statistical agency, it may not be equal to the theoretically appropriate time aggregated unit value price. This use of an inappropriate price at the very lowest level of aggregation gives rise to time aggregation error.

20.14 In Section G, a sampling framework for the collection of prices that can reduce the above three types of error will be discussed. In Section C, the five main elementary index number formulas are defined, and in Section D, various numerical relationships between these five indices are developed. Sections E and F develop the axiomatic and economic approaches to elementary indices, and the five main elementary formulas used in practice will be evaluated in light of these approaches.

C. Elementary Indices Used in Practice

20.15 Suppose that there are M lowest-level products or specific products in a chosen elementary category. Denote the period t price of product m by pmt for t = 0,1 and for products m = 1,2,…,M. Define the period t price vector as pt ≡[p1t, p2t,…, pMt] for t = 0,1.

20.16 The first widely used elementary index number formula is from the French economist Dutot (1738):

Thus the Dutot elementary price index is equal to the arithmetic average of the M period 1 prices divided by the arithmetic average of the M period 0 prices.

20.17 The second widely used elementary index number formula is from the Italian economist Carli (1804):

Thus the Carli elementary price index is equal to the arithmetic average of the M product price ratios or price relatives, pm1pm0.

20.18 The third widely used elementary index number formula is from the English economist Jevons (1863):

Thus the Jevons elementary price index is equal to the geometric average of the M product price ratios or price relatives, pm1pm0.

20.19 The fourth elementary index number formula PH is the harmonic average of the M product price relatives, and it was first suggested in passing as an index number formula by Jevons (1865, p. 121) and Coggeshall (1887):

20.20 Finally, the fifth elementary index number formula is the geometric average of the Carli and harmonic formulas; that is, it is the geometric mean of the arithmetic and harmonic means of the M price relatives:

This index number formula was first suggested by Fisher (1922, p. 472) as his formula 101. Fisher also observed that, empirically for his data set, PCSWD was very close to the Jevons index PJ, and these two indices were his best unweighted index number formulas. In more recent times, Carruthers, Sellwood, and Ward (1980, p. 25) and Dalén (1992a, p. 140) also proposed PCSWD as an elementary index number formula.

20.21 Having defined the most commonly used elementary formulas, the question now arises: which formula is best? Obviously, this question cannot be answered until desirable properties for elementary indices are developed. This will be done in a systematic manner in Section E, but in the present section, one desirable property for an elementary index will be noted: the time reversal test, noted in Chapter 15. In the present context, this test for the elementary index P(p0, p1) becomes

20.22 This test says that if the prices in period 2 revert to the initial prices of period 0, then the product of the price change going from period 0 to 1, P(p0, p1), times the price change going from period 1 to 2, P(p1, p0), should equal unity; that is, under the stated conditions, the index should end up where it started. It can be verified that the Dutot; Jevons; and Carruthers, Sellwood, and Ward indices, PD, PJ, and PCSWD, all satisfy the time reversal test, but the Carli and harmonic indices, PC and PH, fail this test. In fact, these last two indices fail the test in the following biased manner:

with strict inequalities holding in formulas (20.7) and (20.8), provided that the period 1 price vector p1 is not proportional to the period 0 price vector p0.6 Thus the Carli index will generally have an upward bias while the harmonic index will generally have a downward bias. Fisher (1922, pp. 66 and 383) seems to have been the first to establish the upward bias of the Carli index,7 and he made the following observations on its use by statistical agencies:

  • In fields other than index numbers it is often the best form of average to use. But we shall see that the simple arithmetic average produces one of the very worst of index numbers. And if this book has no other effect than to lead to the total abandonment of the simple arithmetic–type of index number, it will have served a useful purpose (Irving Fisher, 1922, pp. 29–30).

20.23 In the following section, some numerical relationships between the five elementary indices defined in this section will be established. Then, in the subsequent section, a more comprehensive list of desirable properties for elementary indices will be developed, and the five elementary formulas will be evaluated in light of these properties or tests.

D. Numerical Relationships Between the Frequently Used Elementary Indices

20.24 It can be shown8 that the Carli, Jevons, and harmonic elementary price indices satisfy the following inequalities:

that is, the harmonic index is always equal to or less than the Jevons index, which in turn is always equal to or less than the Carli index. In fact, the strict inequalities in formula (20.9) will hold, provided that the period 0 vector of prices, p0, is not proportional to the period 1 vector of prices, p1.

20.25 The inequalities in formula (20.9) do not tell us by how much the Carli index will exceed the Jevons index and by how much the Jevons index will exceed the harmonic index. Hence, in the remainder of this section, some approximate relationships among the five indices defined in the previous section will be developed, which will provide some practical guidance on the relative magnitudes of each of the indices.

20.26 The first approximate relationship derived is between the Carli index PC and the Dutot index PD. For each period t, define the arithmetic mean of the M prices pertaining to that period as follows:

Now define the multiplicative deviation of the mth price in period t relative to the mean price in that period, etm, as follows:

Note that formula (20.10) and formula (20.11) imply that the deviations em t sum to zero in each period; that is,

Note that the Dutot index can be written as the ratio of the mean prices, p1*p0*; that is

Now substitute formula (20.11) into the definition of the Jevons index, formula (20.3):

using formula (20.13),

where et ≡ [e1t,…, emt] for t = 0 and 1, and the function f is defined as follows:

Expand f(e0, e1) by a second-order Taylor series approximation around e0 = 0M and e1 = 0M. Using formula (20.12), it can be verified9 that the following second-order approximate relationship between PJ and PD results:

where var(et) is the variance of the period t multiplicative deviations; that is, for t = 0,1:

since et* = 0 using equation (20.12) =(1M)etet.

20.27 Under normal conditions,10 the variance of the deviations of the prices from their means in each period is likely to be approximately constant, and so under these conditions, the Jevons price index will approximate the Dutot price index to the second order. With the exception of the Dutot formula, the remaining four elementary indices defined in Section C are functions of the relative prices of the M products being aggregated. This fact is used to derive some approximate relationships between these four elementary indices. Thus define the mth price relative as

20.28 Define the arithmetic mean of the m price relatives as

where the last equality follows from the definition of formula (20.2) of the Carli index. Finally, define the deviation em of the mth price relative rm from the arithmetic average of the M price relatives r* as follows:

20.29 Note that formula (20.19) and formula (20.20) imply that the deviations em sum to zero; that is,

Now substitute formula (20.20) into the definitions of PC, PJ, PH, and PCSWD, formulas (20.2) to (20.5), to obtain the following representations for these indices in terms of the vector of deviations,e ≡ [e1,…, em]:

where the last equation in formulas (20.22) to (20.25) serves to define the deviation functions, fC(e), fJ(e), fH(e), and fCSW(e). The second-order Taylor series approximations to each of these functions around the point e = 0M are

where repeated use is made of formula (20.21) in deriving the above approximations.11 Thus to the second order, the Carli index PC will exceed the Jevons and Carruthers, Sellwood, and Ward indices, PJ and PCSWD, by (½)r*var(e), which is onehalf of the variance of the M price relatives pm1/pm0. Much like the second order, the harmonic index PH will lie below the Jevons and Carruthers, Sellwood, and Ward indices, PJ and PCSWD, by one-half of the variance of the M price relatives p1*p0*

20.30 Thus, empirically, it is expected that the Jevons and Carruthers, Sellwood, and Ward indices will be very close to each other. Using the previous approximation result formula (20.16), it is expected that the Dutot index PD also will be fairly close to PJ and PCSWD, with some fluctuations over time because of changing variances of the period 0 and 1 deviation vectors e0 and e1. Thus, it is expected that these three elementary indices will give similar numerical answers in empirical applications. On the other hand, the Carli index can be expected to be substantially above these three indices, with the degree of divergence growing as the variance of the M price relatives grows. Similarly, the harmonic index can be expected to be substantially below the three middle indices, with the degree of divergence growing as the variance of the M price relatives grows.

E. The Axiomatic Approach to Elementary Indices

20.31 Recall that in Chapter 16, the axiomatic approach to bilateral price indices, P(p0, p1, q0, q1), was developed. In the present chapter, the elementary price index P(p0, p1) depends only on the period 0 and 1 price vectors, p0 and p1, not on the period 0 and 1 quantity vectors, q0 and q1. One approach to obtaining new tests (T) or axioms for an elementary index is to look at the 20 or so axioms listed in Chapter 16 for bilateral price indices P(p0, p1, q0, q1), and adapt those axioms to the present context; that is, use the old bilateral tests for P(p0, p1, q0, q1) that do not depend on the quantity vectors q0 and q1 as tests for an elementary index P(p0, p1).12

20.32 The first eight tests or axioms are reasonably straightforward and uncontroversial:

T1: Continuity: P(p0, p1) is a continuous function of the M positive period 0 pricesp0 ≡ [p10,…, pM0] and the M positive period 1 pricesp1 ≡ [p11,…, pM1].

T2: Identity: P(p, p) = 1; that is, if the period 0 price vector equals the period 1 price vector, then the index is equal to unity.

T3: Monotonicity in Current-Period Prices: P(p0, p1)<P(p0, p) if p1 < p; that is, if any period 1 price increases, then the price index increases.

T4: Monotonicity in Base-Period Prices:P(p0, p1) > P(p, p1) if p0<p; that is, if any period 0 price increases, then the price index decreases.

T5: Proportionality in Current-Period Prices: P(p0,λp1) = λP(p0, p1) if λ > 0; that is, if all period 1 prices are multiplied by the positive number λ, then the initial price index is also multiplied by λ.

T6: Inverse Proportionality in Base-Period Prices: P(λp0, p1) = λ-1 P(p0, p1) if λ > 0; that is, if all period 0 prices are multiplied by the positive number λ, then the initial price index is multiplied by 1/λ.

T7: Mean Value Test: minm{pm1pm0:m=1,,M}P(p0,p1)maxm{pm1pm0:m=1,,M} that is, the price index lies between the smallest and largest price relatives.

T8: Symmetric Treatment of Establishments/Products: P(p0, p1) = P(p0*, p1*), where p0* and p1* denote the same permutation of the components of p0 and p1; that is, if there is a change in ordering of the establishments from which the price quotations (or products within establishments) are obtained for the two periods, then the elementary index remains unchanged.

20.33Eichhorn (1978, p. 155) showed that tests T1, T2, T3, and T5 imply T7, so that not all of the above tests are logically independent. The following tests are more controversial and are not necessarily accepted by all price statisticians.

T9: The Price-Bouncing Test: P(p0, p1) = P(p0*, p1**), where p0* and p1** denote possibly different permutations of the components of p0 and p1; that is, if the ordering of the price quotes for both periods is changed in possibly different ways, then the elementary index remains unchanged.

20.34 Obviously, test T8 is a special case of test T9, where in test T8 the two permutations of the initial ordering of the prices are restricted to be the same. Thus test T9 implies test T8. Test T9 is from Dalén (1992a, p. 138), who justified this test by suggesting that the price index should remain unchanged if outlet (for CPIs) prices “bounce” in such a manner that the outlets are just exchanging prices with each other over the two periods. While this test has some intuitive appeal, it is not consistent with the idea that outlet prices should be matched to each other in a one-to-one manner across the two periods. If elementary aggregates contain thousands of individual products that differ not only by outlet, there still is less reason to maintain this test.

20.35 The following test was also proposed by Dalén (1992a) in the elementary index context:

T10: Time Reversal:P(p1, p0) =1/P(p0, p1); that is, if the data for periods 0 and 1 are interchanged, then the resulting price index should equal the reciprocal of the original price index.

20.36 Since many price statisticians approve of the Laspeyres price index in the bilateral index context, and this index does not satisfy the time reversal test, it is obvious that not all price statisticians would regard the time reversal test in the elementary index context as being a fundamental test that must be satisfied. Nevertheless, many other price statisticians do regard this test as fundamental, since it is difficult to accept an index that gives a different answer if the ordering of time is reversed.

T11: Circularity: P(p0, p1)P(p1, p2) = P(p0, p2); that is, the price index going from period 0 to 1, times the price index going from period 1 to 2, equals the price index going from period 0 to 2 directly.

20.37 The circularity and identity tests imply the time reversal test (just set p2 = p0). Thus, the circularity test is essentially a strengthening of the time reversal test, so price statisticians who did not accept the time reversal test are unlikely to accept the circularity test. However, if there are no obvious drawbacks to accepting the circularity test, it would seem to be a very desirable property: it is a generalization of a property that holds for a single price relative.

T12: Commensurability:

P(λ1p10,,λMpM0;λ1p11,,λMpM1)=P(p10,,pM0;p11,,pM1)=P(p0,p1)forallλ1>0,,λM>0;

that is, if the units of measurement for each product in each establishment are changed, then the elementary index remains unchanged.

20.38 In the bilateral index context, virtually every price statistician accepts the validity of this test. However, in the elementary context, this test is more controversial. If the M products in the elementary aggregate are homogeneous, then it makes sense to measure all of the products in the same units. The very essence of homogeneity is that quantities can be added up in an economically meaningful way. Hence, if the unit of measurement is changed, then test T12 should restrict all of the λm to be the same number (say, λ) and the test T12 becomes

This modified test T12 will be satisfied if tests T5 and T6 are satisfied. Thus, if the products in the elementary aggregate are very homogeneous, then there is no need for test T12.

20.39 However, in actual practice, there usually will be thousands of individual products in each elementary aggregate, and the hypothesis of product homogeneity is not warranted. Under these circumstances, it is important that the elementary index satisfy the commensurability test, since the units of measurement of the heterogeneous products in the elementary aggregate are arbitrary and hence the price statistician can change the index simply by changing the units of measurement for some of the products.

20.40 This completes the listing of the tests for an elementary index. There remains the task of evaluating how many tests each of the five elementary indices defined in Section C passed.

20.41 The Jevons elementary index, PJ, satisfies all of the tests, and hence emerges as being best from the viewpoint of the axiomatic approach to elementary indices.

20.42 The Dutot index, PD, satisfies all of the tests with the important exception of the commensurability test T12, which it fails. Heterogeneous products in the elementary aggregate constitute a rather serious failure, and price statisticians should be careful in using this index under these conditions.

20.43 The geometric mean of the Carli and harmonic elementary indices, PCSWD, fails only the price-bouncing test T9 and the circularity test T11. The failure of these two tests is probably not a fatal failure, so this index could be used by price statisticians if, for some reason, they decided not to use the Jevons formula. It particularly would be suited to those who favor the test approach for guidance in choosing an index formula. As observed in Section D, numerically, PCSWD will be very close to PJ.

20.44 The Carli and harmonic elementary indices, PC and PH, fail the price-bouncing test T9, the time reversal test T10, and the circularity test T11, and pass the other tests. The failure of tests T9 and T11 is not a fatal failure, but the failure of the time reversal test T10 is rather serious, so price statisticians should be cautious in using these indices.

F. The Economic Approach to Elementary Indices

20.45 Recall the notation and discussion in Section B. First, it is necessary to recall some of the basics of the economic approach from Chapter 17. This allowed the aggregator functions representing the producing technology and the behavioral assumptions of the economic agents implicit in different formulas to be identified. The more realistic these were, the more support was given to the corresponding index number formula. The economic approach helps identify what the target index should be.

20.46 Suppose that each establishment producing products in the elementary aggregate has a set of inputs, and the linearly homogeneous aggregator function f(q) describes what output vector q ≡ [q1,…, qM] can be produced from the inputs. Further assume that each establishment engages in revenue-maximizing behavior in each period. Then, as was seen in Chapter 17, it can be shown that that certain specific functional forms for the aggregator f(q) or its dual unit revenue function R(p)13 lead to specific functional forms for the price index, P(p0, p1, q0, q1), with

20.47 Suppose that the establishments have aggregator functions f defined as follows:14

where the αm are positive constants. Then under these assumptions, it can be shown that equation (20.31) becomes15

and the quantity vector of products produced during the two periods must be proportional; that is,

20.48 From the first equation in formula (20.33), it can be seen that the true output price index, R(p1) /R(p0), under assumptions of formula (20.32) about the aggregator function f, is equal to the Laspeyres price index,PL(p0, p1, q0, q1) ≡ p1.q0 /p0.q0. The Paasche formula PP(p0, p1, q0, q1) ≡ p1q1/p0q1 is equally justified under formula (20.34).

20.49 Formula (20.32) on f thus justifies the Laspeyres and Paasche indices as being the “true” elementary aggregate from the economic approach to elementary indices. Yet this is a restrictive assumption, at least from an economic viewpoint, that relative quantities produced do not vary with relative prices. Other less restrictive assumptions on technology can be made. For example, as shown in Section B.3, Chapter 17, certain assumptions on technology justify the Törnqvist price index, PT, whose logarithm is defined as

20.50 Suppose now that product revenues are proportional for each product over the two periods so that

Under these conditions, the base-period revenue shares sm0 will equal the corresponding period 1 revenue shares sm1, as well as the corresponding β(m); that is, formula (20.36) implies

Under these conditions, the Törnqvist index reduces to the following weighted Jevons index:

20.51 Thus, if the relative prices of products in a Jevons index are weighted using weights proportional to base-period (which equals current-period) revenue shares in the product class, then the Jevons index defined by equation (20.38) is equal to the following approximation to the Törnqvist index:

20.52 In Section G, the sampling approach shows how, under various sample designs, elementary index number formulas have implicit weighting systems. Of particular interest are sample designs where products are sampled with probabilities proportionate to quantity or revenue shares in either period. Under such circumstances, quantity weights are implicitly introduced, so that the sample elementary index is an estimate of a population-weighted index. The economic approach then provides a basis for deciding whether the economic assumptions underlying the resulting population estimates are reasonable. For example, the above results show that the sample Jevons elementary index can be justified as an approximation to an underlying Törnqvist price index for a homogeneous elementary aggregate under a price sampling scheme with probabilities of selection proportionate to base-period revenue shares.

20.53 Two assumptions have been outlined here: the assumption that the quantity vectors pertaining to the two periods under consideration are proportional, formula (20.34), and the assumption that revenues are proportional over the two periods, formula (20.36).

20.54 The choice between formulas depends not only on the sample design used, but also on the relative merits of the proportional quantities versus proportional revenues assumption. Similar considerations apply to the economic theory of the CPI (or an intermediate input PPI), except that the aggregator function describes the preferences of a cost-minimizing purchaser. In this context, index number theorists have debated the relative merits of the proportional quantities versus proportional expenditures assumption for a long time. Authors who thought that the proportional expenditures assumption was more likely empirical include Jevons (1865, p. 295) and Ferger (1931, p. 39; 1936, p. 271). These early authors did not have the economic approach to index number theory at their disposal, but they intuitively understood, along with Pierson (1895, p. 332), that substitution effects occurred and, hence, the proportional expenditures assumption was more plausible than the proportional quantities assumption. This is because cost-minimizing consumers will purchase fewer sampled products with above-average price increases; the quantities can be expected to fall rather than remain constant. Such a decrease in quantities combined with the increase in price makes the assumption of constant expenditures more tenable. However, this is for the economic theory of CPIs. In Chapter 17, the economic theory of PPIs argued that revenue-maximizing establishments will produce more sampled products with above-average price increases, making assumptions of constant revenues less tenable. However, the theory presented in Chapter 17 also indicated that technical progress was a complicating factor largely absent in the consumer context.

20.55 If quantities supplied move proportionally over time, then this is consistent with a Leontief technology, and the use of a Laspeyres index is perfectly consistent with the economic approach to the output price index. On the other hand, if the probabilities used for sampling of prices for the Jevons index are taken to be the arithmetic average of the period 0 and 1 product revenue shares, and narrowly defined unit values are used as the price concept, then the weighted Jevons index becomes an ideal type of elementary index discussed in Section B. In general, the biases introduced by the use of an unweighted formula cannot be assessed accurately unless information on weights for the two periods is somehow obtained.

G. The Sampling Approach to Elementary Indices

20.56 It can now be shown how various elementary formulas can estimate this Laspeyres formula under alternative assumptions about the sampling of prices.

20.57 To justify the use of the Dutot elementary formula, consider the expected value of the Dutot index when sampling with base-period product inclusion probabilities equal to the sales quantities of product m in the base period relative to total sales quantities of all products in the product class in the base period. Assume that these definitions require that all products in the product class have the same units.16

20.58 The expected value of the sample Dutot index is17

which is the familiar Laspeyres index,

20.59 Now it is easy to see how this sample design could be turned into a rigorous sampling framework for sampling prices in the particular product class under consideration. If product prices in the product class were sampled proportionally to their base-period probabilities, then the Laspeyres index formula (20.41) could be estimated by a probability-weighted Dutot index, where the probabilities are defined by their base-period quantity shares. In general, with an appropriate sampling scheme, the use of the Dutot formula at the elementary level of aggregation for homogeneous products can be perfectly consistent with a Laspeyres index concept. Put otherwise, under this sampling design, the expectation of the sample Dutot is equal to the population Laspeyres.

20.60 The Dutot formula also can be consistent with a Paasche index concept at the elementary level of aggregation. If sampling is with period 1 item inclusion probabilities, the expectation of the sample Dutot is equal to

which is the familiar Paasche formula,

20.61 Put otherwise, under this sampling design, the expectation of the sample Dutot is equal to the population Paasche index. Again, it is easy to see how this sample design could be turned into a rigorous sampling framework for sampling prices in the particular product class under consideration. If product prices in the product class were sampled proportionally to their period 1 probabilities, then the Paasche index formula (20.43) could be estimated by the probability-weighted Dutot index. In general, with an appropriate sampling scheme, the use of the Dutot formula at the elementary level of aggregation (for a homogeneous elementary aggregate) can be perfectly consistent with a Paasche index concept.18

20.62 Rather than use the fixed-basket representations for the Laspeyres and Paasche indexes, the revenue-share representations for the Laspeyres and Paasche indexes could be used along with the revenue shares sm0 or sm1 as probability weights for price relatives. Under sampling proportional to base-period revenue shares, the expectation of the Carli index is

which is the population Laspeyres index. Of course, formula (20.44) does not require the assumption of homogeneous products as did formula (20.40) and formula (20.42). On the other hand, one can show analogously that under sampling proportional to period 1 revenue shares, the expectation of the reciprocal of the sample harmonic index is equal to the reciprocal of the population Paasche index, and thus that the expectation of the sample harmonic index,

will be equal to the Paasche index.

20.63 The above results show that the sample Dutot elementary index can be justified as an approximation to an underlying population Laspeyres or Paasche price index for a homogeneous elementary aggregate under appropriate price sampling schemes. The above results also show that the sample Carli and harmonic elementary indexes can be justified as approximations to an underlying population Laspeyres or Paasche price index for a heterogeneous elementary aggregate under appropriate price sampling schemes.

20.64 Thus, if the relative prices of products in the product class under consideration are sampled using weights that are proportional to the arithmetic average of the base- and current-period revenue shares in the product class, then the expectation of this sample Jevons index is equal to the population Törnqvist index formula (20.35).

20.65Sample elementary indices sampled under appropriate probability designs were capable of approximating various population economic elementary indices, with the approximation becoming exact as the sampling approached complete coverage. Conversely, it can be seen that, in general, it will be impossible for a sample elementary price index, of the type defined in Section C, to provide an unbiased estimate of the theoretical population ideal elementary price index defined in Section B, even if all product prices in the elementary aggregate were sampled. Hence, rather than just sampling prices, it will be necessary for the price statistician to collect information on the transaction values (or quantities) associated with the sampled prices to form sample elementary aggregates that will approach the target ideal elementary aggregate as the sample size becomes large. Thus instead of just collecting a sample of prices, it will be necessary to collect corresponding sample quantities (or values) so that a sample Fisher, Térnqvist, or Walsh price index can be constructed. This sample-based superlative elementary price index will approach the population ideal elementary index as the sample size becomes large. This approach to the construction of elementary indices in a sampling context was recommended by Pigou (1924, pp. 66–7), Fisher (1922, p. 380), Diewert (1995a, p. 25), and Balk (2002).19 In particular, Pigou (1924, p. 67) suggested that the sample-based Fisher ideal price index be used to deflate the value ratio for the aggregate under consideration to obtain an estimate of the quantity ratio for the aggregate under consideration.

20.66 Until fairly recently, it was not possible to determine how close an unweighted elementary index, defined in Section C, was to an ideal elementary aggregate. However, with the availability of scanner data (that is, of detailed data on the prices and quantities of individual products that are sold in retail outlets), it has been possible to compute ideal elementary aggregates for some product strata and compare the results with statistical agency estimates of price change for the same class of products. Of course, the statistical agency estimates of price change usually are based on the use of the Dutot, Jevons, or Carli formulas. These studies relate to CPIs, the data collected from the bar-code readers of retail outlets. But the concern here is with the discrepancy between unweighted and weighted indices used at this elementary aggregate level, and the discrepancies are sufficiently large to merit highlighting in this PPI context. The following quotations summarize many of these scanner data studies:

  • A second major recent development is the willingness of statistical agencies to experiment with scanner data, which are the electronic data generated at the point of sale by the retail outlet and generally include transactions prices, quantities, location, date and time of purchase and the product described by brand, make or model. Such detailed data may prove especially useful for constructing better indexes at the elementary level. Recent studies that use scanner data in this way include Silver (1995), Reinsdorf (1996), Bradley, Cook, Leaver and Moulton (1997), Dalén (1997), de Haan and Opperdoes (1997) and Hawkes (1997). Some estimates of elementary index bias (on an annual basis) that emerged from these studies were: 1.1 percentage points for television sets in the United Kingdom; 4.5 percentage points for coffee in the United States; 1.5 percentage points for ketchup, toilet tissue, milk and tuna in the United States; 1 percentage point for fats, detergents, breakfast cereals and frozen fish in Sweden; 1 percentage point for coffee in the Netherlands and 3 percentage points for coffee in the United States respectively. These bias estimates incorporate both elementary and outlet substitution biases and are significantly higher than our earlier ballpark estimates of .255 and .41 percentage points. On the other hand, it is unclear to what extent these large bias estimates can be generalized to other commodities (Diewert, 1998a, pp. 54–55).

  • Before considering the results it is worth commenting on some general findings from scanner data. It is stressed that the results here are for an experiment in which the same data were used to compare different methods. The results for the U.K. Retail Prices Index can not be fairly compared since they are based on quite different practices and data, their data being collected by price collectors and having strengths as well as weaknesses (Fenwick, Ball, Silver and Morgan (2002)). Yet it is worth following up on Diewert’s (2002c) comment on the U.K. Retail Prices Index electrical appliances section, which includes a wide variety of appliances, such as irons, toasters, refrigerators, etc. which went from 98.6 to 98.0, a drop of 0.6 percentage points from January 1998 to December 1998. He compares these results with those for washing machines and notes that “..it may be that the non washing machine components of the electrical appliances index increased in price enough over this period to cancel out the large apparent drop in the price of washing machines but I think that this is somewhat unlikely.” A number of studies on similar such products have been conducted using scanner data for this period. Chained Fishers indices have been calculated from the scanner data, (the RPI (within year) indices are fixedbase Laspeyres ones), and have been found to fall by about 12 for televisions (Silver and Heravi, 2001a), 10 for washing machines, 7.5 for dishwashers, 15 for cameras and 5 for vacuum cleaners (Silver and Heravi, 2001b). These results are quite different from those for the RPI section and suggest that the washing machine disparity, as Diewert notes, may not be an anomaly. Traditional methods and data sources seem to be giving much higher rates for the CPI than those from scanner data, though the reasons for these discrepancies were not the subject of this study (Silver and Heravi, 2002, p. 25).

20.67 These quotations summarize the results of many elementary aggregate index number studies based on the use of scanner data. These studies indicate that when detailed price and quantity data are used to compute superlative indexes or hedonic indexes for an expenditure category, the resulting measures of price change are often below the corresponding official statistical agency estimates of price change for that category. Sometimes the measures of price change based on the use of scanner data are considerably below the corresponding official measures.20 These results indicate that there may be large gains in the precision of elementary indices if a weighted sampling framework is adopted.

20.68 Is there a simple intuitive explanation for the above empirical results? The empirical work is on CPIs, and the behavioral assumptions relate to such indices, though they equally apply to input PPIs. Furthermore, the analysis can be undertaken readily based on the behavioral assumptions underlying output PPIs, its principles being more important. A partial explanation may be possible by looking at the dynamics of product demand. In any market economy, firms and outlets sell products that are either declining or increasing in price. Usually, the products that decline in price experience an increase in sales. Thus, the expenditure shares associated with products declining in price usually increase, and the reverse is true for products increasing in price. Unfortunately, elementary indices cannot pick up the effects of this negative correlation between price changes and the induced changes in expenditure shares, because elementary indices depend only on prices and not on expenditure shares.

20.69 An example can illustrate this point. Suppose that there are only three products in the elementary aggregate, and that in period 0, the price of each product is pm0 = 1, and the expenditure share for each product is equal, so that sm0 = 1/3 for m = 1,2,3. Suppose that in period 1, the price of product 1 increases to p11 = 1 + i, the price of product 2 remains constant at p21 = 1, and the price of product 3 decreases to p31 = (1 + i)-1, where the product 1 rate of increase in price is i > 0. Suppose further that the expenditure share of product 1 decreases to s11=(⅓)−σ, where σ is a small number between 0 and ⅓, and the expenditure share of product 3 increases to s31=(⅓)+σ. The expenditure share of product 2 remains constant at s21=(⅓). The five elementary indices, defined in Section C, all can be written as functions of the product 1 inflation rate i (which is also the product 3 deflation rate) as follows:

20.70 Note that in this particular example, the Dutot index fD(i) turns out to equal the Carli index fC(i). The second-order Taylor series approximations to the five elementary indices formulas (20.46) to (20.50) are given by formulas (20.51) to (20.55) below:

Thus for small i, the Carli and Dutot indices will be slightly greater than 1,21 the Jevons and Carruthers, Sellwood, and Ward indices will be approximately equal to 1, and the harmonic index will be slightly less than 1. Note that the first-order Taylor series approximation to all five indices is 1; that is, to the accuracy of a first-order approximation, all five indices equal unity.

20.71 Now calculate the Laspeyres, Paasche, and Fisher indices for the elementary aggregate:

First-order Taylor series approximations to the above indices formulas (20.56) to (20.58) around i = 0 are given by formulas (20.59)–(20.61):

An ideal elementary index for the three products is the Fisher ideal index fF(i). The approximations in formulas (20.51) to (20.55) and formula (20.61) show that the Fisher index will lie below all five elementary indices by the amount σ i using firstorder approximations to all six indices. Thus all five elementary indices will have an approximate upward bias equal to σ i compared with an ideal elementary aggregate.

20.72 Suppose that the annual product inflation rate for the product rising in price is equal to 10 percent, so that i = .10 (and, hence, the rate of price decrease for the product decreasing in price is approximately 10 percent as well). If the expenditure share of the increasing price product declines by 5 percentage points, then σ = .05, and the annual approximate upward bias in all five elementary indices is σ i = .05 × .10 = .005 or one-half of a percentage point. If i increases to 20 percent and σ increases to 10 percent, then the approximate bias increases to σ i = .10 × .20 = .02, or 2 percent.

20.73 The above example is highly simplified, but more sophisticated versions of it are capable of explaining at least some of the discrepancy between official elementary indices and superlative indices calculated by using scanner data for an expenditure class. Basically, elementary indices defined without using associated quantity or value weights are incapable of picking up shifts in expenditure shares induced by fluctuations in product prices.22 To eliminate this problem, it will be necessary to sample values along with prices in both the base and comparison periods.

20.74 In the following section, a simple regression-based approach to the construction of elementary indices is outlined, and, again, the importance of weighting the price quotes will emerge from the analysis.

H. A Simple Stochastic Approach to Elementary Indices

20.75 Recall the notation used in Section B. Suppose the prices of the M products for period 0 and 1 are equal to the right-hand sides of formulas (20.62) and (20.63) below:

where α and the βm are positive parameters. Note that there are two M prices on the left-hand sides of equations (20.62) and (20.63) but only M + 1 parameters on the right-hand sides of these equations. The basic hypothesis in equations (20.62) and (20.63) is that the two price vectors p0 and p1 are proportional (with p1 = αp0, so that α is the factor of proportionality) except for random multiplicative errors, and, hence, α represents the underlying elementary price aggregate. If logarithms are taken of both sides of equations (20.62) and (20.63) and some random errors em0 and em1 added to the right-hand sides of the resulting equations, the following linear regression model results:

where

20.76 Note that equations (20.64) and (20.65) can be interpreted as a highly simplified hedonic regression model.23 The only characteristic of each product is the product itself. This model is also a special case of the country product dummy method for making international comparisons among the prices of different countries.24 A major advantage of this regression method for constructing an elementary price index is that standard errors for the index number α can be obtained. This advantage of the stochastic approach to index number theory was stressed by Selvanathan and Rao (1994).

20.77 It can be verified that the least-squares estimator for γ is

If γ* is exponentiated, then the following estimator for the elementary aggregate α is obtained:

where PJ(p0, p1) is the Jevons elementary price index defined in Section C above. Thus, there is a regression model-based justification for the use of the Jevons elementary index.

20.78 Consider the following unweighted least-squares model:

It can be verified that the γ solution to the unconstrained minimization problem (20.69) is the γ* defined by (20.67).

20.79 There is a problem with the unweighted least-squares model defined by formula (20.69): the logarithm of each price quote is given exactly the same weight in the model, no matter what the revenue on that product was in each period. This is obviously unsatisfactory, since a price that has very little economic importance (that is, a low revenue share in each period) is given the same weight in the regression model compared with a very important product. Thus, it is useful to consider the following weighted least-squares model:

where the period t revenue share on product m is defined in the usual manner as

Thus, in the model (20.70), the logarithm of each product price quotation in each period is weighted by its revenue share in that period.

20.80 The γ solution to (20.70) is

where

and h(a, b) is the harmonic mean of the numbers a and b. Thus γ** is a share-weighted average of the logarithms of the price ratios pm1/pm0. If γ** is exponentiated, then an estimator α** for the elementary aggregate α is obtained.

20.81 How does α** compare with the three ideal elementary price indices defined in Section B? It can be shown25 that α** approximates those three indices to the second order around an equal price and quantity point; that is, for most data sets, α** will be very close to the Fisher, Törnqvist, and Walsh elementary indices.

20.82 The results in this section provide some weak support for the use of the Jevons elementary index, but they provide much stronger support for the use of weighted elementary indices of the type defined in Section B above. The results in this section also provide support for the use of value or quantity weights in hedonic regressions.

I. Conclusions

20.83 The main results in this chapter can be summarized as follows:

  • (i) To define a “best” elementary index number formula, it is necessary to have a target index number concept. In Section B, it is suggested that normal bilateral index number theory applies at the elementary level as well as at higher levels, and hence the target concept should be one of the Fisher, Törnqvist, or Walsh formulas.

  • (ii) When aggregating the prices of the same narrowly defined product within a period, the narrowly defined unit value is a reasonable target price concept.

  • (iii) The axiomatic approach to traditional elementary indices (that is, no quantity or value weights are available) supports the use of the Jevons formula under all circumstances. If the products in the elementary aggregate are very homogeneous (that is, they have the same unit of measurement), then the Dutot formula can be used. In the case of a heterogeneous elementary aggregate (the usual case), the Carruthers, Sellwood, and Ward formula can be used as an alternative to the Jevons formula, but both will give much the same numerical answers.

  • (iv) The Carli index has an upward bias and the harmonic index has a downward bias.

  • (v) All five unweighted elementary indices are not really satisfactory. A much more satisfactory approach would be to collect quantity or value information along with price information and form sample superlative indices as the preferred elementary indices.

  • (vi) A simple hedonic regression approach to elementary indices supports the use of the Jevons formula. However, a more satisfactory approach is to use a weighted hedonic regression approach. The resulting index will closely approximate the ideal indices defined in Section B.

This chapter draws heavily on the recent contributions of Dalén (1992a), Balk (1994, 1998b, 2002) and Diewert (1995a, 2002a, 2002b).

See Balk (1994) for a similar approach.

Walsh explained his reasoning as follows: “Of all the prices reported of the same kind of article, the average to be drawn is the arithmetic; and the prices should be weighted according to the relative mass quantities that were sold at them (1901, p. 96). “Some nice questions arise as to whether only what is consumed in the country, or only what is produced in it, or both together are to be counted; and also there are difficulties as to the single price quotation that is to be given at each period to each commodity, since this, too, must be an average. Throughout the country during the period a commodity is not sold at one price, nor even at one wholesale price in its principal market. Various quantities of it are sold at different prices, and the full value is obtained by adding all the sums spent (at the same stage in its advance towards the consumer), and the average price is found by dividing the total sum (or the full value) by the total quantities” (1921a, p. 88).

See, for example, Szulc (1987, p. 13), Dalén (1992a, p. 135), Reinsdorf (1994b), Diewert (1995a, pp. 20–21), Reinsdorf and Moulton (1997), and Balk (2002).

Theorem 5 in Diewert (1978, p. 888) showed that PF, PT, and PW will approximate each other to the second order around an equal price and quantity point; see Diewert (1978, p. 894), R.J. Hill (2000), and Chapter 19, Section B, for some empirical results.

These inequalities follow from the fact that a harmonic mean of M positive numbers is always equal to or less than the corresponding arithmetic mean; see Walsh (1901, p. 517) or Fisher (1922, pp. 383–84). This inequality is a special case of Schlömilch’s Inequality; see Hardy, Littlewood, and Polyá (1934, p. 26).

See also Pigou (1924, pp. 59 and 70), Szulc (1987, p. 12), and Dalén (1992a, p. 139). Dalén (1994, pp. 150–51) provides some nice intuitive explanations for the upward bias of the Carli index.

Each of the three indices PH, PJ, and PC is a mean of order r where r equals −1, 0, and 1, respectively, and so the inequalities follow from Schlömilch’s inequality; see Hardy, Littlewood, and Polyá (1934, p. 26).

This approximate relationship was first obtained by Carruthers, Sellwood, and Ward (1980, p. 25).

If there are significant changes in the overall inflation rate, some studies indicate that the variance of deviations of prices from their means also can change. Also, if M is small, there will be sampling fluctuations in the variances of the prices from period to period

These second-order approximations are from Dalén (1992a, p. 143) for the case r* = 1 and Diewert (1995a, p. 29) for the case of a general r*.

This was the approach used by Diewert (1995a, pp. 5– 17), who drew on the earlier work of Eichhorn (1978, pp. 152–60) and Dalén (1992a).

The unit revenue function is defined as R(p) ≡ maxq{p.q : f(q) = 1}.

The preferences that correspond to this f are known as Leontief (1936) or no substitution preferences.

See (1983a).

The inclusion probabilities are meaningless unless the products are homogeneous.

There is a technical bias since E(x/y) is approximated by E(x)/E(y), but this will approach zero as m gets larger.

Of course, the Dutot index as an estimate of a population Paasche index will differ from the Dutot index as an estimate of a population Laspeyres index because of representativity or substitution bias.

Balk (2002) provides the details for this sampling framework.

However, scanner data studies do not always show large potential biases in official CPIs. Masato Okamoto of the National Statistics Center in Japan informed us in a personal communication that a large-scale internal study was undertaken. Using scanner data for about 250 categories of processed food and daily necessities collected over the period 1997 to 2000, it was found that the indices based on scanner data averaged only about 0.2 percentage points below the corresponding official indices per year. Japan uses the Dutot formula at the elementary level in its official CPI.

Recall the approximate relationship in formula (20.16) in Section C between the Dutot and Jevons indices. In the example, var(e0) = 0, whereas var(I1) > 0. This explains why the Dutot index is not approximately equal to the Jevons index in the example.

Put another way, elementary indices are subject to substitution or representativity bias.

See Chapters 7, 8, and 21 for material on hedonic regression models.

See Summers (1973). In our special case, there are only two “countries,” which are the two observations on the prices of the elementary aggregate for two periods.

Use the techniques discussed in Diewert (1978).

    Other Resources Citing This Publication