A. What the CPI actually measures

As consumers, most of us are aware of the day-to-day expenditures we make in order to live in a life style we can afford. One would think that keeping track of the cost of living would be fairly simple—compare this month’s expenses with last month’s and the change is the change in the cost of living. Immediately we begin questioning this approach because expenditures vary from month to month and we are not buying the same things through time. While there are standard items that we purchase regularly each month, there are also a number of items such as clothing and durable goods that we only purchase periodically. We begin looking at a longer period over which to gauge expenditures, say a year, but even then we note that there are changes Many durable goods such as automobiles and appliances are purchased less frequently, so we look at purchases over a longer period of two or three years.

The CPI uses a simplified approach to measuring changes in living costs. It takes expenditures for a fixed market basket of goods and services at some point in the past (called a base period) and estimates what it would cost today to purchase that same market basket. The market basket consists of total expenditures on consumption items by all urban consumers. It represents aggregate expenditures across the entire urban population; it is not necessarily representative of any particular consumer or group of consumers. BLS uses scientific sampling techniques to select specific items and to measure the change in prices of these items over time. The price changes in this sample provide a measure over time of the average change in prices of goods and services included in the market basket when purchased in the same quantities as in the base period. The CPI attempts to measure what it costs today to purchase the same market basket consumers purchased in the 1982-84 base period.

B. The CPI and a Cost-of-Living Index (CLI)

A true cost-of-living index (CLI) would be much more inclusive than the CPI. The primary conceptual difference between the two is that the CPI only measures the change in cost of market traded goods and services, while a CLI would extend household preferences to choices between work and leisure, and include the effects of government goods and services as well as the effects of the environment on consumer welfare. The CPI might be viewed as a subcomponent of the CLI that measures changes in pure prices of marketed goods and services. To measure pure price change, the quantities and quality of the items purchased must be kept fixed or the resulting measure will reflect changes in these two factors also. Thus, the CPI uses a Laspeyres index in which quantities are fixed during the base period. We can’t use current quantities because the data are not available for the current year. Even if we did have current quantities and wanted to use them instead of base period quantities, we would have to revise our indexes back through time to reflect the current quantity weights in previous periods to make the series comparable across periods. Otherwise, changes in the index would reflect both price and quantity changes.

A problem with the Laspeyres approach is that we don’t always buy the same quantities that were purchased in the base period. The economic theory of the cost of living, where consumers hold utility constant, is helpful in analyzing this situation. Consumers make purchases of goods and services to satisfy wants and needs for day-to-day living. There are a variety of goods and services they can purchase to meet this objective, but there is also the constraint of the income they have available. Rational consumers will make choices that give them the highest levels of satisfaction given their current income levels and the prices of goods and services that they can purchase. If consumers are indifferent between purchases of some items that yield the same level of satisfaction, they will substitute these items to choose the lowest cost combination. For example, if bananas, oranges and apples are types of fresh fruits I enjoy eating, I will purchase different amounts from period to period based on the relative prices of each. If bananas are $.50 per pound, apples $.25 per pound and oranges $.20 per pound, I may purchase larger quantities of oranges and apples than bananas. If prices change so that apples are $.50 per pound, bananas $.25 per pound and oranges $.20 per pound, I may buy more bananas and oranges than apples. If I originally purchased 1 pound of bananas, 2 pounds of apples and 2 pounds of oranges, my fresh fruit expenditures would have been $1.40 ($.50 + $.50 + $.40). If I now purchase one pound of apples and two pounds each of bananas and oranges to attain the same satisfaction, my fresh fruit expenditures are still the same. Using a fixed market basket approach, my estimated expenditures would be $1.65 ($1.00 for apples, $.25 for bananas and $.40 for oranges). This example illustrates the difference between a fixed market basket measure where quantities stay fixed and a cost of living approach where consumer satisfaction is fixed and quantities can vary in response to price change.

In addition, consumer purchasing patterns will change because our tastes for items change as styles or new fads emerge. Also, the preferences we once had for certain items become less important and our purchases of other items become more important. For example, consumers in general are preparing fewer meals at home and having more meals outside the home. For a wide variety of reasons—hectic schedules, the growth of multiple worker families, the variety and convenience of fast food and ethnic restaurants, to name a few—families eat outside the home more frequently than in the past. The result is that the expenditures that used to be for groceries are now being used for meals at restaurants. Even within grocery store items, the types of products purchased have shifted from basic foods to be prepared in the home to pre-packaged prepared food to be heated and served at home. The traditional household production function for food preparation prevalent in earlier decades has changed gradually over time.

As real incomes rise, consumer choices may also change. In the CPI we assume that as incomes rise, households will purchase more of the exact same items in proportion to their base period quantities. If we continue to price the same market basket with the fixed quantity shares of goods and services, the resulting measure may not reflect what consumers are currently purchasing and, therefore, does not reflect the current purchasing pattern that consumers experience. One would expect that as household incomes rise purchasing patterns would shift to reflect more expensive items that offer higher levels of satisfaction. Neither the CPI nor a CLI necessarily should reflect these changes since they assume consumers maintain the same standard of living. The CPI reflects the same standard of living by measuring the change in cost of the original market basket, while the CLI reflects the least cost combination of goods and services at today’s prices that maintain the same level of consumer satisfaction as the base period. As incomes rise, both measures assume that consumers spend the same proportion of their incomes on goods and services as they did in the base period.

C. Updating the market basket every year

As the forgoing analysis indicates, changes in consumer living costs are the result of direct price change, changes in quantities purchased due to changes in relative prices, changes due to tastes and preferences and changes in real income. The Laspeyres price index includes only the first factor. The CLI should also include the effects of changes in relative prices. The last two factors (changes in tastes and income) often are reflective of changes in consumer satisfaction which the CLI holds constant. Each time a new market basket is used, any of these factors can result in changes. The factors thought to be the most prevalent are direct price change and the change in relative prices, while the other two are assumed to be minimal. Any time there are changes in consumption patterns due to income and tastes, the Laspeyres or other indices used to approximate a CLI would no longer be keeping consumer utility constant. Thus, an index in which the market basket is updated annually may not reflect changes in the true cost of living either. For example, changes in consumer spending patterns between 1994 and 1995 may not reflect much in the way of changes in preferences and income, but changes in spending patterns between 1984 and 1995 would include significant differences due to preferences and income.

Another factor affecting annual updates is that when new weights are introduced, there is the possibility that such a shift could cause an upward drift in the index following the change. A good example of this might occur when petroleum-based energy prices decline for a period and then suddenly begin to rise after the new weights are introduced. When prices were declining, consumers used greater and greater quantities of fuel-oil for home heating and traveled more by automobile, using more gasoline. The new weights would reflect larger quantities of these commodities. If, after the new weights are introduced, petroleum prices begin to rise, the aggregate consumer price index will rise more rapidly than before since gasoline and fuel oil now have more importance.

D. The choice of index formula used to compute the CPI makes a difference in the results

Different formulas can deliver substantially different results when computing a price index. Consider the hypothetical information in Table 1 related to fresh fruits. Assume these represent actual purchases over a three year period, 1993-95. If we look only at prices, we observe that apples rise in price between 1993 and 1995 with the jump occurring in 1994. Banana prices decline and then return to their original price as do grapes. Oranges rise in price and return to their original price, while pears show a continuous decline in price. What has happened, in general to fresh fruit prices? The purpose of a price index is to try to represent the price experience for fresh fruits as a single number. To do this, quantities are held fixed and prices are allowed to vary. In this way, only prices change so that the index captures only price changes. If one compares expenditures over this period, it does not tell the true story about prices since expenditures are also affected by changes in quantities purchased. Consider two simple cases in which we use a fixed-weight index. First, one can compute an index in which the quantities purchased are held constant at the 1993 levels. Then one can compute how much it would cost in each consecutive year to purchase the same quantities. This is a Laspeyres price index. In 1993 the market basket of fresh fruit cost $70.00. It would cost $91.00 at 1994 prices to purchase the same amount of fresh fruit that was purchased in 1993 and $85.00 to purchase the same fruit in 1995. The Laspeyres price index in each year is the ratio of these expenditures expressed as a percent. The index for 1993 is 100.00 (70/70 × 100); the 1994 index is 130.00 (91/70 × 100); and the 1995 index is 121.43 (85/70 × 100). What this demonstrates is that, on average, prices for fresh fruits would have risen by 30 percent in 1994 if I purchased the same amount of each fruit as in 1993. The index for 1995 indicates that fruit prices, on average, would have been 21.4 percent higher than in 1993. It also indicates that prices in 1995 declined by 8.6 index points or 6.6 percent (8.6/130 × 100) relative to prices in 1994 using fixed 1993 quantities.

Table 1.

Alternative Price Indices

Table 1.

Alternative Price Indices

	1993			1994			1995
	Price/lb.	Quantity	Expenditures	Price/lb.	Quantity	Expenditures	Price/lb.	Quantity	Expenditures
Apples	$0.25	80	$20.00	$0.50	50	$25.00	$0.50	40	$20.00
Bananas	$0.50	30	$15.00	$0.25	60	$15.00	$0.50	40	$20.00
Oranges	$0.25	60	$15.00	$0.50	40	$20.00	$0.25	80	$20.00
Grapes	$1.00	10	$10.00	$0.75	20	$15.00	$1.00	20	$20.00
Pears	$0.50	20	$10.00	$0.30	50	$15.00	$0.25	80	$20.00
Fresh Fruit		200	$70.00		220	$90.00		260	$100.00
Expenditures at 1993 quantities			$ 70.00			$ 91.00			$ 85.00
Expenditures at 1994 quantities			$ 97.50			$ 90.00			$ 97.50
Expenditures at 1995 quantities			$ 110.00			$ 109.00			$ 100.00
	1993	1994	1995				Price Indices
	Qty.Share	Qty.Share	Qty.Share				1993	1994	1995
Apples	0.4000	0.2273	0.1538		Laspeyres (1993 base)		100.00%	130.00%	121.43%
Bananas	0.1500	0.2727	0.1538		Paasche (1993 base)		100.00%	92.31%	90.91%
Oranges	0.3000	0.1818	0.3077		Geometric Mean (‘93)		100.00%	108.76%	110.41%
Grapes	0.0500	0.0909	0.0769		Fisher		100.00%	109.54%	105.07%
Pears	0.1000	0.2273	0.3077		Tornqvist		100.00%	109.52%	105.08%
	1993	1994	1995				1993	1994	1995
	Exp.Share	Exp.Shar	Exp.Share		Chain Laspeyres		100.00%	130.00%	108.33%
Apples	0.2857	0.2778	0.2000		Chain Paashce		100.00%	92.31%	91.74%
Bananas	0.2143	0.1667	0.2000		Chain Geomean		100.00%	108.76%	97.93%
Oranges	0.2143	0.2222	0.2000		Chain Fisher		100.00%	109.54%	99.69%
Grapes	0.1429	0.1667	0.2000		Chain Tornqvist		100.00%	109.52%	100.01%
Pears	0.1429	0.1667	0.2000

View Table

Table 1.

Alternative Price Indices

	1993			1994			1995
	Price/lb.	Quantity	Expenditures	Price/lb.	Quantity	Expenditures	Price/lb.	Quantity	Expenditures
Apples	$0.25	80	$20.00	$0.50	50	$25.00	$0.50	40	$20.00
Bananas	$0.50	30	$15.00	$0.25	60	$15.00	$0.50	40	$20.00
Oranges	$0.25	60	$15.00	$0.50	40	$20.00	$0.25	80	$20.00
Grapes	$1.00	10	$10.00	$0.75	20	$15.00	$1.00	20	$20.00
Pears	$0.50	20	$10.00	$0.30	50	$15.00	$0.25	80	$20.00
Fresh Fruit		200	$70.00		220	$90.00		260	$100.00
Expenditures at 1993 quantities			$ 70.00			$ 91.00			$ 85.00
Expenditures at 1994 quantities			$ 97.50			$ 90.00			$ 97.50
Expenditures at 1995 quantities			$ 110.00			$ 109.00			$ 100.00
	1993	1994	1995				Price Indices
	Qty.Share	Qty.Share	Qty.Share				1993	1994	1995
Apples	0.4000	0.2273	0.1538		Laspeyres (1993 base)		100.00%	130.00%	121.43%
Bananas	0.1500	0.2727	0.1538		Paasche (1993 base)		100.00%	92.31%	90.91%
Oranges	0.3000	0.1818	0.3077		Geometric Mean (‘93)		100.00%	108.76%	110.41%
Grapes	0.0500	0.0909	0.0769		Fisher		100.00%	109.54%	105.07%
Pears	0.1000	0.2273	0.3077		Tornqvist		100.00%	109.52%	105.08%
	1993	1994	1995				1993	1994	1995
	Exp.Share	Exp.Shar	Exp.Share		Chain Laspeyres		100.00%	130.00%	108.33%
Apples	0.2857	0.2778	0.2000		Chain Paashce		100.00%	92.31%	91.74%
Bananas	0.2143	0.1667	0.2000		Chain Geomean		100.00%	108.76%	97.93%
Oranges	0.2143	0.2222	0.2000		Chain Fisher		100.00%	109.54%	99.69%
Grapes	0.1429	0.1667	0.2000		Chain Tornqvist		100.00%	109.52%	100.01%
Pears	0.1429	0.1667	0.2000

One can also produce an index based on current year purchasing patterns. In this case we calculate how much it would have cost to purchase the same quantity of fresh fruit bought in the current year at the prices prevailing in the comparison year. This is a Paasche price index. In 1994 the expenditures were $90 and it would have cost $97.50 at 1993 prices. Similarly, the fruit purchases in 1995 were $100.00, but would have cost $110.00 at 1993 prices. The Paasche price indices are 92.31 (90/97.50x100) in 1994 and 90.91 (100/110 × 100). This indicates that fresh fruit prices, on average, are lower today than in the two previous years and have been declining. Why is there such a difference between the Laspeyres and Paasche indices? It is basically the result of the different mix of fruits I bought in each period. This is seen when we examine the quantity shares for each period also presented in Table 1. In 1995, I purchased a larger amount of pears, which were more expensive in 1993, and fewer apples, which were less expensive. In 1993 apples represented 40 percent of the quantity of fresh fruit purchases and pears only 10 percent. In the Laspeyres index the doubling of apple prices between 1993 and 1995 receives 4 fold more weight than the 50 percent decline in pear prices. In 1995 pears represent 31 percent of the quantity of fresh fruits and apples only 15 percent. In the Paasche index the 50 percent decline in pear prices receives twice as much weight as the increase in apple prices.

The differences between the Laspeyres and Paasche price indexes also show that different economic assumptions are involved. The Laspeyres, with fixed quantity shares from the past, assumes that consumers want to purchase the same quantity levels as in the past and want to know how much more it would take to purchase that market basket today. The Paasche, with fixed quantity shares in the present, uses the current quantity levels and assumes consumers want to know how much today’s purchases cost compared to what they would have cost in the past.

Another possible alternative is to keep the expenditure shares rather than quantity shares fixed for each type of fruit. Such an alternative assumes that when prices change, consumers can shift the quantities of each type of fruit purchased but still maintain the same fixed proportions for each type of fruit within their budget. For example, if one plans to spend $20 on apples (from a budget of $70 for fresh fruit) and apple prices rise from $.25 per pound to $.50 per pound, then one purchases only 40 pounds of apples (20/. 50) instead of the original 80 pounds. Such behavior is the underlying assumption of the geometric mean formula for measuring price change.¹ The assumption of fixed expenditure shares at 1993 purchasing patterns and the use of the geometric mean price index results in yet another measure of price change. Between 1993 and 1994 prices rise by 8.76 percent with another rise in 1995 to a level 10.41 percent above 1993.

E. Measures that come close to the true change in the cost of living

Given these somewhat disparate results, one begins to question whether there is really any measure that can approximate what the average change in the cost of living with respect to fruit might be. The changes shown by the Laspeyres and the Paasche indices appear to be extreme. In research into the economic theory surrounding the cost of living by Robert Pollak (1989) has demonstrated that under certain assumptions the Laspeyres and Paasche indices are upper and lower bounds, respectively, to a true cost-of-living index.² Several works of W. Erwin Diewert (1976, 1989) have shown that a class of price indices, called superlative indices, are close approximations to the cost-of-living index. These indices incorporate both sets of consumer experience into the weighting structure of the price index. The two most frequently used superlative price indices are the Fisher Ideal and Tornqvist indices.³ The Fisher Ideal index (equation 8, Appendix I) is the geometric mean of the Laspeyres and Paasche indices. The Tornqvist index (equation 9) uses a geometric mean formula in which the weights are the average expenditure share (w₀ and w_t) over the two periods for which the index is computed. The results of using these two formulae are quite similar as seen in Table 1. The cost of living rises between 1993 and 1994 by 9.5 percent and in 1995 returns to a level about 5.1 percent higher than in 1993. These measures represent the state of the art in price index methodology. The advantage that these indices offer is that they incorporate the experiences of both the base period situation and the current period situation. This provides an intuitive understanding to their applicability beyond the formal mathematical solutions found in the literature.

F. Use of chain indices

Another alternative that is often considered is the use of chain indices. Fixed-base indices such as those just discussed have a common point of comparison to which all price measures are related. Chain indices enable users to employ the most recent expenditure weights and include new commodities or services more readily. The chain version of an index uses the basic index formula in its core computation. Equations 10 - 13 in Appendix I present formulae for the most commonly used chain indices. The price relative is based on the current period price (p_t) compared with a previous year price (p_t-1). The expenditure shares are for either the current year (w_t) or previous year (w_t-1) for Paasche and Laspeyres indices, respectively, and both years for the Tornqvist index. The chain Fisher index is computed directly as the geometric mean of the chain Laspeyres and chain Paasche indices. The major difference between fixed-base and chain indices is that the chain indices treat each year’s change independently from the previous year and link the result onto the previous year’s index (I_t-1).

The chain indices for the fresh fruit example show mixed results. The indices for 1995 are the first year in which chaining takes effect.⁴ The Laspeyres and Paasche index levels move closer together. However, all the chain indices, except the Paasche, show more drastic declines from 1994 than the fixed-base versions. This occurs because the chain indices are reflecting weight changes in addition to price changes.

A. Axiomatic approach to price index numbers

The axiomatic approach assumes that valid, representative prices and quantities are available for the required computations. Given that an index number formula is a mathematical function, what are the desirable statistical properties that make it a “good” aggregation function? The answer to this question provides a variety of index number axioms about how an index number should perform. Fisher (1922), Eichorn (1978), and Diewert (1987 and 1995) provide a number of such axioms or index number tests. These tests have been used to understand more fully the advantages and disadvantages of one index formula over another. A few of the most commonly used tests are the following.⁵

B. Economic theoretical approach to price index numbers

The economic theoretic approach to index numbers examines the underlying economic behavior of consumers and producers and evaluates the index formula based on appropriate models. Both consumer utility theory and producer production theory provide the basis for economic decisions.

In consumer utility theory, consumers will maximize the satisfaction they receive from the purchase of goods and services given the constraints of their household budgets. Samuelson (1983), Samuelson and Swamy (1974), Diewert (1976) and Pollak (1989) have presented various models of consumer behavior and the role of price indices in the measurement of the cost of living. In addition, consumers make choices that can be measured by expenditure surveys that cover periods when consumer markets are assumed to be in equilibrium. These surveys reflect the levels of utility consumers have revealed to be their preferences.

Production theory tells us that businesses will seek to maximize profits given a fixed set of resources and consumer demand. Fisher and Shell (1972) examine price index measurement from the perspective of the theory of the firm. Surveys and censuses of production and sales that are conducted over periods during which markets are assumed to be in equilibrium can provide information that reflect the optimizing decisions of businesses.

From a theoretical perspective, the Laspeyres price index serves as an upper bound in the measurement of the cost of living because it assumes purchases are made in fixed quantities based on the optimal decisions from some previous period’s experience. There is no opportunity to substitute products or services in response to more current economic conditions.

The Paasche index represents a lower bound on the cost of living because, in part, it assumes that purchases are made using current fixed quantities and this pattern was the same in the past. The Fisher price index and other superlative price indices that use purchasing patterns from both periods closely approximate the CLI for marketed goods and services.

A. Most common price indices

The most commonly used indices are the fixed weight indices where quantities are held constant for either some past period or the current period. The Laspeyres index (equation 1 in the Appendix) keeps the quantities (q) fixed for some period in the past (0). In its simplest form it is a ratio of what it cost today to purchase the same set of goods and services that were purchased in a specific previous period.

The Paasche index (equation 4) keeps the quantities fixed at their levels in the current period (t). In its simplest form it is the ratio of what today’s purchases cost compared to what they would have cost in a specific previous period.

The formulae used in practice are a bit more complex. The Laspeyres index (equations 2 and 3) is usually estimated using a price ratio (p_t/p₀) multiplied by a previous period expenditure share (p₀q₀/Σp₀q0) since most of the measurement systems in place are geared toward measuring some form of expenditures.

The Paasche index (equations 5 and 6), likewise, should be estimated using a formula with a price ratio multiplied by the current expenditure share. In this case, however, the expenditure share is based on current expenditures (p_tq_t/Σp_tq_t). To derive a formula so that current prices cancel in the numerator of the expenditure share portion of the formula with the denominator in the price ratio portion of the formula a reciprocal price ratio (p_o/p_t) is used. This also results in the formula used becoming a harmonic mean (reciprocal).

Both of these indices fail the time reversal, circularity and factor reversal tests discussed in the previous section.⁶ The Laspeyres index is the most frequently used primarily because of data considerations—previous period expenditures are usually more readily available than current period expenditures. Intuitively, this index is more easily understood by many users because the concept of measuring the prices of a fixed market basket and comparing them to a base period is straightforward. In the literature on the theory of the cost-of-living index, as mentioned previously, the Laspeyres price index approximates an upper bound to a CLI while the Paasche index approximates a lower bound.⁷ This bounding property provides some sense of security to practitioners in knowing that the true estimate lies somewhere in between.

The geometric mean formula (equation 7), used to compute basic level indices in several countries, passes four of the six tests previously discussed. The major exception is that it fails the factor reversal test.⁸ It also has the advantage over the Laspeyres index in that it assumes that product substitution, consistent with a unitary elasticity, takes place across products. This is more realistic than the Laspeyres index assumption that no substitution takes place. However, it is not clear that the geometric mean index possesses any approximation properties different than those for other fixed weight indices.⁹ In some instances the geometric mean price index will be greater than a CLI and in others it will be less than a CLI.

Superlative indices pass the time reversal test and have the property of being a closer approximation to the ideal consumer utility function than the Paasche, Laspeyres, or geometric mean. These indices use weight information from both the base and comparison periods. The simplest of these indices is the Fisher index (equation 8) which is the geometric mean (square root) of the product of Laspeyres and Paasche indices. The Fisher index passes five of the six index tests, including the factor reversal test. The exception is the circularity test.¹⁰

Another commonly used superlative index is the Tornqvist or translog index (equation 9). In this formula, the average expenditure share between the two periods ([w₀ + w_t]/2) is used as a weight to compute a geometric mean. The Tornqvist passes four of the index tests but fails the circularity and monotonicity tests.¹¹ Thus, the Fisher and Tornqvist indices have advantages over the Laspeyres and Paasche both from the axiomatic and theoretic points of view.

One major disadvantage of the Fisher and Tornqvist indices is that they are not additive in aggregation. The Fisher index is computed directly from the respective Laspeyres and Paasche indices. When an index is derived for the elementary aggregate (first) level in the Paasche and Laspeyres systems, it is aggregated with other elementary indices, using expenditure shares as weights, to arrive at higher level aggregates. The Fisher index is computed directly from indices at each level. Thus, the higher level Fisher index may not show the precise change implied by the lower level components. For example, an aggregate Fisher index with two subcomponents may show a price change of 0.4 percent while each of the subcomponents show a change of 0.3 percent. A similar problem occurs in the derivation of a higher level Tornqvist index since it is computed directly as a geometric mean of individual price observations of all lower level components.

Chain indices may also have some advantage. When prices are continuously rising without a great deal of oscillation and the weights do not shift very rapidly from period to period,¹² chain indices can reduce some of the extreme differences shown by fixed-base indices. A chain Laspeyres index has less upward substitution bias than the fixed-base version and the chain Paasche index will have less downward bias (Forsyth and Fowler, 1981; Aizcorbe and Jackman, 1993).

B. The price index formula used for the U.S. CPI

The formula used at the elementary aggregate level in the U.S. CPI is similar to equation 3 in Appendix I. This formula would appear to be the standard one used in many countries. There is, however, some ambiguity in the application of this Laspeyres formula. The ambiguity occurs with the price that is used. In most countries, the price represents an unweighted average price for a number of varieties selected to be representative of the item. For example, if the item in the CPI is whole milk and three varieties of whole milk have been selected for monthly pricing, the price used in the calculation of the index is the average price of the three varieties. The base period price represents the average price of these three varieties in the base period. Thus, the index is said to be computed as a ratio of average prices or what Diewert (1995) calls a Dutot index.

In the U.S. CPI, each price is the actual price of the variety that has been selected and is compared to the estimated base price of that variety in the base period. Each variety is assigned a sampling weight and the resulting index number is a weighted average of price relatives. Using the milk example, the U.S. CPI computes a price relative for each of the three varieties of milk and then calculates the index as a weighted average of these price relatives.¹³ An index which uses the average of price relatives is more volatile than an index that uses the ratio of average prices. Turvey, et. al. (1989) demonstrate this problem in the section on index computation and in their appendix 7 on computation of micro-level indices. Schultz (1995) and Moulton and Smedley (1995) provide simulations of these differences using data from the Canadian and U.S. CPIs, respectively. Diewert (1995) shows that the average of price relatives (Carli index) is upward biased in relation to the ratio of average prices (Dutot index) and the geometric mean (Jevons index).¹⁴ There is substantial evidence that the estimation formula used in the U.S. CPI is upward biased relative to other estimators. Due to this fact, Eurostat has notified member countries of the European Union (EU) to avoid using the ratio of price relatives technique in producing their CPIs.¹⁵ Eurostat recommends the use of the ratio of average prices or the geometric mean at the elementary aggregate level.

The choice between the ratio of averages or the geometric mean is not always clear cut. It can depend on the degree of homogeneity of the products or services that are included within the elementary aggregate index. If there are large price differences among the sampled products, higher price items implicitly have more weight than lower price items in a ratio of averages. This means that the price change of the higher price items has more influence on the average price change. The geometric mean of price relatives, on the other hand, treats each price observation equally. In addition, the geometric mean of the ratio of average prices is mathematically equivalent to the geometric mean of the price relatives. Diewert (1995) argues that very homogenous groupings can be obtained through the use of scanner data from many companies and that the unit value computed as the ratio of total sales of a specific product divided by the quantity sold over some specific time period, such as a week, is an appropriate average price to be used for micro-level price indices.

Practitioners in several countries have taken the other view point. Schultz (1995) presents the case that from a practical sampling point of view, most strata of items will contain a certain degree of heterogeneity and that the geometric mean is the more appropriate estimator at the micro-index level. This, and the fact that the geometric mean formula assumes a certain amount of substitution takes place, either across varieties or among outlets, are the two primary reasons Statistics Canada switched from using the ratio of average prices to the geometric mean for the calculation of micro-level indices in January 1995. Woolford (1994) provide similar reasoning for the shift to the use of a geometric mean in the Australian CPI. Viglino (1995) examines the case for using the geometric mean by INSEE in France. He suggests that the Divisia index may be the appropriate price index measure and the geometric mean is a better approximation of a Divisia index than is the Laspeyres index.

It is of interest to note that the ratio of average prices was used in the U.S. CPI prior to the 1978 revision. As part of that revision, the U.S. introduced full probability sampling throughout the entire statistical design—from the selection of geographic areas to the selection of outlets to the selection of representative items within outlets. There were several important factors that were considered in the decision to move to the average of price relatives formula. First, there was concern that probability sampling within outlet would result in a fairly heterogeneous sample within an item stratum. This would result in higher price items having greater influence. Second, the average of price relatives weighted by expenditures provided, for the first time, a modified Laspeyres index. The previous methodology used an expenditure weighted change in average prices which was not a true Laspeyres formulation, but rather, a weighted Dutot index. The new estimator would also allow for a consistent methodology for making quality adjustments. The previous method used the monthly price change in the average price from a matched sample of observations. When quality adjustments were made, the prices in each month reflected the quality difference, but there was no adjustment to the basic weights for this change which would take into account the new level of quality in the base period expenditures for the price observation.¹⁶ As Reinsdorf (1994) points out, the shift to the new Laspeyres estimator has led to an upward drift in the U.S. indices in relation to changes in average prices.¹⁷

It would appear from the preponderance of evidence that a new estimator is in order. This will be discussed further in the last section on potential future directions for the U.S. CPI.

A. The sample design of the U.S. CPI

The sample design for the U.S. CPI is documented in U.S. Department Labor (1992), Dippo and Jacobs (1983), and Leaver, Weber and Cohen (1987). The design involves a multistage probability selection process for sampling by geographic area, retail outlet, item category, and individual goods and services within an outlet and category. The CPI geographic sample has 88 primary sampling units (PSUs) which represent all urban areas. The New York metropolitan area consists of 3 PSUs, the Los Angeles metropolitan area consists of 2 PSUs and the remaining urban areas contain but one PSU each. The 32 largest PSUs are representing consumer purchases for their own index area. The other 56 PSUs were chosen, using probability proportionate to size (PPS) techniques, to represent twelve additional aggregated geographic areas. These twelve additional index areas measure changes in consumer spending in the medium, small an non-metropolitan urban areas in each of the four regions of the country. Thus, there are 44 index areas (32 self-representing areas and 12 city-size class by region areas) that represent changes in consumer purchases for the urban population. For a more complete and rigorous presentation of the geographic area design, the reader is referred to Dippo and Jacobs (1983).

In each index area samples of retail outlets and housing units are drawn using PPS procedures to represent each item stratum. The outlets and item categories are selected from information provided by households in the Continuous Point of Purchase Survey. In this survey consumers enumerate the types of expenditures they have made and the outlets where they purchased the items. For each item category within a retail outlet included in the sample, a unique representative item is selected using probability proportionate to annual sales volume. The prices for the items selected in the outlet are those used to compute the monthly price change in equation (3) of Appendix I. For more details on sampling of housing units see U.S. Department of Labor (1992).

B. Selection of geographic areas

Following the Decennial Census of Population a new sample of geographic areas is drawn and introduced into the CPI coincident with a weight revision. This is a very costly component of each CPI revision and it is not clear that it is worth the added expense given other less costly options. In general the largest cities remain in the sample as they are selected with certainty. The geographic boundaries of these cities will change as more of the surrounding areas are added to the definition of the Standard Metropolitan Area (SMA). The process of expanding the geographic definition is accomplished during the revision cycle through the sample rotation process which is a basic component of the CPI operations. Where the revision becomes somewhat costly is the drawing of new city samples for the representation of non-certainty areas. This process results in the dropping of existing cities and the addition of new cities. This adds additional expense due to the personnel costs in terms of the severance pay for dropping highly trained staff in the old cities, additional training costs of staff in the new cities, and the overlap period needed to bring the new staff up to competent levels of performance.

An alternative approach would be to keep the same sample of areas and add new areas as required by population growth within regions. In this manner, rather than keeping the same number of PSUs and forcing change because of relative shifts in population and income, the sample grows in relation to changes in economic factors. With each revision, new PSUs could be added due to population growth and the old PSUs would have their population weights updated to the latest Census totals.

C. Probability selection of outlets in geographic areas

The item structure for the CPI consists of 207 item strata for commodity groups such as white bread, carbonated drinks, boys’ apparel, etc. The cross of the 44 geographic areas with the 207 item strata creates the 9108 basic strata, which is the level of index calculation at which the weights are fixed. Within each item stratum there are one or more substrata, called entry level items (ELIs). There are 364 ELIs which are the first stage of item selection. ELIs are also the level of item definition at which BLS data collectors begin item sampling within each outlet.

BLS uses three surveys to conduct item/outlet sampling for the CPI. The first is the Consumer Expenditure Survey (CE), which provides the weights for the basic strata and sample weights for ELI selection within the strata. The CE consists of an interview survey and a diary survey. The interview survey collects inventories of items held by the respondent and expenditures for a full year on major consumer purchases (vehicles, durable goods, insurance policies). The diary survey records every purchase made during a two week period by any member of the family. Each expenditure recorded is mapped to one of the 364 ELIs. Estimates of annual expenditures for each ELI and item stratum by area are then produced. The average of these estimated annual expenditures for the 1982-84 period comprise the expenditure weights used in the U.S. CPI

To enable the CPI to reflect changes in the market place, item and outlet samples are re-selected on a rotating basis. Each year, a new sample of ELIs and outlets for 20 percent of the areas (about 17 cities) are re-selected. In this annual process, four regional universes are tabulated at the ELI level²⁰ from the two most recent years of CE data. An independent sample of ELIs is selected within each of the 207 item stratum in each region. The probability of selection for each ELI is determined by the relative importance of the ELI within the regional item stratum. Thus, the ELIs selected for area samples will change based on their importance in the ongoing Consumer Expenditure Surveys.

The second survey used for the U.S. CPI is the Continuous Point of Purchase Survey (POPS), which determines the retail outlets from which consumers’ purchased goods and services. In the areas scheduled for sample rotation in the following year, the Census Bureau, under contract with BLS, conducts a POPS. The POPS is a household survey conducted over a four to six week period, usually beginning in April. The Census Bureau enumerator asks respondents whether or not certain categories of items were purchased within a specified recall period. The recall period varies depending on the type of items purchased. ELIs are grouped into sampling categories (called POPS categories). Some POPS categories consist of only one ELI; other POPS categories contain several ELIs when certain types of commodities or services are generally sold in the same retail outlets. For example, the Meat and Poultry POPS category consists of eight beef ELIs, six pork ELIs, four ELIs for other meats, and three poultry ELIs. These are combined because an outlet that sells beef also tends to sell other meats. For each category the respondent in the household is asked about purchases made within the stated recall period, the names and locations of places of purchase, and the expenditure amounts. After this information is tabulated for the city, a sample of outlets is drawn for each selected POPS category.

Since the ELI and outlet samples are selected in separate processes for each geographic area, they must be merged before data collection. A concordance maps each ELI to a POPS category. Each sample ELI is assigned for price collection to the outlet selected for the corresponding POPS category. The number of price quotes collected for an ELI in each outlet is equivalent to the number of times the ELI was selected for the area in the item sampling process.

The number of price quotes assigned for collection in a sample outlet is determined through the item/outlet sample merge. In the outlet sample process, an outlet may be selected more than once for a given POPS category, provided the expenditures reported for the outlet are large. The outlet may also be selected for more than one POPS category. If an outlet is selected multiple times for a given POPS category, the same multiple of price quotes will be assigned for collection for each sample ELI matching the category. If an outlet is selected for more than one POPS category, price quotes will be assigned for collection for all sample ELIs matching the categories.

An alternative method for selecting outlets would be to use a process similar to the United Kingdom’s approach and select outlets from registers of businesses within the geographic areas selected. The U.K. refines the geographic areas to identify clusters of retail businesses and then selects outlets based on sales by type of retail outlets. In most other countries the outlet identification is purposive, whereby retail centers are identified and then shops within those centers are chosen based on where most people shop. However, the use of business registers²¹ or, even better, information from the economic censuses (conducted every five years in the U.S.) could be an alternative source that potentially provides more reliable sales or revenue information for sampling purposes.²² The use of information from the census of business has been prohibited by confidentiality restrictions. The removal of such restrictions to allow for the exchange of information among statistical agencies would be necessary. This has been one proposal that has been in review for a number of years and has most recently been proposed by former Commissioner Janet Norwood (1995). The use of the business census information should allow for highly accurate PPS selection of outlets within areas.

D. Probability selection of item varieties within outlets

The selection of samples of products and services within outlets involves another process of multistage probability sampling referred to by BLS as disaggregation. The process involves working with a knowledgeable respondent in the outlet to gain information on annual sales for items included in the CPI category that has been chosen during the outlet/category selection process previously described. This process is explained in the CPI technical note (Chapter 19 of U.S. Department of Labor, 1992) and in more detail in Armknecht and Ginsburg (1992). The essential feature is to start at some high level category (the ELI definition) and successively proceed to more narrow specifications until a unique product or service is identified. For example, if men’s shirts are the category, then the first stage may involve choosing among dress shirts, casual shirts, sports shirts, etc. The next stages may involve choosing among brands, then selecting between short and long sleeve, then perhaps collar type, etc. The process continues until a uniquely described shirt has been selected.

This whole process is not always based on highly accurate sales information. While data collectors are taught to elicit information on sales over the previous year, few respondents within stores have the actual records readily available or are willing to use them for this time consuming process. The respondent most often is providing broad ranged estimates from memory or judgment, and all too frequently data collection agents must resort to equal probability selection. The CPI Quality Assurance staff conducted a study on this process using a sample of disaggregation worksheets returned from the BLS regional offices. The study found that the most frequently used disaggregation technique was ranking. In this type of disaggregation the respondent merely ranks the order of the specific groupings and interval limits can be determined from a ranking table based on the number of groups ranked. In the shirt example, if the stage of disaggregation was choosing between long and short sleeve varieties and the respondent could not provide estimates of the proportions, s/he would be asked to rank them. If the respondent said long sleeve shirts were first, then long-sleeves would be given a 67 percent chance of selection and short sleeves a 33 percent chance. Otherwise, with no information at all, equal probabilities would be assigned.

An alternative to this process would be to use information at the national, regional, and city level for the major retail chains to select the specific items for which prices would be collected. If the outlets can be selected in the second stage of selection, then products, potentially, could be selected within those outlets from information available in company data bases. For some large segment of products such as packaged goods, private companies that use scanner technology such as Nielsen or Information Resources, Inc. could provide information from which samples can be drawn. BLS is currently testing such information sources to assess their applicability. The scanner data would only cover about 20-25 percent of the CPI universe. This means that substantial efforts to work with many retail chains would be necessary. This is one area in which BLS regional offices could play a role. The Chicago office has suggested this as a possibility on more than one occasion. The magnitude of the initial effort to undertake such a program has been thought to be immense. Resources used for sample research and sampling operations would have to be concentrated on such a project for a substantial period of time.

A potential benefit of drawing samples at the national, regional and city level is that once a specification has been determined, the product could be priced in all outlets. Prices for a large number of products priced in most locations would facilitate spatial comparisons across CPI areas. This also would enable the U.S. to participate more readily in the International Comparisons Project for purposes of developing annual price parity comparisons with trading partners.

Another area of potential improvement is that of the optimal sample sizes for strata. In the U.S. the experience has been that the contribution to variance across outlets has been greater than the contribution across items. The research on this subject in Europe has shown the opposite to be the case—the variance across items has had greater contribution to the total than variance across outlets.²³ The result of this has been that in the U.S. there is more emphasis on getting more outlets in the sample at the expense of fewer items. The marginal cost of getting more outlets is considerably more expensive than securing more quotes because the time and expense of going to another outlet is significantly greater than getting another item within existing outlets. It may be beneficial to examine the reasons for this interesting discrepancy.

E. Probability selection of housing units with in area

The current method of selecting housing units for the sample of renters is a timing-consuming and tedious task. BLS selects segments and census blocks from the latest public use files of the most recent census of population and housing. They do not have access to micro level information on address, income and ethnicity. A BLS field agent is given a map for the area with the selected blocks identified. The agent then must list all addresses of housing units within the block. From this list the sample of units is selected using equal probability. It is obvious that this process could be simplified if BLS had access to the micro-level census data to select the renter sample directly. It could also result in more efficient samples by use of all the important stratification variables rather than using simple random samples within blocks.

A. Methods used for making quality adjustments in the U.S. CPI

The different methods used have received much attention in the literature in recent years. Armknecht and Weyback (1989) discuss a variety of methods that are used in the U.S. CPI and estimate the effect of quality adjustment in reducing the CPI change in 1984.²⁴ Kokoski (1993) presents the results of hedonic regression techniques for measuring the value of quality characteristics across a broad range of items. Liegey (1993 and 1994) presents the application of hedonic regression techniques to the quality adjustment of apparel commodities in the CPI. Reinsdorf, Liegey and Stewart (1995) present examples of indirect, hedonic and new imputation methods for making quality adjustments in the CPI. Armknecht and Moulton (1995) examine standard approaches for making imputations for quality change on a routine basis with information already available within the sample of priced items. Feenstra (1995) provides a theoretical justification for the use of hedonic methods in the case of competitive markets. He points out that in other market situations the results need to be interpreted correctly and that the regression parameters are may require further adjustment to measure the precise effect of quality factors. The methods predominantly used in the U.S. CPI are direct adjustments or imputations.

B. Frequency of monthly quality adjustments in the U.S. CPI

The information appearing in Table 2 represents the number of price observations collected annually in the U.S. CPI between 1993 and 1995 along with the number of substitute products introduced during monthly price collection. The bottom rows show the average over the period. The rate of product substitution was 3.6 percent. This compares to about 3.9 percent in 1983-1984 as reported in Armknecht and Weyback (1989) and Armknecht (1984). Of the replacement products, 56 percent were used in the CPI as a direct comparison with the old variety they replaced. In 1983-84 only about 40 percent of substitute varieties were used in direct comparisons. This is a significant shift and is due to improvements in defining comparability criteria.²⁷

Table 2.

Product Substitution in the U.S. CPI and Frequency of Quality Adjustment, 1993-95

Note: This table excludes residential rent, homeowner’s equivalent rent and health insurance.

^1.This includes counts of some versions for an imputation mehtodology that was being phased out.

^2.No data for used cars were available to include for the 1993 comparisons.

SOURCE: US Bureau of Labor Statistics. The author thanks Paul Liegey for providing this information.

Table 2.

Product Substitution in the U.S. CPI and Frequency of Quality Adjustment, 1993-95

1993			Number of Substitute Products by Type of Action
CPI Major Groups		Annual Observations	Total	Direct Comparison	Direct Qual Adj	Imputed Quality Adj.1
						Overall Mean	Class Mean
Food and Beverages		467,396	7,854	3,200	41	4,299	314
Housing (exc. Shelter)		123,876	4,164	2,479	149	1,008	528
Apparel and Upkeep		58,739	8,558	6,307	912	775	564
	Women’s Apparel	18,068	4,328	3,117	718	291	202
Transportation 2		67,635	4,204	2,216	1,036	431	521
	New Vehicles	12,738	1,972	402	1,013	116	441
Medical Care		50,490	876	254	317	281	24
Entertainment		27,403	1,196	673	68	356	99
Other Goods & Service		17,535	452	216	55	150	31
Total		813,074	27,304	15,345	2,578	7,300	2,081

1994			Number of Substitute Products by Type of Action
CPI Major Groups		Annual Observations	Total	Direct Comparison	Direct Qual Adj	Imputed Quality Adj.
						Overall Mean	Class Mean
Food and Beverages		464,618	8,233	3,601	37	4,409	186
Housing (exc. Shelter)		129,542	4,539	2,725	211	733	870
Apparel and Upkeep		63,255	8,866	6,475	876	535	980
	Women’s Apparel	19,390	4,408	3,127	690	223	368
Transportation		79,287	6,131	2,881	2,111	495	644
	New Vehicles	13,540	2,307	733	880	180	514
Medical Care		50,827	1,077	312	337	405	23
Entertainment		29,442	1,335	822	98	205	210
Other Goods & Service		19,353	429	231	39	132	27
Total		836,324	30,610	17,047	3,709	6,914	2,940

1995			Number of Substitute Products by Type of Action
CPI Major Groups		Annual Observations	Total	Direct Comparison	Direct Qual Adj	Imputed Quality Adj.
						Overall Mean	Class Mean
Food and Beverages		459,759	7,552	3,626	168	3,642	116
Housing (exc. Shelter)		135,660	4,629	2,734	168	730	997
Apparel and Upkeep		68,839	9,172	6,215	1,189	373	1,395
	Women’s Apparel	21,547	4,673	3,019	859	169	626
Transportation		90,365	6,807	3,290	2,483	373	661
	New Vehicles	14,372	2,182	414	1,129	132	507
Medical Care		50,316	1,153	314	365	430	44
Entertainment		31,029	1,352	835	113	179	225
Other Goods & Service		20,964	499	262	57	136	44
Total		856,932	31,164	17,276	4,543	5,863	3,482
Average over period		835,443	29,693	16,556	3,610	6,692	2,834
Frequency			3.55%	55.76%	12.16%	22.54%	9.55%

Note: This table excludes residential rent, homeowner’s equivalent rent and health insurance.

^1.This includes counts of some versions for an imputation mehtodology that was being phased out.

^2.No data for used cars were available to include for the 1993 comparisons.

SOURCE: US Bureau of Labor Statistics. The author thanks Paul Liegey for providing this information.

View Table

Table 2.

Product Substitution in the U.S. CPI and Frequency of Quality Adjustment, 1993-95

1993			Number of Substitute Products by Type of Action
CPI Major Groups		Annual Observations	Total	Direct Comparison	Direct Qual Adj	Imputed Quality Adj.1
						Overall Mean	Class Mean
Food and Beverages		467,396	7,854	3,200	41	4,299	314
Housing (exc. Shelter)		123,876	4,164	2,479	149	1,008	528
Apparel and Upkeep		58,739	8,558	6,307	912	775	564
	Women’s Apparel	18,068	4,328	3,117	718	291	202
Transportation 2		67,635	4,204	2,216	1,036	431	521
	New Vehicles	12,738	1,972	402	1,013	116	441
Medical Care		50,490	876	254	317	281	24
Entertainment		27,403	1,196	673	68	356	99
Other Goods & Service		17,535	452	216	55	150	31
Total		813,074	27,304	15,345	2,578	7,300	2,081

1994			Number of Substitute Products by Type of Action
CPI Major Groups		Annual Observations	Total	Direct Comparison	Direct Qual Adj	Imputed Quality Adj.
						Overall Mean	Class Mean
Food and Beverages		464,618	8,233	3,601	37	4,409	186
Housing (exc. Shelter)		129,542	4,539	2,725	211	733	870
Apparel and Upkeep		63,255	8,866	6,475	876	535	980
	Women’s Apparel	19,390	4,408	3,127	690	223	368
Transportation		79,287	6,131	2,881	2,111	495	644
	New Vehicles	13,540	2,307	733	880	180	514
Medical Care		50,827	1,077	312	337	405	23
Entertainment		29,442	1,335	822	98	205	210
Other Goods & Service		19,353	429	231	39	132	27
Total		836,324	30,610	17,047	3,709	6,914	2,940

1995			Number of Substitute Products by Type of Action
CPI Major Groups		Annual Observations	Total	Direct Comparison	Direct Qual Adj	Imputed Quality Adj.
						Overall Mean	Class Mean
Food and Beverages		459,759	7,552	3,626	168	3,642	116
Housing (exc. Shelter)		135,660	4,629	2,734	168	730	997
Apparel and Upkeep		68,839	9,172	6,215	1,189	373	1,395
	Women’s Apparel	21,547	4,673	3,019	859	169	626
Transportation		90,365	6,807	3,290	2,483	373	661
	New Vehicles	14,372	2,182	414	1,129	132	507
Medical Care		50,316	1,153	314	365	430	44
Entertainment		31,029	1,352	835	113	179	225
Other Goods & Service		20,964	499	262	57	136	44
Total		856,932	31,164	17,276	4,543	5,863	3,482
Average over period		835,443	29,693	16,556	3,610	6,692	2,834
Frequency			3.55%	55.76%	12.16%	22.54%	9.55%

Note: This table excludes residential rent, homeowner’s equivalent rent and health insurance.

^1.This includes counts of some versions for an imputation mehtodology that was being phased out.

^2.No data for used cars were available to include for the 1993 comparisons.

SOURCE: US Bureau of Labor Statistics. The author thanks Paul Liegey for providing this information.

In recent years about 12 percent of substitutes were quality adjusted using direct quality adjustment methods while in the earlier period only 0.3 percent were directly quality adjusted. The large difference is due to a higher incidence of direct quality adjustment in three CPI categories. First, used car prices are now quality adjusted as newer model cars enter the sample each year. The quality factors (ratios) estimated for the model year when the cars were new is now used for estimating the quality differences as these models enter the used car sample. Second, hedonic regression techniques are now used for estimating quality adjustment values in women’s and girls’ apparel. Third, gasoline prices have been quality adjusted as producers implement improvements to meet new federally mandated clean air standards.

The frequency of price linking as a quality adjustment method has declined relatively. In recent years about 32 percent of substitutes (22.5 percent with overall mean imputation and 9.6 percent with class mean imputation) were quality adjusted via the linking technique. This represents a decline from the 1983-84 period in which quality adjustments by linking accounted for about 45 percent of substitutions. The new class mean imputation was first introduced in October, 1989 as part of the new vehicle price imputation during the model change-over period. It was extended to other categories of commodities and services in February, 1992.²⁸

C. Quality adjustments coincident with sample rotations

In addition to the monthly quality adjustments just presented, another form of quality adjustment is implicit in the sample rotation process. Each year the outlet and item samples are reselected in about 17 of 88 PSUs that represent urban areas of the U.S. During the course of the year each PSU is brought into the CPI through a process of overlap pricing. Prices in the old sample and new sample are collected in the same month. The old sample is used in the index during this double collection month. In the following month (referred to as the link month), the new sample is used to represent price change. All price level differences between the old and new samples are treated as quality differences. This approach applies the overlap method of quality adjustment discussed above. There is no direct comparison between the prices in the new sample with those in the old sample.²⁹

Whether or not this overlap linking technique for introducing new samples is appropriate has been of considerable debate. Anecdotal evidence can be found to support both contentions. Many people believe that there is no difference in quality between identical products sold in full service department stores versus discount or mass merchandise outlets. The 1993 System of National Accounts manual (1993, p. 398) published by the United Nations advises that the same product sold in different types of outlets should be considered of different qualities due the different levels of services provided as part of the sales transaction by the outlets. Reinsdorf (1993) found small differences, lower on average, between the average price of grocery store food prices and gasoline prices after sample rotations.³⁰ Fixler (1993, pp. 8-9) notes that the overlap linking approach has theoretical underpinnings in a cost-of-living framework, but “…can cause the difference between the CPI and an ideal CLI to be larger than it would be if value of retail services were properly accounted for.” This approach may or may not be warranted and further empirical testing on the value of retail services is needed. Hedonic models similar to those found in Kokoski (1993) with additional price collection on outlet characteristics is being pursued to assist in providing the additional information.

Sample rotation is the primary source by which totally new goods enter the U.S. CPI. When a specific variety is no longer available during monthly pricing, the variety that most closely fits the specifications of the old variety is selected to replace it. If there are varieties that are a close match, then one of them will be selected even though the type of product is becoming obsolete (e.g., records and compact discs). This is not too say that new goods will not enter the sample in this way. They can, but the outlet has to discontinue sales of all close matches to the old variety. While product substitutions result in mostly replacement varieties for existing products, it is during sample rotation that the whole category of items (ELI) is open for potential selection. When the new good enters the index through sample rotation, it is because the product had sufficient market share to be selected within the ELI. The new good has no immediate effect on the index because of the overlap linking procedure. So if, for example, compact discs appear in the new sample and records are dropped in the old sample, no price change is recorded between the two items and the difference in price level is assumed to be the amount of quality difference.

D. Quality adjustments in medical care

In the U.S. CPI the price of medical services usually relates to a single service such as an office visit to a physician or a day’s charge for a hospital room. The pricing of medical services is discussed in more detail by Armknecht and Ginsburg (1992). This specific service approach has come under criticism because it does not take into account enough of the factors surrounding medical services to allow for adequate changes in quality. When the fee for a doctor’s office visit changes because of the addition of a test or an increase or decrease in the time with the patient, the quality adjustment is straight forward. But when pricing separate services such as that for a surgeon, hospital room, operating room, etc., there is no opportunity to include improvements in efficiency that patients might experience such as shorter stays in the hospital, less intensive use of the operating room because of improved procedures, or fewer visits with the surgeon. For this reason, the CPI medical services analysts have been evaluating and, hopefully, planning to switch to a treatment path approach, similar to that used in the PPI for hospitals. This involves selecting a specific treatment that a patient would receive and collecting the price for that treatment. So when treatment technology improves and the price of the treatment path is lowered, this will be reflected in the index. This will help resolve some major issues such as shortening of hospital stays following surgery or heart attack.

Such an approach still will not solve the problem of major medical breakthroughs that entirely change treatment technology nor adjust for the fact that some procedures become less risky/more successful. If, for example, treatment in the emergency room prevents major surgery, reduces the stay of the patient, and improves that survival rate, this should be reflected as a quality improvement in medical services. The current and the treatment path approaches still cannot adequately handle such quality improvements. A new dimension needs to be included in the pricing of medical services that includes outcomes, so that if cancer treatment results in improved survival rates, this is reflected in the index. This area appears to have promise and is one that also needs to be pursued as part of the research agenda in quality adjustment.

A. Measurement objective and estimation formula

The preponderance of evidence that there is a formula bias is overwhelming and a change is needed to move to a more accurate estimation system. The question then becomes “what is the measurement objective?” Should there be an attempt to provide the best measure of a Laspeyres index or should the attempt be to measure as close as possible a cost-of-living index? Currently BLS technical publications (U.S. Department of Labor, 1992, p. 177) say:

“The CPI uses a fixed market basket to hold the base-period living standard constant. The CPI equals the ratio of the cost of the base-period basket at this month’s prices to the actual cost of the base-period basket in the base period. The formula used for calculating the CPI is the one known in price index literature as the Laspeyres index.”

At the same time BLS notes that “A unifying framework for dealing with practical questions that arise in construction of the CPI is provided by the concept of the cost-of-living (COL) index.” With all the recent discussion that has taken place about the U.S. CPI, it would appear that users expect the CPI to measure as close as possible a cost-of-living index.

Given this user objective, it becomes clear that the appropriate index formula for use in the CPI at the second stage of aggregation should be one of the superlative indices—either the Fisher or the Tornqvist index. The practical issue is that these indices cannot be calculated for current periods due to the lack of current period weights. Weights for current periods are at least a year behind due to processing constraints related to the CE survey and, because of small sample sizes, three years worth of information must be used to obtain the area/item weights for the 9,108 strata. It would be possible, however, to compute a Fisher or Tornqvist index with a two year delay.³¹ Such a scenario would mean that the CPI is subject to revision for a period of up to three years. This is not unprecedented since other major series such as payroll employment, hours and earnings, productivity, and GDP estimates are all subject to revisions at later dates. It would be a significant change in policy, however, to move to annual or bi-annual weight updates.

In addition, the current period estimates of CPI and the all the micro-level indices (first stage of aggregation) should use a formula that gives estimates closer to a true cost-of-living index than the current Laspeyres estimator The geometric mean estimator, which is closely associated to the Tornqvist index if the expenditure shares are relatively stable, would be the most likely candidate. In this scenario the CPI would be revised every year with new weights that are two years old. The historical CPI series would be a chained superlative index while the most recent two years of information would be a geometric mean index. Using such an approach is similar to the one adopted by BEA in the measurement of real GDP.³² Given the fact that the weights do not change drastically from year to year and that there has historically been steady growth in inflation over the shorter periods, the magnitude of revisions should be minimal, say within 0.1 to 0.2 percent on the annual change. This certainly is a scenario that can be simulated with the information BLS has at its disposal.³³

A slightly different alternative would be to project the weights for the current year based on historical patterns. If there were some standard, accepted model or practice for doing so, this could be viable. However, there is no such forecasting model that is standardized and widely accepted. Most such models would be sensitive to the underlying assumptions. So while this alternative is practical, this would put BLS in a very sensitive position of forecasting the weighting pattern that it will use for current estimation. Such a process could open Pandora’s box should any politicization whatsoever of the forecasts be intimated. For this reason I would not recommend such a solution.

B. New approaches to probability sampling

Probability sampling is an extremely costly endeavor compared with judgmental sampling. This is the primary reason that most countries do not have full probability sampling in their CPI. In the U.S. CPI the number of geographic areas required is driven by the population size and income of urban areas within the U.S. as well as the precision of the final estimate of price change. This would argue for maintaining the current area sample with some potential for growth as population and income changes dictate. However, it is not clear that it is cost effective to re-draw the area sample in entirety with each population census. Adding additional areas when required and updating population weights would make more sense.

Where more savings might be achieved is drawing samples of outlets and items from data sources that already exist and can be tapped to provide the type of detailed information needed for probability sampling. Developing redundant surveys and maintaining redundant universe files forces gross inefficiencies on the statistical system. Such information is available in the records produced by the population and business censuses and in company data bases. What is needed is the ability for federal statistical agencies to have access to such universe files for sampling purposes. There is also a need to develop a partnership with the private sector to obtain cooperation and access to company sales records for sampling purposes.

Census records can be used to draw item and outlet samples similar to those currently available (perhaps even better) for the nation, regions and cities. Specific items and varieties could be selected for a national component (items so common they are priced everywhere) and regional and local components (supplemental items where there are regional and local variations). At the local level, where specific local varieties are important, selection from company records or other secondary sources (Nielsen and other private marketing firms), practically all available in electronic format, could be tapped. As part of this process, an emphasis on standardizing national and regional specifications for items should be instituted to enable spatial comparisons across cities for domestic analysis and across the country for international comparisons. This is an important aspect of the CPI where there is an increasing need on the part of stakeholders.³⁴

C. Quality adjustment and introduction of new goods

Significantly more quality adjustment takes place in the CPI than most users are aware. The problem is that the quality adjustment is associated with the replacement of one very similar variety with another. As long as close substitutes are available, even if they have significant declines in their market share, aging products will not be replaced with newer technologies. Historically in the U.S. CPI a product continued to be priced as long as it was available. When a product was no longer available and there was not a close substitute within the ELI category definition, no product was priced. This pattern has changed gradually over time. In the 1987 revision, the item categories were more broadly defined so that when products disappeared, a replacement product with similar functionality could be introduced in its place. This doesn’t solve the problem of declining market share of certain products and increasing market share for new products. What is needed is a systematic pattern of sample re-initiation in those categories where new products are emerging.³⁵ In this way, these categories could be re-selected every year or two to enable new market entries into the CPI sample more quickly. One possibility is to monitor more closely the new products that are being introduced through the diary and interview surveys in the CE. This would have to be done systematically. In addition, a formal procedure should be instituted for commodity analysts to identify newly emerging products that are identified from trade sources and commercial vendors. BLS field staff can be requested to identify, on an ongoing basis, those products that are gaining popularity. Such sources can be used as indicators of categories that potentially are eligible for sample re-initiation. Also, if an annual weight update process were adopted, these new products could more readily be introduced within categories as part of that process.

Even with such a system in place, an issue arises concerning the price trend from the time the item was first introduced in the market until the time it appears in the CPI sample. With current procedures all new samples would be introduced using the overlap link procedure. Any influence the new product would have on the index begins after its introduction. Any difference in price level between old and new products would be the amount of the implicit quality adjustment. Diewert (1987) suggests estimating the shadow (market reservation) price of the new product just before introduction. If this could be done, an adjustment might be possible to the price trend of the series during the annual updating process by replacing the old product’s price with the estimates of the new product’s. This, however, would be a data management nightmare and would raise questions about the integrity of the index with the use of estimated prices. It is also not clear how accurate any estimated price would be. It is possible that information from computer records could be used for transaction prices, but the initial shadow price may have to be estimated using hedonic techniques as suggested by Griliches (1990). This approach was worrisome to Feenstra because of potential bias in the regression parameters (1995, pp. 34-5). The best solution for new products in the CPI may be to include them as soon as possible and not worry about the price trend from the introduction point. The weight of such items in the index would be very small at the time of introduction, and it does not seem feasible to assume that they would have a substantially large impact on the index.

One final note relates to two areas where additional, promising research needs to be continued at a heightened level. The first issue is that of quality adjustment of items across sample rotations. Given some of the suggestions discussed previously, where samples would include a core of standard specifications, it would be possible to observe outlet effects for specific products. Modeling techniques could be possible to develop adjustment factors by type of outlet and other characteristics. Further research is warranted to determine the magnitude of such effects and the realism of the assumptions underlying the overlap link technique. The second issue is quality adjustment of medical services and using an outcomes approach to pricing these services. Some promising work is being accomplished in this area using outside researchers. Given the limited staffing patterns that may be on the horizon for the CPI within BLS, some savings from other efficiencies should be set aside for additional contracts on conceptual and empirical research projects.

D. Final considerations

As a final point of emphasis (even if it is overly redundant), two major institutional/statistical policy changes are required to achieve many of the efficiencies and improvements discussed in this paper. The first is that the CPI staff need access to the detail files from the censuses of population and business for statistical purposes. Legislation has frequently been proposed to enable sharing of such data but has always run into obstacles from one of the agencies involved. The simple solution would be to pass enabling legislation for BLS and Census to share data. The more complex solutions involve national statistical legislation covering all agencies and/or merging of the Federal statistical agencies into a single central statistical office. These are policy issues that require administration and congressional action.

The second change relates to the policy of revising the CPI data historically. Most other major economic surveys allow for periodic revisions of historical data, recognizing that all the required information for accurate estimation is not available at the time of the first estimates. BLS has a correction policy for the CPI that enables revision of the CPI when errors in data collection, processing, or procedures are uncovered that affect the all items CPI by more than 0.1 %. However, this has little effect in practice because there are few errors large enough to impact the national index by that large an amount. It would appear reasonable, however, in light of the evidence of the substitution and other potential biases to re-examine the policy of no historical changes and allow for revisions (annually or bi-annually) to data over a specific time period (2-3 years). The CPI is not the only data series that is used for escalation purposes (although it is the most important one). Users can be provided with sufficient notification of such a change and advice provided on appropriate wording to add to new contracts. Also, a mechanism can be developed to provide beneficiaries of Federal programs with adjustments based on the cumulative effects of any revisions.

Related to this policy change is the reconstruction of series back in time that will be comparable to the new series when it is introduced. When BLS introduced the rental equivalency measure for homeownership in the CPI-U in 1983, the official series was not revised for historical comparability.³⁶ This has caused a great deal of consternation in the academic community through the years because the CPI has a serious conceptual break that most users are not aware of. While BLS did develop the CPI-U-X1 series going back to 1967 as a comparable measure, it is not widely known that this series exists and only the more sophisticated users, who begin questioning the different movements in the series historically, find out about its existence. If and when a new series using a superlative index methodology is introduced, BLS should make an effort to reconstruct the historical series backwards as far as possible. At a minimum, the series should begin in 1992 since this is the time period that BLS researchers have been using to test the effects of different estimators (Moulton and Smedley, 1995; Armknecht, Moulton and Stewart, 1994). The new series also should have an impact on measures of real GDP and productivity since the annual change in the CPI will be lower than originally published based on the new methodology. The CPI is a major series used in deflation of nominal components of personal consumption expenditures, the largest component of GDP by expenditures. Revisions to real GDP will, in turn, result in changes to productivity measures in the same direction.

This is an exciting time for the U.S. CPI because it has had a great deal of attention focused on many of its weak points. With this focus many users are waiting to see what improvements can be introduced to resolve these shortcomings. BLS has a new opportunity to examine the underlying measurement objective of the program and change it to meet stakeholders needs and expectations. It also is the time to highlight the necessary practical conditions that would be involved for any such change in focus to be implemented.