Abstract

8.1 When routinely compiling an export or import price index (XMPI), specific varieties of commodities in the index regularly appear and disappear. New goods and services can appear because technical progress makes production of new varieties possible. Even without technical progress in the supplying activity, however, commodities previously feasible, but not produced, may emerge because the technology of the using activity or the tastes of the final consumer have shifted. Existing varieties often decrease in importance or disappear from the market altogether as new varieties appear. For price collection using establishment surveys, the priced set of commodities often is a small sample of the full range of commodities that exist at any given time. Commodities in the sample may appear and disappear not because they are truly new to, or no longer produced or used by, all establishments, but because they may be only new to, or no longer produced by, the establishments in the sample. New goods and services may be new only in the sense that they were not traded in the previous or reference month but now reappear, while old goods and services may be old only in the sense that they are not traded in the current month, but once were.

A. Introduction

A.1 Why quality change is an issue

8.1 When routinely compiling an export or import price index (XMPI), specific varieties of commodities in the index regularly appear and disappear. New goods and services can appear because technical progress makes production of new varieties possible. Even without technical progress in the supplying activity, however, commodities previously feasible, but not produced, may emerge because the technology of the using activity or the tastes of the final consumer have shifted. Existing varieties often decrease in importance or disappear from the market altogether as new varieties appear. For price collection using establishment surveys, the priced set of commodities often is a small sample of the full range of commodities that exist at any given time. Commodities in the sample may appear and disappear not because they are truly new to, or no longer produced or used by, all establishments, but because they may be only new to, or no longer produced by, the establishments in the sample. New goods and services may be new only in the sense that they were not traded in the previous or reference month but now reappear, while old goods and services may be old only in the sense that they are not traded in the current month, but once were.

8.2 This chapter covers how to deal with the problem of continuous change in the assortment of transactions whose prices make up XMPIs. The overarching principle for designing methods to deal with variety turnover is that, at the most detailed level, the prices of items between any two periods may be directly compared only if the items are essentially the same. Violating this principle would mean that a given monthly price ratio measures not only the change in price, but also the value of the qualitative difference between two items. This contaminates the estimate of relative price change with an element, quality, that measures relative volume rather than price. It degrades the accuracy of the price index formed with the price ratios or relatives for the measured transactions.

8.3 What does “essentially the same” mean in practical terms? For measurement purposes, a commodity equates to a complete description of price-determining characteristics. It may, for example, be the case that an imported good can be purchased with a warranty or an extra option for the same price. But then the product would not be the same good, and the price of the option may decrease over time while that of the warranty increases. All items have to have the same commodity description. The form of this description often is simply text. It also can be highly structured. In structured commodity descriptions, the commodity’s characteristics are specific levels of indicators for several dimensions that are known to affect the average transaction price.1 Each set of these indicators’ levels frames a specific commodity. Examples of these dimensions are the horsepower of an automobile, the speed of a computer, or the species of a piece of fruit. Examples of commodity-determining levels or specific settings of these respective dimensions are 325 horsepower, 2 gigahertz, or flame red grape. The comparison of prices of varieties with like characteristics over time requires that detailed information is available to the price statistician each month on price and quality specifications, something that can be collected from establishment surveys but not from customs documentation.

8.4 For price measurement purposes, the comparative quality of a commodity comprises its description and price. Distinct descriptions represent different qualities of commodities, to the extent that they contain different levels of characteristics that affect the average price of transactions of things with that description in a given month. If commodities with two distinct descriptions are transacted at the same time, the description with the higher price must be the higher quality. This corresponds to what is called a revealed preference or value in use of the commodity (demand side), as well as higher content in the input needed to make the commodity (supply side). For index compilers, then, quality is an ordinal concept, comprising the set of complete commodity descriptions ordered by price for a given month.

8.5 When an existing variety of a commodity disappears and a new one appears, a new description manifests itself as well. The new description is different from the descriptions of existing commodities because the level of at least one characteristic in the description has changed. The difference in the characteristic explains the difference in price compared with varieties already available. For example, a new variety of computer emerges with a processor speed of 3 gigahertz instead of 2, and it has, say, a $325 premium over 2 gigahertz computers already available. Thus, the value of the additional gigahertz of speed is $325 and the new computer is, by implication, of higher quality than the old one. Some commodities may be quite distinct in their newness in a manner that cannot be described by the existing characteristic set, and the treatment of such completely new commodities is the subject of Chapter 9.

8.6 When a variety of a commodity is no longer sold, and price collection is based on establishment surveys, there are a number of options open to the price statistician. As will be outlined below, these include the use of a comparable replacement; a noncomparable replacement, with an explicit quality adjustment to one of the prices to make the items comparable, or an adjustment based on their relative prices in some overlap period; or more simply, the dropping of the variety from the sample until a new set of commodities is selected on rebasing or sample rotation. In each case, a quality adjustment is being undertaken, be it implicit or explicit. It is sometimes said by statistical offices that they do not undertake quality adjustments in their XMPI compilation. Some adjustments are always made; they are implicit in the practice of treating missing varieties. A number of empirical studies for consumer and producer price indices (CPIs and PPIs) have found that the choice of method for dealing with such missing values can matter substantially. This chapter outlines alternative methods and is also a guide to selecting methods based on the measurement circumstances so that the methods chosen can be the appropriate ones.

8.7 For price collection based on unit values from customs returns, there is no careful matching of the prices of like items with like. The implicit method of dealing with new and old varieties is to bundle their value with the value of existing varieties, for the commodity group or shipment in question, and calculate unit values. This method of ignoring the quality change and the mix of quality varieties within the commodity group makes, as discussed in Chapter 2, the resulting index subject to unit value bias. Indeed the information on customs documentation is insufficiently detailed for the price statistician to even be aware that changes in varieties are taking place, let alone make the changes required for proper price measurement. As such this chapter will be largely devoted to issues that arise from price measurement using establishment surveys.

8.8 The issue of missing prices is particularly severe for XMPIs as compared with CPIs and PPIs. For CPIs an item’s price is missing if it is not in the sampled outlet when the price collector visits the outlet. It may be out of stock or no longer sold. For a PPI, an item’s price is missing if the reporting establishment has not sold/purchased or priced the item in the period that constitutes the inquiry. It may have had no sales in that period and not be priced, or be no longer produced. However, for XMPIs, an item’s price is missing if the reporting establishment does not export/import (or price) the item in the period that constitutes the inquiry. It may still be produced/purchased for/from the domestic market, but it is still missing. Feenstra and Diewert (2000) investigated various imputation techniques for missing prices for XMPIs making use of a data set from the International Price Program of the United States Bureau of Labor Statistics (BLS) that consisted of all price quotes received from January 1997 to December 1999 at the most elementary “item” level. Included in this data set was an indicator variable for whether each price quote is imputed or not owing to being unavailable at the time of reporting. Of the 893,935 monthly observations at the elementary “item” level, over the three years of data more than one-third was imputed.

A.2 Why the matched-models method may fail

8.9 The matched-models approach to variety turnover described in Section A.1 is subject to three broad sources of error: (1) missing commodities (hereafter also including varieties of a commodity), (2) sample space change (sampling issues), and (3) new commodities. Missing commodities are concerned with the solution of the problem of “what to do when a commodity is no longer imported or exported in the month in question” by means of replacement varieties or imputations.

In solving the problem the methods attempt to preserve the original matched sample with one-on-one replacements. For sample space changes, the issue of concern is with incorporating the price changes of unmatched old and new models that are no longer traded or are introduced outside of the matched sample. However, when a very new commodity arrives, which cannot by definition be easily linked to existing ones, a different problem arises.

A.2.1 Missing commodities

8.10 Compilers measure the long-run price change for a commodity by comparing the price of the commodity in the current period, usually month, with its corresponding price in the price reference period. This reference period would be the month when the commodities in the index entered the sample. When a commodity is missing, it may be because it has been discontinued, or it may not be available to the same specification—its quality has changed. We thus encounter the first potential source of error in the matched-models method. There are several specific contexts for this. It may be a seasonal commodity, or the commodity may be a custom-made good or service supplied each time to a customer’s specification, or it may be that it is in short supply, or simply no longer demanded and supplied. There are four main approaches for dealing with missing commodities:

Approach 1: The price change of the discontinued commodity may be imputed by the aggregate price change of a group of other commodities whose price evolution compilers judge to be similar to that of the missing commodity. Such imputations should only be undertaken for short periods.

Approach 2: A replacement commodity may be selected, comparable in quality to the missing commodity, and its price used directly to form a price relative.

Approach 3: The replacement may be deemed noncomparable with the missing commodity, but prices of both the missing and replacement commodities may have been available in an overlap period before the commodity was missing. Compilers use the price difference in this overlap period to adjust the quality of the replacement commodity’s price until there are at least two observations on the replacement commodity.

Approach 4: The price of a noncomparable replacement may be used with an explicit adjustment for the quality difference to extract the pure price change.

8.11 This chapter discusses these four approaches to quality adjustment in some detail along with the assumptions they imply. Because the prices of the unavailable commodities are not measured by definition, the veracity of some of the maintained assumptions about their price changes, had they been available, is difficult to establish. Nevertheless, the objective of each of the methods is to produce matched comparisons of the prices of commodities: to compare like with like from month to month. When commodities are replaced with new ones of a different quality, then a quality-adjusted price is required to produce a match. If the adjustment is inappropriate, there is an error, and, if it is inappropriate in a systematic direction, there is a bias. Careful quality-adjustment practices are required to avoid error and bias.

A.2.2 Sampling issues

8.12 Sampling issues comprise four main areas of concern. First, samples lose relevance. A given set of matched models or commodities is likely to become increasingly unrepresentative of the population of transactions over time. There is evidence that the prices of old commodities when dropped are relatively low and the prices of new ones relatively high, and their prices are different even after quality adjustment. For strategic reasons, firms may wish to dump old models, to make way for the introduction of new models priced relatively high. Ignoring such unmatched old and new models in XMPI measurement will bias the indexes downward (see Section G.2.3 in this chapter). Ironically, the matched-models method that compilers employ to ensure constant quality may itself lead to bias, by ignoring such unmatched models, especially if used with an infrequently updated commodity sample.

8.13 Second, because of the additional resources required to make quality adjustments to prices, it may be in the interests of respondents and their counterparts in statistical offices to avoid making noncomparable replacements and quality adjustments. They keep with their commodities until they are no longer produced—that is, they continue to monitor old commodities with limited sales. Such commodities may exhibit unusual price changes as they near the end of their life cycle. These unusual price changes arise because marketing strategies typically identify gains to be made from different pricing strategies at different times in the life cycle of commodities, particularly at the introduction and end of the cycle. Yet their weight in the index, which is based on their export revenue/cost share when they were sampled, would remain constant in the index and probably would be too high at the end of the life cycle. Further, new, and therefore unmatched commodities with possibly large sales would be ignored. Undue weight would be given to the unusual price changes of matched commodities at the end of their life cycle. This issue again is resolved by more frequent sample reselection of commodities within a given sample of establishments.

8.14 Third, the methodology for selecting replacement commodities may be such that respondents are advised to choose comparable replacements to avoid the need for explicit quality adjustments to price. Obsolete commodities are by their nature at the end of their cycle, and replacements, to be comparable, must also be near or at the end of their cycles. Obsolete commodities with unusual price changes at the end of their cycle may be replaced by other obsolete commodities with unusual price changes. This compounds the problem of unrepresentative samples and continues to bias the index against technically superior commodities delivering cheaper service flows.

8.15 Finally, the sampling problem with the matching procedure occurs when the respondent continues to report prices of commodities until replacements are forced, that is, until the commodities are no longer available, but the respondent has instructions to replace them with popular commodities. This improves the coverage and representativity of the sample. But the wide disparity between the characteristics of the old, obsolete commodities and new, popular ones makes accurate quality adjustment more difficult. The (quality-adjusted) price changes of very old and very new commodities may not be similar as required by the imputation methods under approach 1. The differences in quality are likely beyond what can be attributed to price differences in some overlap period under approach 3, because one commodity is in the last stages of its life cycle and the other in its first. Further, the technical differences between the commodities are likely to be of an order that makes it more difficult to provide reliable, explicit estimates of the effect of quality differences on prices under approach 4. By implication, many of the methods of dealing with quality adjustment for unavailable commodities will work better if the switch to a replacement commodity is made sooner rather than later. Sampling issues thus are closely linked to quality-adjustment methods. This will be taken up in Chapter 9, in the section on commodity selection and the need for an integrated approach to dealing with both representativity and quality-adjusted prices.

A.2.3 New commodities

8.16 The third potential source of error is distinguishing between new commodities and quality changes in old ones, also covered in this chapter. When a truly new commodity is introduced, there are at least two reasons why early sales are at high prices that later fall, often precipitously: capacity limitations and market imperfections. Both of these may be present shortly after introduction of a new commodity because there are few suppliers for it.

8.17 Early in the commodity life cycle, production processes may have limited capacity; therefore, producers find themselves operating at relatively high and increasing marginal costs of production. Marginal costs of operation tend to decline as more producers enter the market or as existing producers redesign and upgrade production facilities for higher volume. Both of these bring operating levels back from high marginal cost, near full capacity levels.

8.18 With or without early capacity constraints, the small number of suppliers early in the life cycle allows what economists call market imperfections to arise. In an imperfectly competitive market, the producer can charge a monopoly price higher than the marginal cost of production. As more competitors enter the market for the new good or service, the monopoly power of early sellers decreases and the price tends to drop toward marginal cost. Further, the market may be one in which subgroups are willing to pay a premium for a new variety/commodity and suppliers will accordingly practice price discrimination.

8.19 The initially high price at introduction and its full subsequent decline would not be brought into the index fully by the usual methods. Compilers commonly either wait until the index is rebased or until a commodity in the sample becomes unavailable to seek a replacement commodity and admit the possibility of detecting a new good. After capacity constraints or monopoly profits diminish, subsequent price changes may show little difference from other broadly similar commodities. Standard approaches thus wait too long to pick up these early downward trends in the prices of new goods.

8.20 At the extreme, capturing the initial price decline requires a comparison between the first observed price and a hypothetical price for the period before its introduction. The hypothetical price would be the price below which there would be no positive market equilibrium quantity bought and sold.2 Again, frequent resampling offers the possibility of catching new goods early in the commodity cycle when their prices are high and market share relatively low, thereby capturing early price declines as producers relieve capacity constraints and new entrants compete market imperfections away.

8.21 Finally, it is important to emphasize that there is not only a price decline but also a market share increase in the stylized commodity life cycle. Frequent resampling and focused scanning for new commodities should be at least somewhat effective in capturing the price declines in early commodity cycles. Compilers face a potentially serious problem, however, if they have no market share information to go with the prices. The stylized facts of the commodity cycle are that a new commodity comes in at a high price and a low market share. The price then declines and market share increases. Both prices and market share then stabilize for a period, until a successor commodity emerges at a high price and low market share and then begins to take market share from the now mature existing commodity. Early and normally large price declines for new commodities thus should figure into the elementary aggregate price index at a relatively low weight, while later and normally smaller price declines figure in at a successively higher weight. Without current market share data, early price declines may well be overemphasized and the growth in the price index for the elementary aggregate underestimated.

A.3 Temporarily missing commodities

8.22 Commodities that are temporarily missing are not available and thus not priced in the month in question, but are expected to be priced in subsequent months. The lack of availability may be because, for example, inventories are insufficient to meet demand, or material inputs are seasonal, as is the case with some fruits and vegetables for food canning. There may also be shortages. The treatment of seasonal goods is the subject of Chapter 23. Such goods may go missing in some months, but are distinguished by the fact that they are expected to reappear, at a similar level of quality, in the next season. It may be that commodities are only temporarily missing because they are switched in a particular month to be sold to or purchased from the domestic market, thus not figuring in export and imports.

8.23 Standard practice for good survey management requires that seasonal commodities be separately identified by the respondents as “temporarily missing” or “seasonal,” so compilers can remain alert to the commodity’s reappearance later in the year. Principles and methods for dealing with such commodities are outlined in Armknecht and Maitland-Smith (1999), Feenstra and Diewert (2000), and Chapter 23.

A.4 Outline for the remainder of the chapter

8.24 Section B considers further what is meant by quality change and then considers conceptual issues for the valuation of quality differences. The meaning of quality change requires a conceptual and theoretical platform so that adjustments to prices for quality differences are made against a well-considered framework. Readers interested only in methods of quality adjustment will find them in Sections C through G. Section C provides an overview of the methods available for dealing with unavailable price observations. Methods for quality-adjusting prices are classified into two types: implicit and explicit adjustments, covered in greater depth in Sections D and E, respectively. Section F considers how to choose among methods of quality adjustment.

8.25 The implicit and explicit adjustment methods to be used when matching fails—when matched models are missing—are first outlined. However, where commodities are experiencing rapid technological change, these methods may be unsuitable. The use of imputations and patching in of quality-adjusted replacement prices when the matching fails is appropriate when failed matches are the exception. But in high-technology commodity markets likely to experience rapid turnover of models, they are the rule. Section G considers alternative methods using chained or hedonic frameworks to meet the needs of rapidly changing production and purchasing portfolios. Section H examines frequent resampling as an intermediary, and for imputation a more appropriate, approach. Chapter 23 discusses issues relating to seasonal commodities in more detail.

B. What Is Meant by Quality Change

8.26 Before turning to the methods of quality adjustment, the nature of quality change is first discussed, along with a brief outline of the conceptual basis for the indices. Sections C to G include material on the methods price statisticians might employ in dealing with the problem of quality adjustment. In choosing between, and applying, some of these methods, arguments may be made concerning the conceptual basis as to how quality should be valued.

B.1 Nature of quality change

8.27 Bodé and van Dalen (2001) undertook an extensive study of the prices of new automobiles in the Netherlands between 1990 and 1999. The average price increase per car over this period was about 20 percent, but the mix of average quality characteristics changed at the same time. For example, the horsepower (HP) of new cars increased on average from 79 to 92 HP; the average efficiency of fuel consumption improved from 9.3 to 8.4 litres/100 km; the share of cars with fuel injection went from 51 percent to 91 percent; the share of cars with power steering went from 27 percent to 94 percent; and the share of cars with airbags went from 6 percent to 91 percent. There were similar increases for central locking, tinted glass, and many more features.

8.28 Standard price index practice matches the prices of a sample of models in, for example, January with the same models in subsequent months. This holds the characteristics mix constant to keep quality differences from contaminating the estimate of price change. However, as considered later in this chapter, the resulting sample of matched models (commodities) is one that gives less weight (if any) to models subsequently introduced. Yet the later models benefit from more recent technological developments and may have different price changes given the quality of services they provide. One approach to correct for such quality changes using the whole sample of both new and existing models is a dummy variable hedonic regression (see Section G.2.1). Bodé and van Dalen (2001), using a variety of formulations of hedonic regressions, found the quality-corrected prices of these new automobiles to be about constant over this period. In this case, the value of the quality improvements explained the entire nominal price increase.

8.29 Recorded changes in prices are the outcome of shifts in both demand and supply. Chapter 22 explains that these shifts arise from a number of sources, including environmental changes; changes in users’ technology, tastes, and preferences; and changes in producers’ technology. More formally, the observed data on prices are the loci of the intersections of the demand curves of different final users with varying tastes or intermediate users with possibly varying technologies, and the supply curves of different producers with possibly varying technologies. Separately identifying the effects of changes in environment, technology, and tastes and preferences on the spectrum of commodity characteristics present in markets at any given time is conceptually and empirically difficult. Fortunately, compilers do not have to separately identify these effects to produce a good price index in the face of quality change. They need only to identify their overall impact.

8.30 Our concern is not just with the changing mix of the observed characteristics of commodities. There is the practical problem of not always being able to observe or quantify characteristics, such as style, reliability, ease of use, and safety. There are other, less obvious, differences in quality, including different times of the day or periods of the year. For example, electricity3 or transport provided at peak times must be treated as being of higher quality than the same amount of electricity or transport provided at off-peak times. The fact that pricing peaks exist shows that purchasers or users attach greater utility to the services at these times.

8.31 It may be that the commodities, although essentially the same, differ in other respects, say, the warranties attached, reliability or speed of delivery, or versatility of batch size. The matter is of some importance. If the item is the same in all respects and a higher price is obtained by, say, exporting to country A rather than B, then a switch of some output from A to B is essentially a price and revenue increase from the same output. The measurement approach would be to add values of exports of A and B together and divide by the total quantity. However, if they are considered to be different commodities, then the aggregate price change is a weighted average of the price changes of the two countries.

8.32 There is a very strong likelihood some price-determining characteristics will be unmeasured in any quality-adjustment situation. Compilers cannot produce timely statistics if they are perpetually seeking more data on characteristics to produce a still better quality adjustment. How many characteristics data are enough? Characteristics data are sufficient when commodities are described completely enough. Commodities are described completely enough when there is low variability of prices over transactions with that description in any given month. If we use characteristics from a structured commodity description to estimate a hedonic regression model, the model will fit well only if the structured descriptions are reasonably complete. The first criterion for sufficiency of structured characteristics data, then, is a good fit to a hedonic model. If there is a good fit using a set of objective characteristics, there may be still other characteristics, such as style and reliability, not yet included in the structured description and thus unmeasured, but they cannot contribute much more to the fit of the model. A second, qualitative criterion is that the included characteristics be meaningful to the participants in the market for the commodity.

B.2 Conceptual issues

8.33 An export price index (XPI) from a resident producing unit’s perspective is an index designed to measure the average change in the price of goods and services exported. Section B.2.1 focuses on a conceptual framework for the quality-adjustment problem for XPIs and the restrictive assumptions that have to be maintained to use the resource cost approach. The principles relating to an import price index (MPI) follow in Section B.2.2. It outlines the quality-adjustment framework for MPIs and the restrictive assumptions that have to be maintained to use the user value approach to quality adjustment. The discussion continues in Section B.2.3 with a brief introduction to two problems associated with resource cost and user value approaches. The first, in Section B.2.4, is when technology substantially changes and fixed-input output export indices make little sense for valuing higher-quality commodities produced at much lower unit cost. The second is the reconciliation problem in national accounts at constant prices, a problem that leads this Manual to recommend a unified valuation system in Section B.2.5.

B.2.1 Fixed-input output export price index

8.34 In this Manual, the principal conceptual basis for the XPI is the fixed-input output export price index (FIOXPI). It is an XPI that is based on the resident producers’ perspective—that is, an output one as opposed to a nonresident-producing establishment that treats exports as uses or inputs. The FIOXPI thus aims to measure an output export price index constructed on the assumption that inputs and technology are fixed.4Chapter 18 defines the economic theoretical XPI as a ratio of revenue functions. The revenue function of an establishment expresses the value of its output for export as a function of the prices it receives and the quantities of inputs required to produce the output. It recognizes that only a finite number of varieties or commodities are producible for export at any given time, but also grants that, for given inputs and technology, there may be a continuum of designs from which producers select this finite number of commodities. Hence, in response to changes in preferences or the technologies of producers using a given establishment’s output, there may be different sets of commodities produced from period to period from a given set of inputs and technology.

8.35 Compilers and price index theorists are used to thinking in the narrower framework comparing the prices of exactly the same things from period to period. For example, they would (want to) measure the price change of shirts exported on the assumption that the cutting, sewing, folding, packaging, and so forth were all undertaken in the same way from the same labor, capital, and material inputs in the two periods being compared. If the revenue from exports increased by 5 percent, given that everything else remained the same, then this is the change in the output export price. If everything else does not change, then a measure of a “pure” price change results.

8.36 Even if technology and inputs remain the same, however, the way things are produced and sold may change. For example, the shirtmaker may start improving the quality of its shirts by using extra cloth and more stitching using the same machinery. The “price basis” or commodity description underlying this comparison has changed within a given technological framework. A direct comparison of shirt prices in successive months includes in this case not only the effects on revenue from price changes from exports, but also changes in commodity characteristics and thus quality. To include the increase in revenue resulting from improved quality would be to misrepresent price change—to bias the index upward. Prices would not, in fact, be rising as fast as indicated by such an unadjusted index.

8.37 A pure price relative for a commodity fixes the commodity description or price basis by definition. For the price basis not to change, the commodity’s observable characteristics and the manner in which the commodity is sold for export must remain fixed. The FIOXPI for an elementary aggregate may evolve because producers adjust quantities and revenue shares in response to changes in the relative prices of commodities. Further, “new” commodities that are feasible with the same inputs and technology, but were not previously produced, may appear and supplant existing commodities.

8.38 A variant of the FIOXPI framework underlies the resource cost approach to explicit quality adjustment for export prices. In the resource cost approach, when quality changes, the compiler asks the establishment representative how much it cost to produce for export the new commodity and how much it would have cost to produce the old commodity for export in the current period. She then divides the price relative between the new and the old commodities by their relative cost. Resource cost adjustment relies on holding input prices relative to total cost fixed, rather than holding input quantities fixed, when comparing the prices for a given set of commodities between two periods. This variant of the FIOXPI is based on the concept of a ratio of indirect revenue functions, so named because they maximize revenue subject to a cost function constraint rather than a production function constraint.5 Although the direct revenue function of the FIOXPI increases with inputs, the indirect revenue function increases with total cost. If commodity characteristics change along with prices, the resource cost adjustment for the change in quality is the factor that, when used as a multiplier for observed total cost, would produce the same revenue (given the initial set of commodity characteristics) as the revenue realized through producing the new commodities in the current period. Thus, if the new good is of a higher quality, we would expect this cost multiplier to be positive, and the cost of producing the old commodity in the current period to be less than the cost of producing the new commodity. The cost relative between the two commodities, thus, is greater than one, and, when divided into the price relative between them, lowers the estimate of price change by the percentage value of the quality increase.

B.2.2 Utility maximization and cost minimization—fixed-output input import price index and other indices

8.39 This Manual has two principal conceptual bases for import price indices. The first is from the perspective of the resident householder directly importing goods and services for final demand. Approaching the consumer price index from the standpoint of economic theory has led to the development of the concept of a cost of living index (COLI). The theory of the COLI was first developed by Konüs (1924). It rests on the assumption of optimizing behavior on the part of a rational consumer. The COLI for such a consumer has been defined succinctly as the ratio of the minimum expenditures needed to attain the given level of utility, or welfare, under two different price regimes. Details of the theory are provided in Chapters 17 and 21 of the CPI Manual and will not be duplicated here. Given that the theoretical COLI measures price changes in terms of expenditure changes necessary to maintain a constant level of utility, it follows that relating, at a conceptual level, issues of quality to the utility derived from the quality change is theoretically appropriate for price measurement.

8.40 The second approach to MPIs is taken from the perspective of the resident producer and is the fixed-output input import price index (FOIMPI). It is an MPI that is based on the resident producer’s perspective of purchasing imports for intermediate consumption—that is, an input one—as opposed to a nonresident producing establishment that treats imports as outputs.6 It is the relative change in the cost of imports—the market value of inputs—required to produce a fixed level of output, when input prices change between the current period and a base period. Assuming producers minimize the cost of producing output, the input import price index thus is a ratio of cost functions that relate establishment total production cost to its outputs and the input import prices it pays.7 The prices of imported inputs should include all of the amounts purchasers pay per unit of the commodities they use, including transportation, insurance, wholesale/retail margins, and indirect taxes. Chapter 15 calls these purchasers’ prices, following the Commission of the European Communities and others (2008), System of National Accounts 2008 (2008 SNA).

8.41 A variant of the FOIMPI framework underlies the user value approach to explicit quality adjustment for input prices. User value adjustment relies conceptually on a variant of the FOIMPI. It holds output prices fixed relative to total revenue, rather than holding output quantities fixed, when comparing the prices for a given set of imported input commodities between two periods. The variant is based on the concept of a ratio of indirect cost functions, so named because they minimize cost subject to a revenue function constraint rather than a production function constraint.8 Whereas the direct cost function of the FIOMPI increases with outputs, the indirect cost function increases with total revenue. If commodity characteristics change along with prices, the user value adjustment for the change in quality is the factor that, when multiplied by observed total revenue, would produce the same cost in the current period (given the initial set of commodity characteristics) as the cost realized using the new commodities as imported inputs. Thus, if the new imported input is of a higher quality, we would expect this revenue multiplier to be positive, and the revenue possible from using the old commodity in the current period to be less than the revenue realized from using the new commodity. The revenue relative between the two commodities thus is greater than one, and, when divided into the input price relative between the two input commodities, lowers the estimate of their price change by the percentage value of the quality increase.

8.42 Triplett (1990, pp. 222–23) summarized the history of thought on the resource cost and user value methods of quality adjustment:

Fisher and Shell (1972) were the first to show that different index number measurements (they considered output price indexes and consumer price indexes) imply alternative treatments of quality change, and that the theoretically appropriate treatments of quality change for these two indexes correspond respectively, to “resource-cost” and “user-value” measures. Triplett (1983) derives this same result for cases where “quality change” is identified with characteristics of goods—and therefore with empirical hedonic methods [discussed later]; the conclusions are that the resource-cost of a characteristic is the appropriate quality adjustment for the output price index, and its user-value is the quality adjustment for the COL index or input index.

Intuitively, these conclusions are appealing. The output index is defined on a fixed value of a transformation function. The position of a transformation function, technology constant, depends on resources employed in production; accordingly, “constant quality” for this index implies holding resources constant, or a resource-cost criterion.

On the other hand, the COL index is defined on a fixed indifference curve, and the analogous input-cost index is defined on a fixed (user) production isoquant. For these two “input” price indexes, “constant-quality” implies holding utility or output constant, or a user-value criterion….

B.2.3 A problem with these concepts and their use

8.43 Chapter 18 recognizes the FIOXPI as the appropriate basis for an XPI and the FOIMPI for the MPI if a resident’s perspective is utilized.

8.44 As shown in Section B.2.1, the resource cost method has a microeconomic rationale within the indirect revenue framework for quality-adjusted output export price measurement. However, the correctness of dividing a price relative by a resource cost ratio for a given commodity requires two potentially restrictive assumptions. The production process for the export commodity whose price is adjusted must be separable from the process for the rest of the outputs of an establishment, and the returns to scale of that process must be constant and equal to one.9 These assumptions would be unlikely to be confirmed were the data available to empirically test them (and these data usually are not available to compilers).

8.45 As shown in Section B.2.2, the user value method also has a microeconomic rationale within the indirect cost framework for quality-adjusted input import price measurement. However, the correctness of dividing a price relative by a user value ratio for a given commodity requires two potentially restrictive assumptions. The input requirements for the item whose price is adjusted must be separable from the requirements for the rest of the inputs an establishment uses, and the returns to scale of that process must be constant and equal to one. These assumptions would be unlikely to be confirmed were the data available to empirically test them (and these data usually are not available to compilers).

B.2.4 When technology changes

8.46 The problems with traditional resource cost and user value approaches to explicit quality adjustment compound in the presence of technical (and taste) change. Throughout the earlier sections, this chapter has noted the similarity of effects on XMPIs between relative price change, preference change, change in the using technology, and change in the supplying technology. Broadly, all affect the assortment of commodities available at any given time and the relative importance of the commodities in the subset of that assortment persisting from period to period. As noted in Chapter 18, however, changes in weights arising from suppliers’ and users’ responses to relative price changes given fixed technology and preferences have predictable outcomes. They are the foundation for well-known theorems on the downward (upward) bias of Laspeyres price indices and the upward (downward) bias of Paasche price indices for output (input) price indices. Normally, considering substitution effects alone leads to the standard expectation that the Laspeyres output (input) price index will lie below (above) the Paasche output (input) price index.

8.47 The export and import value shares compilers observe and use as weights reflect changes in relative prices, technology, and tastes simultaneously. Changes in the relative importance of commodities, including their emergence and disappearance, can be unpredictable. Technology change can augment the substitution effects from relative price change, or it can more than offset substitution effects. As a result, the Laspeyres output price index may lie above the Paasche output price index, and the Laspeyres input price index may lie below the Paasche input price index in any given period-to-period comparison.

8.48 Regarding the resource cost method, an establishment representative can find it problematic to assess the cost of changes in the price basis of an output good or service arising partly or wholly from a change in production technology. Much of the cost of the improved reliability, efficiency, design, flexibility, durability, and other characteristics are difficult to measure. Moreover, the changes in technology that generate them include changes in plant and machinery, quality monitoring, inventory control, labor requirements, the manner in which work is organized, types of materials used, packaging, and selling techniques, all of which are difficult to measure in terms of the simple costing referred to above. The new technologies in high-technology commodities require new methods of production. These production technologies may change, possibly more than once during a year. Asking the cost of a previous variety produced under the current production process or the cost of the current variety under the previous generation process may be conceptually appropriate, but practically impossible. Yet not answering the cost question under the condition that technology is fixed in the current or previous generation can produce wildly inaccurate results. Consider the market for personal computers, where price declines have been accompanied by rapid quality improvements.

8.49 Holdway (1999) illustrated the problem of using a fixed-input output price index (FIOPI) for computer microprocessing units (cpus) such as an Intel Pentium III. He considered changes in the speed of new generations of microprocessors and used the example of the transition from 66 megahertz (MHz), costing $230 when it was discontinued, to be replaced by the 90 MHz model valued at $247 in the same month. The additional cost of the 24 MHz at that month’s technology’s resource costs has to be estimated. Say the cost of a single unit MHz was estimated to be $2.0833, which when multiplied by 24 equals $50. So what is the pure price difference between these two cpus? To make the new 90 Mhz cpu equivalent to the old 66 MHz one, the $50 has to be subtracted from its price and compared with the price of the old one, that is, [(247 – 50)/230] – 1 = –0.143, a 14.3 percent fall. This is instead of a nominal price increase of [(247/230) – 1] = 0.074 or 7.4 percent.

8.50 Suppose, however, the establishment reports the unit cost of the 66 MHz unit at the technology prevailing when the older, slower unit was designed rather than the unit cost of a 66 MHz unit from the newer technology underlying the 90 MHz chip. In this case it is very easy to misapply the resource cost method by not comparing costs within a given generation of production technology. The new 90 Mhz cpus were built using a better technology. They used 0.50 as opposed to 0.80 micron technology, allowing more features to be packed into a smaller section of a silicon wafer, which improved performance. Also, the technology used to produce them, including an amortization factor for plant and capital equipment, lowered unit costs (see Holdway, 1999, for details). Say an estimate was requested as to how much extra it would cost to produce a 90 Mhz cpu than a 66 Mhz one, while maintaining that the cost assessment should assume the 66 Mhz wafer technology. Suppose unit costs for the higher-performance cpu were $100 more because the old technology was less efficient than the new technology, a common occurrence in high-technology industries. Application of the resource cost method now provides an estimate of (247)/(230 + 100) – 1 = –.252, a 25.2 percent decrease.

8.51 In the latter cases the method breaks down. The unadjusted price increase was 7.4 percent. With a resource cost adjustment using estimates based on the new technology there was a decline of 14.3 percent. Adjusting the prices based on estimates using the old technology to produce the new, higher-performing chip results in a decrease of 25.2 percent. In both cases the cost declines represent different levels of technology, and the resource cost approach can give widely different answers. In the computer and electronics and other industries where unit prices are falling and technology rapidly changing, resource cost quality-adjustment procedures can be very misleading as major technology shifts occur.

8.52 PPIs, like XMPIs, cannot, of course, hold the price basis constant over very long periods. For example, in the 45 years since the introduction of the commercial computer the price of computing power has been estimated to be less than one-half of one-tenth of 1 percent (0.0005) of what it was at its introduction. It has decreased by more than two thousand fold (Triplett, 1999). Yet if these price changes reflected overall changes in producer prices, absurd estimates of output growth at constant prices would result. The tastes and expectations of consumers along with the technology of the producers change over time, and these changes will be shown in Chapter 22, Section H, to affect the implicit prices attributed to the quality characteristics of what is bought and sold.

8.53 Because of the effects of relative price change, technology change, and taste change, we again would prefer to use the (observed) overlap price and hedonic methods, if it is feasible to use them, rather than the resource cost and user value approaches. Further, rapid technical and taste changes must also be met by more frequent sample updates to avoid rapid loss of sample relevance.

B.2.5 Consistency between supply and use price statistics: Assessing commodity quality at supply and use values

8.54 Quality assessments must be consistent throughout the supply and use accounts for goods and services. As discussed in Chapter 15, the XMPIs cover aggregates in the supply and use tables of the 2008 SNA, balancing the sources of goods and services supply in the current period with the uses of those goods and services. The sources of supply are domestic production and imports, plus adjustments for transport and distribution services to get goods to their users and taxes and subsidies on commodities. The uses of goods and services are intermediate consumption, final consumption, capital formation, and exports. Each good or service commodity thus has its own row in the matrix of supply and use, whose columns are the aforementioned components of supply and use. Even at this highest level of detail, the supply of every distinct good or service, adjusted for transport and distribution margins and taxes, must balance its uses. This will be identically true in both value and volume terms.

8.55 Because every transaction cannot be tracked, however, supply and use tables cannot be produced at the level of elementary items. It is feasible to track supply and use only at the level of elementary aggregates, basic headings, or even higher-level aggregates of goods and services. Thus, each row of such a supply and use table necessarily contains some quality heterogeneity and we can speak of it only in average terms. Changes in the total supply and total uses of these detailed aggregates of goods and services thus comprise four parts. There are average quality changes, changes in basic prices, changes in taxes and subsidies on commodities, and average quantity changes of the elementary commodities comprising the aggregate. Volume change for an aggregate is an amalgam of quality and quantity changes. Clearly, adjusting price change to eliminate the effects of changes in quality is important here, lest volume be understated or overstated by the amount of quality change erroneously ascribed to price change. The context also highlights the need to have a single valuation of quality change, not one from the supply side—for example, imports of a good—and one from the uses side—intermediate consumption of that same good. Thus, as outlined in the 2008 SNA, Chapter 15, similar estimates should be used for supply and use quality adjustments if the supply and use accounts are to balance in both value and volume terms.

C. An Introduction to Methods of Quality Adjustment When Matched Items Are Unavailable

C.1 Introduction

8.56 It may be apparent from the preceding text that quality adjustments to prices are not going to be a simple issue or involve routine mechanical methods whereby a single methodology will be applied to prices in all commodity groups to yield adjustments. A number of alternative approaches will be suggested, and some will be more appropriate than others for specific items regardless of their commodity group. An understanding of the technological features of the exporting/importing industry, the commodity market, and alternative data sources will be required for the successful implementation of a quality-adjustment program. Specific attention must be devoted to commodity areas with relatively high weights and where large proportions of commodities are turned over. Some of the methods are not straightforward and require some expertise, although methods learned and used on some commodities may be applicable elsewhere. The issue of quality adjustment is met by developing a gradual approach on a commodity-by-commodity basis. It is emphasized that such concerns should not be used as reasons to obviate the estimation of quality-adjusted prices. The practice of statistical agencies in dealing with missing commodities, even if it is to ignore them, implicitly involves a quality adjustment, and the form of the implicit one undertaken may not be the most appropriate one and may even be misleading. The extent of quality changes and the pace of technological change require that appropriate methods be used.

8.57 To measure aggregate price changes, a representative sample of commodities is selected along with a sample of exporting/importing firms and a host of details that define each price, including details on the conditions of the sale where relevant. This is to establish an insight into the price basis of the commodity. This is then followed by a periodic survey for which the firms report prices (r eprice the commodity) each month for these selected commodities. They do so to the same specifications, that is, on the same price basis. The detailed specifications are included on the repricing form each month as a prompt to ensure that the price basis has remained the same. Respondents must be aware of the need to report the details of any change in the price basis; confusion may lead to biased results. It must be borne in mind that firms have no incentive to report such changes because this will invariably involve additional work in costing the change. Attention should also be devoted to ensuring that the description of the price basis contains all pertinent, price-determining elements. If an element is excluded, any change is much less likely to be reported. In both of these cases, the quality change would be invisible to the price-measurement process.

C.2 Methods for making quality adjustments

8.58 When a commodity is missing in a month for reasons other than being off-season or off-cycle, the replacement may be of a different quality—the price basis may have changed, and like may be no longer compared with like. A number of approaches exist for dealing with such situations. Though the terms differ among authors and statistical agencies, they include the following:

  • Imputation—Used when no information is available to allow reasonable estimates to be made of the effect on price of a quality change. The price change of all commodities—or of more or less similar commodities—is assumed to be the same as that for the missing commodity.

  • Overlap—Used when no information is available to allow reasonable estimates to be made of the effect on price of a quality change, but a replacement commodity exists in the same period as the old commodity. The price difference between the old commodity and its replacement in the same overlap period is then used as a measure of the quality difference.

  • Direct comparison—Used if another commodity is directly comparable, that is, so similar it has more or less the same quality characteristics as the missing one. Its price then replaces the unavailable price. Any difference in price level between the new and old is assumed to be because of price changes and not quality differences.

  • Explicit quality adjustment—Used when there is a substantial difference in the quality of the old and replacement commodities. Estimates of the effect of quality differences on prices are made to enable quality-adjusted price comparisons.

8.59 Before these methods are outlined and evaluated, the extent of the problem should be addressed. This situation arises when the commodity is unavailable. It is not just a problem when comparable commodities are unavailable, for the judgment as to what is and what is not comparable itself requires an estimate of quality differences. Part of the purpose of a statistical meta-information system for statistical offices (outlined in Chapter 9) is to identify and monitor the sectors that are prone to such replacements and determine whether the replacements used really are comparable.

8.60 Quality-adjustment methods for prices are generally classified into the implicit or imputed (indirect) methods explained in Section D (the differences in terminology are notorious in this area) and explicit (direct) methods explained in Section E. Both decompose the price change between the old commodity and its replacement into quality and pure price changes. However, in the latter, an explicit estimate is made of the quality difference, usually on the basis of external information. The pure price effect is identified as a remainder. For implicit adjustments, a measurement technique is used to compare the old commodity with the replacement, so that the extent of the quality and pure price change is implicitly determined by the assumptions of the method. The accuracy of the method relies on the veracity of the assumptions, not the quality of the explicit estimate. In Sections D and E, the following methods are considered in detail:

Implicit methods:

  • Overlap,

  • Overall mean/targeted mean imputation,

  • Class mean imputation,

  • Comparable replacement,

  • Linked to show no price change, and

  • Carryforward.

Explicit methods:

  • Expert judgment,

  • Quantity adjustment,

  • Differences in production/option costs, and

  • Hedonic approach.

C.3 Some points

C.3.1 Additive versus multiplicative

8.61 The quality adjustments to prices may be undertaken by either adding a fixed amount or multiplying by a ratio. For example, where m is the old commodity and n its replacement for a comparison over periods t, t + 1, t + 2, the use of the overlap method in period t + 1 required the ratio pnt+1/pmt+1 to be used as a measure of the relative quality difference between the old item and its replacement. This ratio could then be multiplied by the price of the old item in period t, pmt to obtain the quality-adjusted prices pm*t shown in Table 8.1. Such multiplicative formulations are generally advised because the adjustment is invariant to the absolute value of the price. It would be otherwise possible for the absolute value of the change in specifications to exceed the value of the commodity in some earlier or—with technological advances—later period. Yet for some commodities, the worth of the constituent parts is not in proportion to their price. Instead, they have their own intrinsic, absolute, additive worth, which remains constant over time. Producers selling over the Internet may, for example, include postage, which in some instances may remain the same irrespective of what is happening to price. If postage is subsequently excluded from the price, the fall in quality should be valued as a fixed sum.

Table 8.1.

Estimating a Quality-Adjusted Price

article image

C.3.2 Base- versus current-period adjustment

8.62 Two variants of the approaches to quality adjustment outlined in Section C.2 are to either make the adjustment to the price in the base period or make the adjustment to the price in the current period. For example, in the overlap method described above, the implicit quality-adjustment coefficient was used to adjust pmt. An alternative procedure would have been to multiply the ratio pmt+1/pnt+1 by the prices of the replacement commodity pnt+2 to obtain the quality-adjusted prices pn*t+2, and so forth. The first approach is easier because once the base-period price has been adjusted, no subsequent adjustments are required. Each new replacement price can be compared with that of the adjusted base period. For multiplicative adjustments, the end result is the same whichever approach is used. For additive adjustments, the results differ. It is more appropriate to make the adjustment to prices near the overlap period.

C.3.3 Long-run versus short-run comparisons

8.63 Much of the analysis of quality adjustments in this Manual has been undertaken by comparing prices between two periods (e.g., periods 0 and 1). For long-run comparisons, suppose the base period is taken as period t and the index is compiled by comparing prices in period t first with t + 1, then with t + 2, then with t + 3, and so forth. The short-run framework allows long-run comparisons—say, between periods t and t + 3—to be built as a sequence of links joined by successive multiplication—say, period t with t + 2 and period t + 2 with t + 3. This can also be done by chaining period t with t + 1, t + 1 with t + 2, and t + 2 with t + 3. In Section H, the advantages of the short-run framework for imputations and to facilitate the introduction of new commodities are outlined. In Section G.3, chained indices and a hedonic approach are considered for industries experiencing a rapid turnover in commodities. These quality-adjustment methods are now examined in turn, and in Section F, the choice of method is discussed.

C.3.4 Statistical metadata

8.64 In Sections D and E, implicit and explicit methods of quality adjustments to prices are discussed. In Section F, the choice between these methods is examined. Any consideration of the veracity of these methods, resource implications, and the choice between them needs to be informed by appropriate information on an commodity-by-commodity basis. Section C of Chapter 9 considers information requirements for a strategy for such quality adjustment that makes use of a statistical metadata system.

D. Implicit Methods

D.1 Overlap method

8.65 Consider an example where the items are sampled in January and prices are compared over the remaining months of the year. Matched comparisons are undertaken between the January prices and their counterparts in successive months. Five commodities are assumed to be sold in January with prices p11,p21,p51,p61,andp81 (Table 8.2, panel a). Two types of similar commodities are produced in the commodity group concerned, A and B. An index at the elementary level is required for the overall price change of these two commodity types. At this level of aggregation, the weights can be ignored. A price index for February compared with January = 100.0 is straightforward in that prices of commodities 1, 2, 5, 6, and 8 are used and compared only by way of the geometric mean of price ratios, known as the Jevons index (which is equivalent to the ratio of the geometric mean in February over the geometric mean in January—see Chapter 21). In March, the prices for commodities 2 and 6—one of type A and one of type B—are missing.

Table 8.2.

Example of Overlap Method of Quality Adjustment

article image
Note: Figures in bold are imputations/estimates that may be made by the statistical authorities, as explained in the text.

8.66 In Table 8.2, the lower panel (b) is a numerical counterpart of the upper panel (a), further illustrating the calculations. The overlap method requires prices of the old and replacement commodities to be available in the same period. In Table 8.2(a), commodity 2 has no price quote for March. Its new replacement is, for example, commodity 4. The overlap method simply measures the ratio of the prices of the old and replacement commodity in an overlap period. In this example, the period is February, and the old and replacement commodities are commodities 2 and 4, respectively. The ratio of their prices is taken to be an indicator of their quality differences. The two approaches outlined in Section C.3.2 are apparent: either to insert a quality-adjusted price in January for commodity 4 and continue to use the replacement commodity 4 series, or to continue the commodity 2 series by patching in quality-adjusted commodity 4 prices. Both yield the same answer. Consider the former. For a Jevons geometric mean from January to March for commodity type A only, assuming equal weights of unity,

PJ(p1,p3)=[p13/p11×p43/((p42/p22)×p21)]1/2=[6/4×8/7((7.5/6)×5)]1/2=1.386.(8.1)

8.67 Note that the comparisons are long-run ones, that is, they are between January and the month in question, March. The short-run (modified) Laspeyres framework provides a basis for short-run changes based on data in each current month and the immediately preceding one. For example, January to October prices for each commodity are compared in terms of January to September and September to October. In Table 8.2(a) and (b), the comparison for commodity type A would first be undertaken between January and February using commodity 1, and then multiplied again by the price change between March and February for commodity 1. Then turn to commodity 2. The price change would first be undertaken between January and February. The result would be multiplied by the comparison between February and March using item 4, commodity 2’s replacement. Still, this implicitly uses the differences in prices in the overlap in February between items 2 and 4 as a measure of this quality difference. It yields the same result as before:

[54×65]12×[65×87.5]12=1.386.

8.68 The advantage of recording price changes for, say, January to October in terms of January to September and September to October is that it allows the compiler to compare immediate month-on-month price changes for data editing purposes. Moreover, it has quite specific advantages for the use of imputations as discussed in Sections D.2 and D.3 for which different results arise for the long- and short-run methods. It further facilitates the introduction of replacement commodities. A fuller discussion of the long-run and short-run frameworks is undertaken in Section H.

8.69 The method is only as good as the validity of its underlying assumptions. Consider i = 1 … m commodities, where pmt is the price of a replacement commodity m in period t, pnt+1 is the price of commodity n in period t + 1, and there are overlap prices for both commodities in period t. Now item n replaces m but is of a different quality. So let A(z) be the quality adjustment to pnt+1, which equates its quality to pmt+1 such that the quality-adjusted price pm*t+1=A(zt+1)pnt+1. Put simply, the index for the commodity in question over the period t – 1 to t + 1 is

It1,t+1=(pmt/pmt1)×(pnt+1/pnt)=pnt+1pmt1×pmtpnt.(8.2)

8.70 The quality adjustment to prices in period t + 1 is defined as before, pm*t+1=A(zt+1)pnt+1, which is the adjustment to pn in period t + 1, which equates its value to pm in period t + 1 (had it existed then). A desired measure of price changes between periods t – 1 and t + 1 is thus

(pm*t+1/pmt1).(8.3)

The overlap formulation equals this when

pm*t+1pmt1=A(zt+1)pnt+1pmt1=pnt+1pnt×pmtpmt1

A(zt+1)=pmtpnt and similarly for future periods of the series

A(zt+i)=pmtpntforpm*t+ipmt1fori=2,T.(8.4)

8.71 The assumption is that the quality difference in any period equates to the price difference at the time of the splice. The timing of the switch from m to n is thus crucial. Unfortunately, responding establishments usually continue to report prices on a commodity until it is no longer imported/exported so that the switch may take place at an unusual period of pricing, near the end of item m’s life cycle and the start of item n’s life cycle.

8.72 But what if the assumption does not hold? What if the relative prices in period t, Rt=pmt/pnt, do not equal A(z) in some future period, say A(zt+i) = αi Rt? If αi, = α, the comparisons of prices between future successive periods—between t + 3 and t + 4—are unaffected, as would be expected, because commodity n is effectively being compared with itself.

pm*t+4pmt1/pm*t+3pm*t1=αRtαRtpnt+4pnt+3=pnt+4pnt+3.(8.5)

8.73 However, if differences in the relative prices of the old and replacement commodities vary over time, then

pm*t+4pmt1/pm*t+3pmt1=α4pnt+4α3pnt+1.(8.6)

Note that the quality difference here is not related to the technical specifications or resource costs but to the relative price that purchasers pay.

8.74 Relative prices may also reflect unusual pricing policies aimed at minority segments of the market. In the example of pharmaceutical drugs (Berndt, Ling, and Kyle, 2003), the overlapping prices of a generic and a name brand commodity were argued to be reflective of the needs of two different market segments. The overlap method can be used with a judicious choice of the overlap period. It should be a period before the use of the replacement, because in such periods the pricing may reflect a strategy to dump the old model to make way for the new one.

8.75 The overlap method is implicitly employed when samples of commodities are rotated, meaning that the old sample of commodities is used to compute the category index price change between periods t – 1 and t, and the new sample is used between t and t + 1. The splicing together of these index movements is justified by the assumption that—on a group-to-group rather than commodity-to-item level—differences in price levels at a common point in time accurately reflect differences in qualities.

8.76 The overlap method has at its roots a basis in the law of one price. The law states that when a price difference is observed, it must be the result of some difference in physical quality or some such factor for which consumers are willing to pay a premium, such as the timing of the sale, location, convenience, or conditions. Economic theory would dictate that such price difference would not persist given markets made up of rational producers and consumers. However, empirically, different prices are observed in markets for identical products sold with the same conditions of sale. There are a number of possible reasons for this. First, purchasers may not be well informed about the range of prices on offer and in general will not search out for the lowest prices because costs are incurred in the process. Second, prices do not change smoothly, there being costs associated with making price changes. When they do occur they may be lumpy to catch up on previous changes and not all suppliers may change their prices at the same time. Third, purchasers may not be free to choose the price at which they purchase because the seller may be in a position to charge different prices to different categories of purchasers for identical goods and services sold under exactly the same circumstances—in other words, to practice price discrimination. Finally, buyers may be unable to buy as much as they would like at a lower price because there is insufficient supply available at that price. This situation typically occurs when there are two parallel markets. There may be a primary, or official, market in which the quantities sold, and the prices at which they are sold, are subject to government or official control, while there may be a secondary market—a free market or unofficial market—whose existence may or may not be recognized officially.

8.77 There is extensive literature in economics dealing with theory and evidence of price dispersion and its persistence, even when quality differences have been accounted for. Similar issues arise for goods and services exported and imported with the added complication that volatile exchange rate changes may increase price dispersion, especially if there is a catch-up on the changes not passed through immediately.

D.2 Overall mean/targeted mean imputation

8.78 This method uses the price changes of other commodities as estimates of the price changes of the missing commodities. Consider a Jevons elementary price index, that is, a geometric mean of price relatives (Chapter 21). The prices of the missing items in the current period, say, t + 1, are imputed by multiplying their prices in the immediately preceding period t by the geometric mean of the price relatives of the remaining matched items between these two periods. The comparison is then linked by multiplication to the price changes for previous periods. It is the computationally most straightforward of methods, because the estimate can be undertaken by simply dropping the items that are missing from both periods from the calculation. In practice, the series is continued by including in the database the imputed prices. It is based on the assumption of similar price movements for the omitted commodities and the class of commodities used for the imputation. A targeted form of the method would use similar price movements of a cell or elementary aggregate of similar commodities, or be based on price changes at a higher level of aggregation if either the lower level had an insufficient sample size or price changes at the higher level were judged to be more representative of the price changes of the missing commodity.

8.79 In the example in Table 8.2(b), the January to February comparison for both commodity types is based on commodities 1, 2, 5, 6, and 8. For March compared with January—weights all equal to unity—the commodity 2 and commodity 6 prices are imputed using the short-run price change for February (p2) compared with March (p3) based on commodities 1, 5, and 8. Because different formulas are used for elementary aggregation, the calculation for the three main formulas are illustrated here (see Chapter 21 for choice of formulas). The geometric mean of the price ratios—the Jevons index—is

pJ(p2,p3)=Πi=1N[pi3/pi2]1/N=[(p13/p12)×(p53/p52)×(p83/p82)]1/3=((6/5)×(12/11)×(10/10))1/3= 1.0939,(8.7)

or a 9.39 percent increase. The ratio of average (mean) prices—the Dutot index—is

pD(p2,p3)=Σi=1Npi3/N/Σi=1Npi2/N=[(p13+p53+p83)/3]÷[(p12+p52+p82)/3]=(6+12+10)/(5+11+10)=1.0769,(8.8)

or a 7.69 percent increase. The average (mean) of price ratios—the Carli index—is

pC(p2,p3)=Σi=1N(pi3/pi2)/N=[(p13/p12)+(p53/p52)+(p83/p82)]/3=[(6/5+12/11+10/10)]/3=1.09697,(8.9)

or a 9.697 percent increase.

8.80 In practice the imputed figure would be entered onto the data sheet. Table 8.2(b) has the overall mean imputation in March for commodities 2 and 6, using the Jevons index, as 1.0939 × 6 = 6.563 and 1.0939 × 12 = 13.127, respectively (bold type). It should be noted that the Dutot index is in this instance lower than the Jevons index, a result not expected from the relationships established in Chapter 21. The relationship in Chapter 21 assumed the variance in prices would increase over time, whereas in Table 8.2(b) it decreases for the three commodities. The arithmetic mean of price relatives—the Carli index—equally weights each price change, but the ratio of arithmetic means—the Dutot index—weights price changes according to the prices of the commodity in the base period relative to the sum of the base-period prices. Item 1 has a relatively low price, and thus weight, in the base period 1 of 4, but this commodity has the highest price increase, one of 6/5. Therefore, the Dutot index is lower than the Carli index. However, in Chapter 21 we establish that the Carli index has an upward bias.

8.81 As noted above, it is also possible to refine the imputation method by targeting the imputation: including the weight for the unavailable commodities in groupings likely to experience similar price changes—say, by commodity type. Any stratification system used in the selection of establishments and commodity varieties would facilitate this. For example, in Table 8.2(b) assume that the price change of the missing commodity 2 in March is more likely to follow price changes of commodity 1, and commodity 6 is more likely to experience price changes similar to commodities 5 and 8. For March compared with February, with weights all equal to unity, the geometric mean of price ratios (Jevons) is

pJ(p2,p3)=Πi=1N[pi3/pi2]1/N=[(p13/p12)2×(p53/p52×p83/p82)3/2]1/5=[(6/4)2×(12/11×10/10)3/2]1/5=1.1041.(8.10)

Note the weights used: for commodity type A, the single price represents two prices; for commodity type B, the prices represent three, or 3/2 = 1.5 each.

8.82 The ratio of average (mean) prices—the Dutot index—is

PD(p2,p3)=Σi=1Npi3/N/Σi=1Npi2/N=[(2p13+1.5p53+1.5p83)/5]÷[(2p12+1.5p52+1.5p82/5]=[(2×6+1.5(12+10))]÷[2×5+1.5(11+10)]=1.0843.(8.11)

8.83 The average (mean) of price ratios—the Carli index—is

PC(p2,p3)=Σi=1N(pi3/pi2)/N=[25(p13/p12)+35(p53/p52+p83/p82)/2]=2/5×6/5+3/5[(12/11+10/10)/2]=1.1073.(8.12)

8.84 Alternatively, and more simply, imputed figures could be entered in Table 8.2(b) for commodities 2 and 6 in March using just the price movements of A and B for commodities 2 and 6, respectively, and indices calculated accordingly. Using a Jevons index for commodity 2, the imputed value in March would be 6/5 × 6 = 7.2, and for commodity 6 it would be [(12/11) × (10/10)]1/2 × 12 = 12.533. It is thus apparent that not only does the choice of formula matter, as discussed in Chapter 21, but so too may the targeting of the imputation. In practice, the sample of commodities in a targeted subgroup may be too small. An appropriate stratum is required with a sufficiently large sample size, but there may be a tradeoff between the efficiency gains from the larger sample and the representativity of price changes achieved by that sample. Stratification by commodity group and source/destination country may be preferred to stratification just by commodity group if differences in source/destination country price changes are expected, but the resulting sample size may be too small to allow this to be undertaken. In general, the stratum used for the target should be based on the analyst’s knowledge of the commodity group and an understanding of similarities of price changes between and within strata. It also should be based on the reliability of the available sample to be representative of price changes.

8.85 The underlying assumptions of these methods require some analysis because—as discussed by Triplett (1999 and 2004)—they are often misunderstood. Consider i = 1 …. commodities where, as before, pmt is the price of commodity m in period t, and pnt+1 is the price of a replacement commodity n in period t + 1. Now n replaces m but is of a different quality. As before, let A(z) be the quality adjustment to pnt+1, which equates its quality services or utility to pmt+1 such that the quality-adjusted price pm*t+1=A(z)pnt+1. For the imputation method to work, the average price changes of the i = 1 …. m commodities, including the quality-adjusted price pm*t+1 given on the left-hand side of equation (8.13), must equal the average price change from just using the overall mean of the rest of the i = 1 …. m – 1 commodities on the right-hand side of equation (8.13). The discrepancy or bias from the method is the balancing term Q. It is the implicit adjustment that allows the method to work. The arithmetic formulation is given here, although a similar geometric one can be readily formulated. The equation for one unavailable commodity is given by

1m[pm*t+1pmt+Σi=1m1pit+1pit]=[1(m1)Σi=1m1pit+1pit]+Q,(8.13)
Q=1mpm*t+1pmt1m(m1)Σi=1m1pnt+1pit,(8.14)

and for x unavailable commodities by

Q=1mΣi=mx+1mpm*t+1pmtxm(mx)Σi=1mxpit+1pit.(8.15)

8.86 The relationships are readily visualized if r1 is defined as the arithmetic mean of price changes of commodities that continue to be recorded and r2 is defined as the mean of quality-adjusted unavailable commodities—that is, for the arithmetic case where

r1=[Σi=1mxpit+1/pit]÷(mx)r2=[Σi=mx+1mpi*t+1/pit]÷x,(8.16)

then the bias of the arithmetic mean of ratios from substituting equation (8.16) into equation (8.15) is

Q=xm(r2r1),(8.17)

which equals zero when r1 = r2. The bias depends on the ratio of unavailable values and the difference between the mean of price changes for existing commodities and the mean of quality-adjusted replacement price changes. The bias decreases as either (x/m) or the difference between r1 and r2 decreases. Furthermore, the method relies on a comparison between price changes for existing commodities and quality-adjusted price changes for the replacement/unavailable comparison. This is more likely to be justified than a comparison without the quality adjustment to prices. For example, let us say there were m = 3 commodities, each with a price of 100 in period t. Let the t + 1 prices be 120 for two commodities, but assume the third is unavailable, that is, x = 1, and is replaced by a commodity with a price of 140, of which 20 is the result of quality differences. Then the arithmetic bias as given in equations (8.16) and (8.17) where x = 1 and m = 3 is

1/3[(20+140)/100[(120100+120100/2)]]=0.

8.87 Had the bias depended on the unadjusted price of 140 compared with 100, the imputation would be prone to serious error. In this calculation, the direction of the bias is given by (r2 – r1). and does not depend on whether quality is improving or deteriorating, that is, whether A(z) < 1 or A(z) > 1. If A(z) < 1, a quality improvement, it is still possible that r2 < r1 and for the bias to be negative, a point stressed by Triplett (2004).

8.88 It is noted that the analysis here is framed in terms of a short-run price change framework. This means that the short-run price changes between two consecutive periods are used for the imputation. This is different from the long-run imputation, where a base-period price is compared with prices in subsequent months and where the implicit assumptions are more restrictive.

8.89 Table 8.3 provides an illustration whereby the (mean) price change of commodities that continue to exist, r1, is allowed to vary for values between 1.00 and 1.50: no price change and a 50 percent increase. The (mean) price change of the quality-adjusted new commodities compared with the commodities they are replacing is assumed to not change, that is, r2 = 1.00. The bias is given for ratios of missing values of 0.01, 0.05, 0.10, 0.25, and 0.50, arithmetic means and geometric means. For example, if 50 percent of price quotes are missing and the missing quality-adjusted prices do not change, but the prices of existing commodities increase by 5 percent (r1 = 1.05), then the bias for the geometric mean is represented by the proportional factor 0.9759; that is, instead of 1.05, the index should be 0.9759 × 1.05 = 1.0247. For an arithmetic mean, the bias is –0.025; instead of 1.05, it should be 1.025.

Table 8.3.

Example of the Bias from Implicit Quality Adjustment for r2 = 1.00

article image

8.90 Equation (8.17) shows that the ratio x/m and the difference between r1 and r2 determine the bias. Table 8.3 shows that the bias can be quite substantial when x/m is relatively large. For example, when x/m = 0.25, an inflation rate of 5 percent for existing commodities translates to an index change of 3.73 percent and 3.75 percent for the geometric and arithmetic formulations, respectively, when r2 = 1.00, that is, when quality-adjusted prices of unavailable commodities are constant. Instead of being 1.0373 or 1.0375, ignoring the unavailable commodities would give a result of 1.05. Even with 10 percent missing (x/m = 0.1), an inflation rate of 5 percent for existing commodities translates to 4.45 percent and 4.5 percent for the respective geometric and arithmetic formulations when r2 = 1.00. However, consider a fairly low ratio of x/m, say, 0.05; then even when r2 = 1.00 and r1 = 1.20, Table 8.3 finds 18.9 percent and 19 percent corrected rates of inflation for the respective geometric and arithmetic formulations. In competitive markets, r1 and r2 are unlikely to differ by substantial amounts because r2 is a price comparison between the new commodity and the old commodity after adjusting for quality differences. If r1 and r2 are the same, then there would be no bias from the method even if x/m = 0.9. There may, however, be more sampling error. It should be borne in mind that it is not appropriate to compare bias between the arithmetic and geometric means, at least in the form they take in Table 8.3. The latter would have a lower mean, rendering comparisons of bias meaningless.

8.91 An awareness of the market conditions relating to the commodities is instructive to any understanding of likely differences between r1 and r2. The concern here is when prices vary over the life cycle of the commodities. Thus, at the introduction of a new model, the price change may be quite different from price changes of other existing commodities. Assumptions of similar price changes, even when quality adjusted, might be inappropriate. Consider the example of personal computers: new computers enter the market at prices equal to or lower than prices of previous models but with greater speed and capability. An assumption that r1 = r2 could not be justified.

8.92 Some of this bias relates to the fact that markets are composed of different market segments and producers tailor their output to meet such needs. Indeed, the very training of marketers involves consideration of developing different market segments and ascribing to each segment appropriate pricing, commodity quality, promotion, and place (methods of distribution). This is known as the “4 Ps” of the marketing mix. In addition, marketers are taught to plan the marketing mix over the commodity’s life cycle. Such planning would allow for different inputs of each of these marketing mix variables at different points in the life cycle. This includes price skimming during the period of introduction, whereby higher prices are charged to skim off the surplus from segment(s) of purchasers willing to pay more. The economic theory of price discrimination would also predict such behavior. Thus, the quality-adjusted price change of an old commodity compared with a new replacement commodity may be higher than price changes of other commodities in the commodity group. After the introduction of the new commodity, its prices may fall relative to others in the group. There may be no law of one price change for differentiated commodities within a market. Berndt, Ling, and Kyle (2003) clearly showed how, after patent expiration, the price of brand name prescription pharmaceuticals can increase with the entry of new generic pharmaceuticals at a lower price, particularly as loyal, less-price-sensitive customers maintain their allegiance to the brand name pharmaceuticals.

8.93 There is little in economic or marketing theory to support any expectation of similar (quality-adjusted) price changes for new and replacement commodities and other commodities in the commodity group. Some knowledge of the realities of the particular market under study would be helpful when considering the suitability of this approach. Two things matter in any decision to use the imputation approach. The first is the proportion of replacements, and Table 8.3 provides guidance here. The second is the expected difference between r1and r2, and it is clear from the above discussion that there are markets in which they are unlikely to be similar. This is not to say the method should not be used. It is a simple and expedient approach. Arguably what should not happen is that the method is used as a default process without any prior evaluation of expected price changes and the timing of the switch. Furthermore, attention should be directed to its targeted use, using commodities expected to have similar price changes. However, the selection of such commodities should also be based on the need to include a sufficiently large sample so that the estimate is not subject to undue sampling error.

8.94 Some mention should be made of the way these calculations are undertaken. A pro forma setting for the calculations—say, on a spreadsheet—would have each commodity description and its prices recorded on a (usually) monthly basis. The imputed prices of the missing commodities are inserted into the spreadsheet being highlighted as imputed. The reasons for highlighting such prices are (1) because they should not be used in subsequent imputations as if they were actual prices and (2) the inclusion of imputed values may give the false impression of a larger sample size than actually exists. Care should be taken in any audit of the number of prices used in the compilation of the index to code such observations as imputed. It is stressed that this is an illustration of a short-run imputation, and, as will be discussed in Section H, there is a strong case for using short-run imputations against long-run ones.

D.3 Class mean imputation

8.95 The class mean (or substitution relative) method of implicit quality adjustment to prices as used in the U.S. CPI is discussed in Armknecht, Lane, and Stewart (1997). It arose from concerns similar to those considered in Section D.2, namely that unusual price changes were found in the early introductory period when new models were being introduced, particularly for consumer durables.

8.96 The class mean method was adopted in the U.S. CPI for automobiles in 1989 and was phased in for most other nonfood commodities beginning in 1992. It differed from the imputation method only in the source for the imputed rate of price change for the old commodity in period t + 1. Rather than using the category index change obtained using all the nonmissing commodities in the category, compilers based the imputed rate of price change on constant quality replacement commodities—those commodities that were judged comparable or that were quality adjusted directly. The class mean approach was seen as an improvement on the overall mean imputation approach because the imputed price changes were based on items that had not just had a replacement. Instead, these items’ replacement prices benefited from a quality adjustment, or the new replacement commodity had been judged to be directly comparable. However, it may be the case that sufficiently large samples of comparable substitutes or directly quality-adjusted commodities are unavailable. Or it may be that the quality adjustments and selection of comparable commodities are not deemed sufficiently reliable. In this case, a targeted imputation might be considered. The targeted mean is less ambitious in that it seeks only to capture price changes of similar commodities, irrespective of their point in the life cycle. Yet it is an improvement on the overall mean imputation as long as sufficiently large sample sizes are used.

D.4 Comparable replacement

8.97 In comparable replacement, the respondent makes a judgment that the replacement is of a similar quality to the old commodity and any price changes are untainted by quality changes. For commodity type A in Table 8.2(b), commodity 3 might be judged to be comparable to commodity 2 and its prices in subsequent months used to continue the series. In March, the price of 6.5 would be used as the price in March for commodity 2, whose January to March price change would be 6.5/6 × 100 = 108.33 or 8.33 percent. The method of comparable replacement relies on the efficacy of the respondents and, in turn, on the adequacy of the specifications used as a description of the price basis. Statistical agencies may be rightly wary of sample sizes being worn down by dropping commodities using imputation and also of the resource-intensive explicit estimates outlined below. The use of repriced commodities of a comparable specification has much to commend it. If, however, the quality of commodities is improving, the preceding commodity will be inferior to the current ones. In addition, continually ignoring the small changes in the quality of replacements can lead to an upward bias in the index. The extent of the problem will depend on the proportion of such occurrences, the extent to which comparable commodities are accepted in spite of quality differences, and the weight attached to them. Proposals in Chapter 9 to monitor types of quality-adjustment methods by commodity area will provide a basis for a strategy for applying explicit adjustments where they are most needed.

D.5 Linked to show no price change

8.98 Linking attributes any price change between the replacement commodity in the current period and the old commodity in the preceding period to the change in quality. A replacement commodity 7 is selected, for example, in Table 8.2(b) from commodity type B for the missing March commodity 6. The replacement commodity 7 may be of a very different quality compared with commodity 6, with the price difference being quite large. The change in price is assumed to be due to a change in quality. An estimate is made for p72 by equating it to p37 to show no change—that is, the assumed price of commodity 7 in February is 14 in Table 8.2(b). There is, therefore, assumed to be no price change over the period February to March for commodity 7. The January to March result for commodity 6 is (12/12) × (14/14) = 1.00, or no change. However, for the period March to April, the price of item 7 in March can be compared with the imputed p72 for February and linked to the preceding results. So the January to April comparison is composed of the January to February comparison for commodity 6 and linked to (multiplied by) the February to April comparison for item 7. This linking is analogous to the procedures used for the chained and short-run framework discussed in Sections G.3 and H.3. The method is born out of circumstances where comparable replacement commodities are not available, and there are relatively large price differences between the old and replacement commodities, having significant differences in price base and quality. It is not possible to separate out how much of this difference is due to price changes and how much to quality changes, so the method attributes it all to quality and holds price constant. The method introduces a degree of undue price stability into the index. It may well be the case that the period of replacement is when substantial price changes are taking place, these changes being wrongly assigned to quality changes by this method. Article 5 of the European Commission (EC) Regulation No. 1749/96 requires member states to avoid such automatic linking. Such linking is equivalent to the assumption that the difference in price between two successive models is wholly attributed to a difference in quality (Eurostat, 2001, p. 125). It should not be used.

D.6 Carryforward

8.99 With this method, when a commodity becomes unavailable—say, in period t + 2—the price change calculation uses the old t price, carried forward as if there was no change. Thus, from Table 8.2(a) for commodity type A for the January to March Jevons and Dutot indices (Chapter 21, Section B),

PJ(p1,p3)=[(p13/p11×p22/p21)]1/2,andPD(p1,p3)=[(p13+p22)/(p11+p21)],(8.18)

with p22 filling in for the missing p23. This introduces undue stability into the index, which is aggravated if the old price p22 continues to be used to fill in the unobserved prices in subsequent periods. It introduces an inappropriate amount of stability into the index and may give a misleading impression of the active sample size. The practice of the carryforward method is banned for harmonized CPIs under Article 6 of the EC Regulation No. 1749/96 for Harmonized Indices of Consumer Prices (Eurostat, 2001, p. 126). To use this method, an assumption is made that the price from this commodity type would not change. This method should be used only if it is fairly certain that there would be no price change. It should otherwise not be used.

E. Explicit Methods

8.100 All of the aforementioned methods do not rely on explicit information on the value of the change in quality, A(z). Now methods that rely on obtaining an explicit valuation of the quality difference are discussed.

E.1 Expert judgment

8.101 Comparable replacements can be considered to be a special case of “subjective quality adjustment,” because the determination of commodity equivalence is based on the judgment of the respondent. It is important to mention this because an objection to subjective methods is the inability to provide results that can be independently replicated. Yet in comparable replacement, and for the selection of representative commodities, a subjective element is part of normal procedure. This is not, of course, a case for its proliferation.

8.102 Respondents may be asked to quantify the “production cost” or relative price change that can be attributed to the quality change. Alternatively, the use of experts’ views may be appropriate for highly complex commodities where alternative methods are not feasible. Experts, as noted above, should be directed to the nature of the estimate required as discussed in the conceptual section. More than one expert should be chosen, and, where possible, they should be from different backgrounds. Some indication of the interval in which their estimate should lie is also advisable. The well-used Delphi method may be applicable. In this approach, a panel of experts work separately to avoid any bandwagon effect regarding their estimates. They are asked to provide an estimate of the average and range of likely responses. The median is taken of these estimates, and any estimate that is considered extreme is sent back to the expert concerned. The expert is asked to identify reasons for the difference. It may be that the particular expert has a useful perspective on the problem that the other experts had not considered. If the expert argues a case, the response is fed back to the panel members, who are asked if they wish to change their views. A new median is taken, and there are possible further iterations. It is time consuming and expensive but illustrates the care needed in such matters. However, if the adjustment is needed for a commodity area with a large weighting in a trade price index, and no other techniques are available, it is a possible alternative. In all of this guidelines are required as to the conceptual base for the valuation, as discussed in Section B above and Section E.3 below.

E.2 Quantity adjustment

8.103 This is one of the most straightforward explicit adjustments to undertake and is applicable to commodities for which the replacement is of a different size than the available one. In some situations, there is a readily available quantity metric that can be used to compare the commodities. Examples are the number of units in a package (e.g., paper plates or vitamin pills), the size or weight of a container, or the size of sheets or towels. Quantity adjustment to prices can be accomplished by scaling the price of the old or new commodity by the ratio of quantities. The index production system may do this scaling adjustment automatically by converting all prices in the category to a price per unit of size, weight, or number. Such scaling is most important. For example, it should not be the case that because a respondent reports that a soft drink is now only available in 1-liter containers instead of the previously recorded 0.5-liter ones, its price has doubled.

8.104 There is, however, a second issue. It should be kept in mind that a pure price change is concerned with changes in the revenue received from the sale of the exact same commodities, produced under the exact same circumstances, and sold under the exact same terms. In the pharmaceutical context, for example, prices of bottles of pills of different sizes differ. A bottle of 100 pills, each pill having 50 milligrams of a drug, is not the same as a bottle of 50 pills of 100 milligrams each, even though both bottles contain 5,000 milligrams of the same drug. It may also be reasonable to decide that a bottle of aspirin, for example, containing 500 tablets may not have 10 times the quality of a 50-tablet bottle. If the smaller size is no longer available and there is a change, for example, to a larger size container, and a unit price decrease of 2 percent accompanies this change, then it should not be regarded as a price fall if there is a differential in the cost of producing and margin on selling the larger size of 2 percent or more. If, however, the respondent acknowledged that the change in packaging size for this commodity led to a 1 percent saving in resource costs (and margin), and prices of other such commodities without any quantity changes were also falling by 1 percent, then the pure price change would be a fall of 1 percent. In practice, the respondent may be able to make some rough estimates of the effect on the unit cost of the change in packaging size. However, it may well be that no such information is available, and the general policy is to not automatically interpret unit price changes arising from packaging size changes as pure price changes if contrary information exists.

8.105 Consider another example: a brand name bag of fertilizer of a specific type, previously available in a 0.5 kg. bag priced at 1.5, is replaced with a 0.75 kg. bag at 2.25. The main concern here is with rescaling the quantities as opposed to differential cost or margin adjustments. The method would use the relative quantities of fertilizer in each bag for the adjustment. The prices may have increased by [(2.25/1.5) × 100 = 150] 50 percent, but the quality (size)-adjusted prices have remained constant [(2.25/1.5) × (0.5/0.75) × 100 = 100].

8.106 The approach can be outlined in a more elaborate manner by referring to Figure 8.1. The concern here is with the part of the unbroken line between the price and quantity coordinates (1.5, 0.5) and (2.25, 0.75), both of which have unit prices of 3 (price = 1.5/0.5 and 2.25/0.75). There should be no change in quality-adjusted prices. The delta symbol (Δ) denotes a change. The slope of the line is β, which is ΔPrice/ΔSize = (2.25 – 1.5)/(0.75 – 0.50) = 3, that is, the change in price arising from a unit (kg.) change in size. The quality (size)-adjusted price in period t – 1 of the old m bag is

Figure 8.1.
Figure 8.1.

Quality Adjustment for Different-Sized Items

p^mt1=pmt1+βΔsize=1.5+3(0.750.5)=2.25.(8.19)

The quality-adjusted price change shows no change as before:

pnt/p^mt1=2.25/2.25=1.00.

The approach is outlined in this form so that it can be seen as a special case of the hedonic approach discussed later, where price is related to a number of quality characteristics of which size may be one.

8.107 The method can be seen to be successful on intuitive grounds as long as the unit price of different-sized bags remains constant. If the switch was from a 0.5 kg. bag to a 0.25 kg. one priced at 0.75, as shown by the continuation of the unbroken line in Figure 8.1, to coordinate (0.75, 0.25) quality-adjusted prices would again not change. However, assume the unit (kg.) prices were 5, 3, and 3 for the 0.25, 0.5, and 0.75 kg. bags, respectively, as shown in the example below and in Figure 8.1 by the broken line. Then the measure of quality-adjusted price change would depend on whether the 0.5 kg. bag was replaced by the 0.25 kg. one (a 67 percent increase) or the 0.75 kg. one (no change). This is not satisfactory because the choice of replacement size is arbitrary. The rationale behind the quality-adjustment process is to ask: does the difference in unit price in each case arise from differences in unit costs of producing and margins on selling? If so, adjustments should be made to the unit prices to bring them in line; if not, adjustments should be made to the unit price for that proportion due to changes in costs or margins from economies or diseconomies of package size production. It may be obvious from the nature of the commodity that a commodity packaged in a very small size with a disproportionately high unit price has an unusually high profit margin or will have quite different unit production costs, and an appropriate replacement for a large-sized commodity would not be this very small one.

Example of Quantity Adjustments

article image

E.3 Differences in production and option costs

8.108 A natural approach is to adjust the price of the old commodity by an amount equal to the costs of the additional features. This approach is associated with resource cost valuations discussed in Section B.2. Yet Section B.2 advocated a user value approach, the appropriate valuation being the change in production costs associated with a quality change plus any price-cost margin. This amounts to a comparison of relative prices using

pnt/p^mt1,wherep^mt1=pmt1+x(8.20)

and x is the cost or contribution to revenue of the additional features in period t – 1. The respondent is a natural expert source of such information. Greenlees (2000) provided an example for new trucks and motor vehicles in the United States in 1999. Just before the annual model year introductions, BLS staff visit selected manufacturers to collect cost information. The data are used in the PPI and International Price Comparison programs, as well as in the CPI, and the information-gathering activity is a joint operation of the three programs. Allowable commodity changes for the purpose of quality adjustments include occupant safety enhancements, mechanical and electrical improvements to overall vehicle operation or efficiency, changes that affect length of service or need for repair, and changes affecting comfort or convenience.

8.109 As an example of option cost adjustments, assume the producer prices for a commodity in periods t and t + 2 were 10,000 and 10,500, respectively, but assume the price in period t + 2 is for the item with a new feature or option. Also, let the price of the additional feature in period t + 2 be 300. Then the price change would be 10,200/10,000 = 1.02, or 2 percent. The adjustment may take a multiplicative form (see Section A); the additional options are worth 300/10,500 = 0.028571 of the period t + 2 price. The adjusted price in period t is, therefore, 10,285.71 and the price change 10,500/10,285.71 = 1.020833, or about 2 percent. If in subsequent periods either of these elements changes, then so too must p^nt1 for those comparisons. Option cost is thus a method for use in stable markets with stable technologies. Alternatively, it may be preferable to estimate a one-off adjustment to the preceding base-period price and then compare all subsequent commodities with the new option to this estimate; that is, 10,500/10,300 = 1.019417, or approximately 2 percent.

8.110 In the example above, the prices available for the options were sales prices. For resource cost estimates, the sales prices as estimates of user values must be adjusted to cost estimates by removing markups and indirect taxes. Similarly, and more appropriate to the context of Section B.2, production costs of options need to be upgraded to user values by adding price-cost markups and indirect taxes. Often such data are available for only one period. If the markups are considered to be in the same proportion in subsequent periods, then there is no problem because the retail price changes would proxy the producer ones after adjustment for proportionate margins. However, if the average age or vintage of the commodities has changed, then they will be at different stages in their life cycles and may have different margins.

8.111 Consider the addition of a feature to a commodity. Chairs, for example, can be produced and sold as standard or with a lever mechanism for height adjustment. The specification may always have been the standard model, but this may no longer be in production. The new spec may be a model with height adjustment. The cost of the option is, therefore, known from before, and a continuing series can be developed by using equation (8.20) and simply adding the option cost back into the base-period, old price. Even this process may have its problems. First, the cost of producing something as standard, because all new chairs now have the height adjuster, may be lower than when it was an option. The option cost method would thus understate a price increase. It may be that the manufacturer has an estimate of the effects of such economies of scale to allow for further adjustments. Triplett (2004) cited a study by Levy and others (1999) in which an automobile antitheft system was installed as standard, but disabled when the option was not required. It was seemingly cheaper to produce this way. Second, by including something as standard, the revenue received may be less for some sales than the marginal cost of producing it. The decision to include it as standard precludes buyers from refusing it. It may be that they will turn to other manufacturers who allow them to exclude the option, although it is unlikely that this will be the sole criterion for the purchase. The overall effect would be that the estimate of the option cost, priced for those who choose it, is likely to be higher than the implicit revenue purchasers accord it as standard. Third, the height adjuster may be valued at an additional amount x when sold separately. There is likely to be a segment of the market that particularly values price adjusters and is willing to spend the additional amount. However, when it is sold as standard, many of the purchasers will not value it so highly because these were the very ones who chose the standard chair. The overall user value would be less than x, although it is not immediately apparent how much less. Some statistical offices take one-half x as the adjustment. Some insight into the proportion of the market purchasing the standard commodities would help generate more precise estimates.

8.112 Option cost adjustments are similar to the quantity adjustments, with the exception that the additional quality feature of the replacement is not limited to size. The comparison is pnt/p^nt1,wherep^mt1=pmt1+βΔz for an individual z characteristic where Δz =(zntzmt1). The characteristic may be that the amount of random access memory (RAM) on a personal computer (PC) as a specific model is replaced by one that is identical except for the amount of RAM. If the relationship between price and RAM is linear, this formulation is appropriate. On the web pages of many computer manufacturers, the price of additional RAM is independent of other features, and a linear adjustment is appropriate. Bear in mind that a linear formulation values the worth of a fixed additional amount of RAM as the same, irrespective of the machine’s total amount of RAM.

8.113 The relationship may be nonlinear. For example, for every additional one unit of x, y increases by 1.5 percent (β = 1.015), in this case

p^mt1=pmt1βz(8.21)

for pnt/p^mt1 as a measure of quality-adjusted price changes. Again, the z change may reflect the service flow, but the nonlinearity in the price–z relationship may reflect the increasing or decreasing utility to the scale of the provision. The characteristic may be priced at a higher rate in up-market models of the commodity versus down-market ones, that is, β ≥ 1 in equation (8.21).

8.114 The similarity between the quantity adjustment and the option cost approach can be identified by simply considering Figure 8.1 with the z characteristic being the option horizontal axis. The similarity between the quantity adjustment and the option cost approach is apparent because both relate price to some dimension of quality: the size or the option. The option cost approach can be extended to more than one quality dimension. Both approaches rely on the acquisition of estimates of the change in price resulting from a change in the options or size: the β slope estimates. In the case of the quantity adjustment, this is taken from a commodity identical to the one being replaced except for the size. The β slope estimate in this case would be perfectly identified from the two pieces of data. It is as if changes in the other factors’ quality were accounted for by the nature of the experiment; this is done by comparing prices of what is essentially the same thing except for change in quantity. There may be, for example, two items that are identical except for a single feature. This allows the value of the feature to be determined. Yet sometimes the worth of a feature or option has to be extracted from a much larger data set. This may be because the quality dimension takes a relatively large range of possible numerical values without an immediately obvious consistent valuation. Consider the simple example of one feature varying in a commodity: processor speed in a PC. It is not a straightforward matter to determine the value of an additional unit of speed. To complicate matters, there may be several quality dimensions to the items, and not all combinations of these may exist as items in the market in any one period. Furthermore, the combinations existing in the second period being compared may be quite different from those in the first. All of this leads to a more general framework.

E.4 Hedonic approach

E.4.1 Principles and method

8.115 The hedonic approach is an extension of the two preceding approaches. First, the change in price arising from a unit change in quality—the slope of the line in Figure 8.1—is now estimated from a data set comprising prices and quality characteristic values of a larger number of varieties. Second, the quality characteristic set is extended to cover, in principle, all major characteristics that might influence price, rather than just the quantity or option adjustment. The theoretical basis for hedonic regressions will be covered in Chapter 22 and is briefly reviewed after the following example.

8.116 First, it should be noted that the method requires an extension of the data set to include values for each commodity of price-determining quality characteristics. Under the matched-models method, each respondent needed to supply sufficient data on each item to allow it to be identified for subsequent repricing. The extension required is that all price-determining characteristics should be available for each item. Checklists for the characteristics of a commodity have been found by Merkel (2000) to improve the quality of data collected, as well as to serve the needs of hedonic adjustments (see also Chapter 7 on price collection). If a commodity is missing, any difference in the characteristics of its replacement can be identified, and, as will be shown, a valuation can be ascribed to such differences using the hedonic approach.

8.117 Appendix 8.1 provides data taken from the U.K. Compaq and Dell websites in July 2000 on the prices and characteristics of 64 desktop PCs. Figure 8.2 is a scatter diagram constructed from these data relating the price (£) to the processing speed (MHz). It is apparent that PCs with higher speeds command higher prices—a positive relationship. Under the option cost framework described above, a switch from a 733 MHz PC to a 933 MHz one would involve a measure of the slope of the line between two unique points. The approach requires that there are 733 MHz and 933 MHz PCs that are otherwise identical. From Figure 8.2 and Appendix 8.1, it is apparent that in each instance there are several PCs with the same speed but different prices, owing to differences in other things. To estimate the required value given to additional units of speed, an estimate of the slope of the line that best fits the data is required.

Figure 8.2.
Figure 8.2.

Scatter Diagram of PC Prices

In Figure 8.1, the actual slope was used; for the data in Figure 8.2, an estimate of the slope needs to be derived from an estimate of the equation of the line that best fits the data, using ordinary least squares (OLS) regression. Facilities for regression are available on standard statistical and econometric software, as well as spreadsheets. The estimated (linear) equation in this instance is

P^rice=658.436+3.261SpeedR¯2=0.820.(8.22)

8.118 The coefficient on speed is the estimated slope of the line: the change in price (£3.261) resulting from a 1 MHz change in speed. This can be used to estimate quality-adjusted price changes for PCs of different speeds. The R¯2 finds that 82 percent of price variation is explained by variation in processing speed. A t -statistic to test the null hypothesis of the coefficient being zero was found to be 18.83; recourse to standard tables on t -statistics found the null hypothesis was rejected at a 1 percent level. The fact that the estimated coefficient differs from zero cannot be attributed to sampling errors at this level of significance. There is a probability of 1 percent that the test has wrongly rejected the null hypothesis. However, the range of prices for a given speed—933 MHz, for example—can be seen from Appendix 8.1 to be substantial. There is a price range of about £1,000, which suggests other quality characteristics may be involved. Table 8.4 provides the results of a regression equation that relates price to a number of quality characteristics using the data in Appendix 8.1. Such estimates can be provided by standard statistical and econometric software, as well as by spreadsheets.

Table 8.4.

Hedonic Regression Results for Dell and Compaq PCs

article image

Read-only memory.

Note: Figures in parentheses are t-statistics testing a null hypothesis of the coefficient being zero.***, **, and * denote statistically significant at a 0.1 percent, 1 percent, and 5 percent level, respectively, tests being one-tailed.

8.119 The first column provides the results from a linear regression model, the dependent variable being price. The first variable is processor speed with a coefficient of 2.731; a unit MHz increase in processing speed leads to an estimated £2.731 increase (positive sign) in price. A change from 733 MHz to 933 MHz would be valued at an estimated 200 (2.731) = £546.20. The coefficient is statistically significant, its difference from zero (no effect) not being due to sampling errors at a 0.1 percent level of significance. This estimated coefficient is based on a multivariate model; the coefficient measures the effect of a unit change in processing speed on price having controlled for the effect of other variables in the equation. The result of 3.261 in equation (8.22) was based on just one variable and did not benefit from this. That number is different from this improved result.

8.120 The brand variables are dummy intercepts taking values of 1 if, for example, it is a Dell computer and zero otherwise. Although brands are not in themselves quality characteristics, they may be proxy variables for other factors such as after-service reliability. The inclusion of such brand dummies also reflects segmented markets as communities of buyers as discussed in Chapter 22, Appendix 22.1. Similar dummy variables were formed for other makes and models, including the Compaq Presario and Compaq Prosignia. The Compaq Deskpro, however, was omitted to form the benchmark against which other models are compared. The coefficient on Dell is an estimate of the difference between the worth of a Dell and a Compaq Deskpro, other variables being constant (i.e., £1,330.78 cheaper). Similarly, an Intel Pentium III commands a premium estimated at £282.78 over an AMD Athlon.

8.121 The estimate for processor speed was based on data for Dell and Compaq PCs. If the adjustment for quality is between two Dell PCs, it might be argued that data on Compaq PCs should be ignored. Separate regressions could be estimated for each make, but this would severely restrict the sample size. Alternatively, an interaction term or slope dummy can be used for variables that are believed to have a distinctive brand-interaction effect. Take Dell × Speed, which takes the value of speed when the PC is a Dell and zero otherwise. The coefficient on this variable is 1.714 (see Table 8.4); it is an estimate of the additional (positive sign) price arising for a Dell PC over and above that already arising from the standard valuation of a 1 MHz increase in speed. For Dell PCs, it is 2.731 + 1.714 = £4.445. Therefore, if the replacement Dell PC is 200 MHz faster than the unavailable PC, the price adjustment to the unavailable PC is to add 200 × £4.445 = £889. Interactive terms for other variables can similarly be defined and used. The estimation of regression equations is easily undertaken using econometric or statistical software or data analysis functions in spreadsheets. An understanding of the techniques is given in many texts, including Kennedy (2003). In Chapter 22, Appendix 22.1, econometric concerns particular to the estimation of hedonic regressions are discussed.

8.122 The R¯2 is the proportion of variation in price explained by the estimated equation. More formally, it is 1 minus the ratio of the variance of the residuals Σin=1(pitp^it)2/n, of the equation to the variance of prices Σin=1(pitp¯it)2/n. The bar on the R2 denotes that an appropriate adjustment for degrees of freedom is made to this expression, which is necessary when comparing equations with different numbers of explanatory variables. At 0.934, R¯2 is high. However, high. However, high R¯2 can be misleading for the purpose of quality adjustment. First, such values inform us that the explanatory variables account for much of price variation. This may be ove a relatively large number of varieties of goods in the period concerned. This is not the same as implying a high degree of prediction for an adjustment to a replacement commodity of a single brand in a subsequent time period. For their accuracy, predicted values depend not just on the fit of the equation but also on how far the characteristics of the commodity whose price is to be predicted are from the means of the sample. The more unusual the commodity, the higher the prediction probability interval. Second, R¯2 informs us as to the proportion of variation in prices explained by the estimated equation. It may be that 0.90 is explained, while 0.10 is not. If the dispersion in prices is large, this still leaves a large absolute margin of prices unexplained. Nonetheless, a high R¯2 is a necessary condition for the use of hedonic adjustments.

8.123 Hedonic regressions should generally be conducted using a semi-logarithmic formulation (Chapter 22). The dependent variable is the (natural) logarithm of the price. However, the variables on the right-hand side of the equation are taken in their normal units, thus the semi-logarithmic formulation. A double-logarithmic formulation also takes logarithms of the right-hand side z variables. However, if any of these z variables are dummy variables—taking the value of zero in some instances—the double-logarithmic formulation breaks down. Logarithms of zero cannot be taken (thus the focus on the semi-logarithmic form). This concern with linear and semi-log formulations is equivalent to the consideration of additive and multiplicative formulations discussed in Section A. A linear model would, for example, ascribe an extra £282.78 to a PC with an Intel Pentium III as opposed to an AMD Athlon, irrespective of the price of the PC. This is common in pricing strategies using the World Wide Web. However, more often than not, the same options are valued at a higher price for up-market goods and services. In this case our equation (8.22) above, for a multivariate model, is

Price=β0×β1z1×β2z2×β3z3×.×βnzn×ϵ orln Price=lnβ0+z1lnβ1+z2lnβ2+z3lnβ3+.znlnβn+lnϵ.(8.23)

8.124 Note that this is a semi-logarithmic form; logarithms are taken of only the left-hand-side variable, that is, price. Each of the z characteristics enter the regression without having logarithms taken. This has the advantage of allowing dummy variables for the possession or otherwise of a feature to be included on the right-hand side. Such dummy variables take the value of 1 if the commodity possesses the feature and zero otherwise, it not being possible to take a logarithm of the value zero. Issues on choice of functional form are discussed in more detail in Chapter 22.

8.125 The taking of logarithms in the first equation (8.23) allows it to be transformed in the second equation to a linear form. This allows the use of a conventional OLS estimator to yield estimates of the logarithm of the coefficients. These are given in column 3 of Table 8.4 and have a useful direct interpretation: if these coefficients are multiplied by 100, they are the percentage change in price arising from a one-unit change in the explanatory variable. For processor speed, there is an estimated 0.1364 percent change in price for each additional MHz the replacement commodity has over and above the unavailable one. When dummy variables are used, the coefficients—when multiplied by 100—are estimates of the percentage change in price given by (eβ – 1) 100; for example, for a rewritable compact disc drive (CD-RW) compared with a read-only CD drive (CD-ROM), it is (e0.08916 – 1)100 = 9.326 percent. There is some bias in these estimated coefficients on dummy variables for the (semi-) logarithmic equation; one-half of the variance of the regression equation should be added to the coefficient before using it (Teekens and Koerts, 1972). For CD-ROM, the t -statistic is 2.88; this is equal to the coefficient divided by its standard error. The standard error is 0.08916/2.88 = 0.03096, and the variance is 0.030962 = 0.000958. To adjust to variance of the regression equation, add 0.000958/2 to 0.08916 = 0.089639 or 8.9639 percent.

8.126 The approach is particularly useful when the market does not reveal the price of the quality characteristics required for the adjustment. Markets reveal prices of commodities, not quality characteristics, so it is useful to consider commodities as tied bundles of characteristics. A sufficiently large data set of commodities with their characteristics, and sufficient variability in the mix of characteristics between the commodities, allows the hedonic regression to provide estimates of the implicit prices of the characteristics. The formal theory is provided in Chapter 22. There are a number of ways of implementing the method, which are outlined below. Before doing so, it is useful to note how these coefficients should be interpreted in light of theoretical needs.

E.4.2 On theory

8.127 Some mention should be made of the interpretation of the coefficients from hedonic regressions. The matter will be discussed in further detail in Chapter 22, Section B.5. This section summarizes the conclusion. There used to be an erroneous perception that the coefficients from hedonic methods represented estimates of user value as opposed to resource cost. Rosen (1974) showed that hedonic coefficients may be reflective of both user value and resource cost, both supply and demand influences. There is, in econometric terms, an identification problem, in which the observed data do not permit the estimation of the underlying demand and supply parameters. However, suppose the production technology of sellers is the same but buyers differ. Then the hedonic function describes the prices of characteristics the firm will supply with the given ruling technology to the current mixture of tastes. There are different tastes on the user side, so what appears in the market is the result of firms trying to satisfy purchasers’ preferences all for a constant technology and profit level; the structure of supply is revealed by the hedonic price function. Now suppose sellers differ but buyers’ tastes are the same. Here the hedonic function p(z) identifies the structure of demand. Of these possibilities, uniformity of tastes is unlikely while uniformity of technologies is more likely, especially when access to technology is unrestricted in the long run. Griliches (1989, p. 120) has argued in the context of a CPI,

My own view is that what the hedonic approach tries to do is to estimate aspects of the budget constraint facing consumers, allowing thereby the estimation of “missing” prices when quality changes. It is not in the business of estimating utility functions per se, though it can also be useful for these purposes …. what is being estimated is the actual locus of intersection of the demand curves of different consumers with varying tastes and the supply curves of different producers with possible varying technologies of production. One is unlikely, therefore, to be able to recover the underlying utility and cost functions from such data alone, except in very special circumstances.

8.128 It is thus necessary to take a pragmatic stance. In many cases, the implicit quality adjustment to prices outlined in Section C may be inappropriate because their implicit assumptions are unlikely to be valid. The practical needs of economic statistics require in such instances explicit quality adjustments. To not do anything on the grounds that the measures are not conceptually appropriate would be to ignore the quality change and provide wrong results. Hedonic techniques provide an important tool, making effective use of data on the price-quality relationship derived from other commodities in the market to adjustment for changes in one or more characteristics.

8.129 The proper use of hedonic regression requires an examination of the coefficients of the estimated equations to see if they make sense. It might be argued that the very multitude of distributions of tastes and technologies and interplay of supply and demand make it unlikely that reasonable estimates will arise from such regressions. A firm may apply and cut a profit margin and prices for reasons related to long-run strategic plans, for example, yielding coefficients that prima facie do not look reasonable. This does not negate the usefulness of examining hedonic coefficients as part of a strategy for evaluating estimated hedonic equations. First, there has been extensive empirical work in this field, and the results for individual coefficients are, for the most part, quite reasonable. Even over time, the empirical evidence is that there are quite sensible patterns of decline in individual coefficients. Second, as shall be seen, it might be argued that the prediction and its error should be our concern and not the values of individual coefficients.

E.4.3 Implementation

8.130 The implementation of hedonic methods to estimate quality adjustments to noncomparable replacements can take a number of forms. The first form is when the repricing is for a commodity with different characteristics. What is required is to adjust either the price of the old or replacement (new) commodity for some valuation of the difference in quality between the two commodities. This patching of missing prices is quite different from the use of hedonic price indices to be discussed in Section G.2 in this chapter and in Chapter 22. These use hedonic regressions to provide hedonic price indices of overall quality-adjusted prices. The former is a partial application, used on noncomparable replacements when commodities are no longer produced. The latter, as will be seen in Section G.2, is a general application to a sample from the whole data set. The partial patching is considered here.

8.131 Hedonic imputation: predicted vs. actual— In this approach, a hedonic regression of the (natural logarithm of the) price of model i in period t on its characteristics set zkit is estimated for each month, as given by

lnpit=β0t+Σk=1kβktzkit+ϵkt.(8.24)

8.132 Let us say the price of a commodity m available in January (period t) is unavailable in March (period t + 2). The price of commodity m can be predicted for March by inserting the characteristics of the old unavailable commodity m into the estimated regression equation for March; this process is repeated for successive months. The predicted price for the old commodity in March and the price comparison with January (period t) are given, respectively, by

p^mt+2=exp[β^0t+2+Σβ^kt+2zk,mt],(8.25a)

and p^mt+2/pmt, that is, the old model’s price is adjusted. In the example in Table 8.2(a), p^23,p^24, and so forth, and p^63,p^64, and so forth would be estimated and compared with p21 and p61, respectively. The blanks for commodities 2 and 6 in Table 8.2(a) would be effectively filled in by the estimated prices from the regression equation.

8.133 An alternative procedure is to select for each unavailable m commodity a replacement commodity n. In this case, the price of n in period t + 2 is known, and a predicted price for n in period t is required. The predicted price for the new commodity n and required

p^nt=exp[β^0t+Σβ^ktzk,nt+2],(8.25b)

and pnt+2/p^nt, that is, the new model’s price is adjusted. In this case, the characteristics of commodity n are inserted into the right-hand side of an estimated regression for period t. The price comparisons of equation (8.25a) may be weighted by wmt, as would those of its replaced price comparison in equation (8.25b).

8.134 A final alternative is to take the geometric mean of the formulations in equations (8.25a) and (8.25b) on grounds analogous to those discussed in Chapter 16 and by Diewert (1997) for similar index number issues.

8.135 Hedonic imputation: predicted vs. predicted— A further approach is the use of predicted values for the commodity in both periods, for example, p^nt+2/p^nt, where n represents the commodity. Consider a misspecification problem in the hedonic equation. For example, there may be an interaction effect between a brand dummy and a characteristic, say, between Dell and speed in the example in Table 8.4. Having both characteristics may be worth more on price (from a semi-logarithmic form) than their separate individual components. The use of pnt+2/p^nt would be misleading because the actual price in the numerator would incorporate the 5 percent premium while the one predicted from a straightforward semi-logarithmic form would not. It is stressed that in adopting this approach, a recorded, actual price is being replaced by an imputation. Neither this nor the form of bias discussed above is desirable.

8.136 The comparisons using predicted values in both periods are given as

p^nt+2/p^nt for the new commodity,

p^mt+2/p^mt for the disappearing or old commodity, or

[(p^nt+2/p^nt)(p^mt+2/p^mt)]1/2(8.26)

as a (geometric) mean of the two.

8.137 Hedonic adjustments using coefficients— In this approach, a replacement commodity is used and any differences between the characteristics of the replacement n in period t + 2 and m in period t are ascertained. A predicted price for n in period t, that is, p^nt, is compared with the actual price pnt+2. However, unlike the formulation in equation (8.25b) for example, p^nt may be estimated by applying the subset of the k characteristics that distinguished m from n, to their respective implicit prices in period t estimated from the hedonic regression, and adjusting the price of pmt. For example, if the nearest replacement for commodity 2 was commodity 3, then the characteristics that differentiated commodity 3 from commodity 2 are identified and the price in the base period p31 is estimated by adjusting p21 using the appropriate coefficients from the hedonic regression in that month. For example, for washing machines, if commodity 2 had a spin speed of 800 revolutions per minute (r pm) and commodity 3 had a spin speed of 1,100 rpm, other things being equal, the shadow price of the 300 rpm differential would be estimated from the hedonic regression, and p21 would be adjusted for comparison with p33. Note that if the z variables in the characteristic set are perfectly independent of each other, the results from this approach will be similar to those from equation (8.25b). This is because interdependence among the variables on the right-hand side of the hedonic equation—multicollinearity—leads to imprecise estimates of the coefficients (see Chapter 22, Appendix 22.1).

8.138 Hedonic indirect adjustment—An indirect current-period hedonic adjustment may be used, which only requires the hedonic regression to be estimated in the base period t:

pnt+2pmt÷p^ntp^mt.(8.27)

8.139 The first term is the change in price between the old and replacement items in periods t and t + 2, respectively. But the quality of the commodity has changed, so this price change needs to be divided by a measure of the change in quality. The second term uses the hedonic regression in period t in both the numerator and denominator. So the coefficients—the shadow prices of each characteristic—remain the same. It is not prices that change. The predicted prices differ because different quantities of the characteristics are being inserted into the numerator and denominator; the replacement n characteristics in the former and old commodity m characteristics in the latter. The measure is the change in price after removing (by division) the change in the quantity of characteristics each valued at a constant period t price. Conceptually, the constant valuation by a period t + 2 regression would be equally valid and a geometric mean of the two ideal. However, if hedonic regressions cannot be run in real time, equation (8.27) is a compromise. As the spread between the current- and base-period results increases, its validity decreases. As such, the regression estimates should be updated regularly using old- and current-period estimates, and results compared retrospectively as a check on the validity of the results.

E.4.4 Need for caution

8.140 The limitations of the hedonic approach should be kept in mind. Some points are summarized below though readers are referred to the Bibliography and to Chapter 22, Appendix 22.1. First, the approach requires statistical expertise for the estimation of the equations. The prevalence of user-friendly software with regression capabilities makes this less problematic. Statistical and econometric software carry a range of diagnostic tests to help judge if the final formulation of the model is satisfactory. These include R¯2 as a measure of the overall explanatory power of the equation and F-test and t -test statistics to enable tests to be conducted to determine whether the differences between the coefficients on the explanatory variables are jointly and individually different from zero at specified levels of statistical significance. Most of these statistics make use of the errors from the estimated equation. The regression equation can be used to predict prices for each commodity by inserting the values of the characteristics of the commodities into the explanatory variables. The differences between the actual prices and these predicted results are the residual errors. Biased or imprecise results may arise from a range of factors, including heteroscedasticity (nonconstant variances in the residuals suggesting nonlinearities or omission of relevant explanatory variables), a nonnormal distribution for the errors, and multicollinearity, where two or more explanatory variables are related. The latter, in particular, has been described as the “bane of hedonic regressions . . .” (Triplett, 1990). Such econometric issues are well discussed in the context of hedonic regressions (Triplett, 1990; Gordon, 1990; Berndt, 1991; Berndt, Griliches, and Rappaport, 1995; Silver, 1999; Triplett, 2004; and Appendix 22.1) and more generally in introductory econometric texts such as Kennedy (2003). The use of predicted values when multicollinearity is suspected is advised, rather than individual coefficients, for reasons discussed above.

8.141 Second, the estimated coefficients should be updated regularly. However, if the adjustment is to the old model, then the price comparison is between the price of the new model and the quality-adjusted price of the old model. The quality difference between the old and new models is derived using coefficients from a hedonic regression from a previous period as estimates of the value of such differences. There is, at first glance, no need to update the hedonic regression each month. The valuation of a characteristic in the price reference period may, however, be quite out of line with its valuation in the new period. For example, a feature may be worth an additional 5 percent in the reference period instead of 10 percent in the current period because it might have been introduced at a discount at that point in its life cycle to encourage usage. Continuing to use the coefficients from some far-off period to make price adjustments in the current period is similar to using out-of-date base-period weights. The comparison may be well defined but have little meaning. If price adjustments for quality differences are being made to the old item in the price-reference period using hedonic estimates from that period, then there is a need to update the estimates if they are considered out of date—for example, due to changing tastes or technology—and splice the new estimated comparisons onto the old. Therefore, regular updating of hedonic estimates when using the adjustments to the old price is recommended, especially when there is evidence of parameter instability over time.

8.142 Third, the sample of prices and characteristics used for the hedonic adjustments should be suitable for the purpose. If they are taken from a particular industry, trade source, or web page and then used to adjust noncomparable prices for commodities sold by quite different industries, there must at least be an intuition that the marginal returns for characteristics are similar among the industries. A similar principle applies for the brands of commodities used in the sample for the hedonic regression. It should be kept in mind that high R¯2 statistics alone do not ensure reliable results. Such high values arise from regressions in periods before their application and inform us of the proportion of variation in prices across many commodities and brands. They are not in themselves a measure of the prediction error for a particular commodity, sold by a specific establishment of a given brand in a subsequent period, although they can be an important constituent of this.

8.143 Fourth, there is the issue of functional form and the choice of variables to include in the model. Simple functional forms generally work well. These include linear, semi-logarithmic (logarithm of the left-hand side), and double-log (logarithms of both sides) forms. Such issues are discussed in Chapter 22, Appendix 22.1. The specification of a model should include all price-determining characteristics, and simpler specifications should be used only when they are adequate parsimonious representations of the more general form. Typically, a study would start with a larger number of explanatory variables and a general econometric model of the relationship; the final model is a more specific, parsimonious one because it has dropped a number of variables. The dropping of variables occurs after experimenting with different formulations and seeing their effects on diagnostic test statistics, including the overall fit of the model and the accordance of signs and magnitudes of coefficients with prior expectations.

8.144 Finally, the successful design and use of hedonic quality adjustment requires heavy investments over a long period. Required are (1) intellectual competencies and sufficient time to develop and reestimate the model and employ it when commodities are replaced; (2) access to detailed, reliable information on commodity characteristics; and (3) a suitable organization of the infrastructure for collecting, checking, and processing information.

8.145 It should be noted that hedonic methods may also improve quality adjustment in CPIs by indicating which commodity attributes do not appear to have material impacts on price. That is, if a replacement commodity differs from the old commodity only in characteristics that have been rejected as price-determining variables in a hedonic study, this would support a decision to treat the commodities as comparable or equivalent and include the entire price difference (if any) as pure price change. Care has to be exercised in such analysis because a feature of multicollinearity in regression estimates is that the imprecision of the parameter estimates may give rise to statistical tests that do not reject null hypotheses that are false, that is, they do not find significant parameter estimates. However, the results from such regressions can nonetheless provide valuable information on the extent to which different characteristics influence price variation. This in turn can help in the selection of replacement commodities. The enhanced confidence in commodity substitution and the quality adjustment of prices from the hedonic approach with its parallel reduction in reliance on linking have been cited as significant benefits in the reliability of the measurement of price changes. The results from hedonic regressions have a role to play in identifying price-determining characteristics and may be useful in the design of quality checklists in price collection (Chapter 7).

F. Choosing a Quality-Adjustment Method

8.146 Choosing a method for quality-adjusting prices is not straightforward. The analyst must consider the technology and market for each commodity and devise appropriate methods. This is not to say the methods selected for one industry will be independent of those selected for other industries. Expertise built up using one method may encourage its use elsewhere, and intensive use of resources for one commodity may lead to less resource-intensive methods in others. The methods adopted for individual industries may vary among countries as access to data, relationships with the respondents, resources, expertise and features of the production, and market for the commodity vary. Guidelines on choosing a method arise directly from the features of the methods outlined above. A good understanding of the methods and their implicit and explicit assumptions is essential when choosing a method.

8.147 Consider Figure 8.3, which provides a useful guide to the decision-making process. Assume the matched-models method is being used. If the commodity is matched for repricing—without a change in the specification—no quality adjustment is required. This is the simplest of procedures. However, a caveat applies. If the commodity belongs to a high-technology industry where model replacement is rapid, the matched sample may become unrepresentative of the universe of transactions. Alternatively, matching may be under a chained framework, where prices of commodities in a period are matched to those in the preceding period to form a link. A series of successive links of matched comparisons combined by successive multiplication makes up the chained matched index. Alternatively, hedonic indices may be used, which require no matching. The use of such methods is discussed in Section G. At the very least, attention should be directed to more regular commodity resampling. Continued long-run matching would deplete the sample, and an alternative framework to long-run matching would be required.

8.148 Consider a change in the quality of a commodity, and assume a replacement commodity is available. The selection of a comparable commodity to the same specification and the use of its price as a comparable replacement require that none of the price difference is due to quality. They also require confidence that all price-determining factors are included on the specification. The replacement commodity should also be representative and account for a reasonable proportion of sales. Caution is required when nearly obsolete commodities at the end of their life cycles are replaced with unusual pricing by similar commodities that account for relatively low sales, or with commodities that have substantial sales but are at different points in their cycle. Strategies for ameliorating such effects are discussed below and in Chapter 9, including early substitutions before pricing strategies become dissimilar.

8.149 Figure 8.3 shows where quality differences can be quantified. Explicit estimates are generally considered to be more reliable, but they are also more resource intensive (at least initially). Once an appropriate methodology has been developed, explicit estimates can often be easily replicated. General guidelines are more difficult here because the choice depends on the host of factors discussed above, which are likely to make the estimates more reliable in each situation. Central to all of this is the quality of the data on which the estimates are based. If reliable data are unavailable, subjective judgments may be used. Commodity differences are often quite technical and very difficult to specify and quantify. The reliability of the method depends on the knowledge of the experts and the variance in opinions. Estimates based on objective data are, as a result, preferred. Good production cost estimates, along with good data on markups and indirect taxes, where applicable, in industries with stable technologies where differences between the old and replacement commodities are well specified and exhaustive, are reliable by definition. The option cost approach is generally preferable when old and new commodities differ by easily identifiable characteristics that have once been separately priced as options, but the price of an option will overstate its value when it becomes standard so care must be taken when using this method. The use of hedonic regressions for partial patching is most appropriate where data on price and characteristics are available for a range of models and where the characteristics are found to predict and explain price variability well in terms of a priori reasoning and econometrics. Use of hedonic regressions is appropriate where the price of an option or change in characteristics cannot be separately identified and has to be gleaned from the prices of commodities sold with different specifications in the market. The estimated regression coefficients are the estimate of the contribution to price of a unit change in a characteristic, having controlled for the effects of variations in the quantities of other characteristics.

8.150 The estimates are particularly useful for valuing changes in the quality of a commodity when only a given set of characteristics change, and the valuation is required for changes in these characteristics only. The results from hedonic regressions may be used to target the salient characteristics for commodity selection. The synergy between the selection of prices according to characteristics defined as price determining by the hedonic regression and the subsequent use of hedonics for quality adjustment should reap rewards. The method should be applied where there are high ratios of noncomparable replacements and where the differences between the old and new commodities can be well defined by a large number of characteristics.

8.151 If explicit estimates of quality are unavailable and no replacement commodities are deemed appropriate, then imputations may be used. The use of imputations has much to commend it in terms of resources. It is relatively easy to employ, although some verification of the validity of the implicit assumptions might be appropriate. It requires no judgment (unless targeted) and is therefore objective. Targeted mean imputation is preferred to overall mean imputation as long as the sample size on which the target is based is adequate. Class mean imputation is preferred when models at the start of their life cycles are replacing those near the end of their life cycles, although the approach requires faith in the adequacy of the explicit and comparable replacements being made.

Figure 8.3.
Figure 8.3.

Flow Chart for Making Decisions on Quality Change

Source: Chart is extended version of work of Fenella Maitland-Smith and Rachel Bevan given in Triplett (2004).

8.152 Bias from using imputation is directly related to the proportion of missing commodities and the difference between quality-adjusted prices of available matched commodities and the quality-adjusted prices of unavailable ones (see Table 8.3). The nature and extent of the bias depends on whether short-run or long-run imputations are being used (the former being preferred) and on market conditions (see Section H). Imputation in practical terms produces the same result as deletion of the commodity, and the inclusion of imputed prices may give the illusion of larger sample sizes. Imputation is less likely to give bias for commodities when the proportion of missing prices is low. Table 8.2 can be used to estimate likely error margins arising from its use, and a judgment can be made as to whether the error margins are acceptable. Its use across many industries need not compound the errors because, as noted in the discussion of this method, the direction of bias need not be systematic. It is cost-effective for industries with large numbers of missing commodities because of its ease of use. But the underlying assumptions required must be carefully considered if widely used. Imputation should by no means be the overall, catchall strategy, and statistical agencies are advised against its use as a default device without due consideration of the nature of the markets, possibility of targeting the imputation, and the viability of estimates from the sample sizes involved if such targeting is employed.

8.153 If the old and replacement commodities are available simultaneously and the quality difference cannot be quantified, an implicit approach can be used whereby the price difference between the old and replacement commodity in a period in which they both exist is assumed to be due to quality. This overlap method, by replacing the old commodity with a new one, takes the ratio of prices in a period to be a measure of their quality difference. It is implicitly used when new samples of commodities are taken. The assumption of relative prices equating to quality differences at the time of the splice is unlikely to hold true if the old and replacement commodities are at different stages in their life cycles and different pricing strategies are used at these stages. For example, there may be deep discounting of the old commodity to clear inventories and price skimming of market segments that will purchase new models at relatively high prices. As with comparable replacements, early substitutions are advised so that the overlap is at a time when commodities are at similar stages in their life cycles.

8.154 The use of the linked to show no change method and the carryforward method is not generally advised for making quality-adjustment imputations for the reasons discussed unless there is deemed to be some validity to the implicit assumptions.

G. High-Technology and Other Sectors with Rapid Turnover of Models

8.155 The measurement of price changes of commodities unaffected by quality changes is primarily achieved by matching models, the aforementioned techniques being applicable when the matching breaks down. But what about industries where the matching breaks down on a regular basis because of the high turnover in new models of different qualities than the old ones? The matching of prices of identical models over time, by its nature, is likely to lead to a depleted sample. There is both a dynamic universe of all commodities produced and a static universe of the commodities selected for repricing. For example, if the sample is initiated in December, by the subsequent May the static universe will be matching prices of those commodities available in the static universe in both December and May but will omit the unmatched new commodities introduced in January, February, March, April, and May, and the unmatched old ones available in December but unavailable in May. There are two empirical questions to answer for any significant bias to be detected. The first question asks whether the sample depletion is substantial; such depletion is a necessary condition for bias. The second asks whether the unmatched new and unmatched old commodities are likely to have different quality-adjusted prices versus the matched ones in the current and base period.

8.156 Thus, the matching of prices of identical models over time may lead to the monitoring of a sample of models increasingly unrepresentative of the population of transactions. There are old models that existed when the sample was drawn but are not available in the current period, and there are new ones coming into the current period that are not available in the base period. It may be that the departures have relatively low prices and the entrants relatively high ones and that by ignoring these prices a bias is being introduced. Using old low-priced commodities and ignoring new high-priced ones has the effect of biasing the index downward. In some industries, the new commodity may be introduced at a relatively low price and the old one may become obsolete at a relatively high one, serving a minority segment of the market. In this case, the bias would take the opposite direction; the nature of the bias depends on the pricing strategies of firms for new and old commodities.

8.157 This sampling bias exists for most commodities. However, our concern is with commodity markets where the statistical agencies are finding the frequency of new commodity introductions and old commodity obsolescence sufficiently high that they may have little confidence in their results. First, some examples of such commodity markets will be given. Then, two procedures will be considered: the use of hedonic price indices instead of partial hedonic patching and chaining.

G.1 Some examples

8.158 Koskimäki and Vartia (2001) attempted to match prices of personal computers over three two-month periods (spring, summer, and fall) using a sample of prices collected as part of the standard price collection for the Finnish CPI, which has some similarities to trade price indexes. Of the 83 spring prices, only 55 matched comparisons could be made with the summer prices, and of those, only 16 continued through to the fall. They noted that the sample of matched pairs became increasingly biased: of the 79 models in the fall, the 16 matched ones had a mean processor speed of 518 MHz compared with 628 MHz for the remaining 63 unmatched ones; the respective hard disk sizes were 10.2 gigabytes (GB) and 15.0 GB; and the percentages of high-end processors (Pentium III and AMD Athlon) were 25 percent and 49.2 percent, respectively. Hardly any change in matched prices was found over this six-month period, whereas a hedonic regression analysis using all of the data found quality-adjusted price falls of about 10 percent. Instructions to respondents to hold on to models until forced replacements are required may lead to a sample increasingly unrepresentative of the population and be biased toward technically inferior variants. In this instance, the hedonic price changes fell faster because the newer models became cheaper for the services supplied.

8.159 Silver and Heravi (2005) found evidence of sample degradation when matching prices of U.K. washing machines over a year. By December, only 53 percent of the January basket of model varieties was used for the December/January index, although this accounted for 81.6 percent of January expenditure. Models of washing machines with lower sales values dropped out quicker. However, the remaining models in December accounted for only 48.2 percent of the value of transactions in December. The active sample relating to the universe of transactions in December had substantially deteriorated. The prices of unmatched and matched models differed, as did their vintage and quality. Even when prices were adjusted for quality using hedonic regressions, prices of unmatched old models were found to be lower than matched ones; there was also evidence of higher prices for unmatched new models. Quality-adjusted prices fell faster for the matched sample than the full sample: about 10 percent for the former compared with about 7 percent for the latter. Residuals from a common hedonic surface and their leverage were also examined. The residuals from unmatched new models were higher than matched ones, while residuals from unmatched old models were much lower. Unmatched observations had nearly twice the (unweighted) leverage than matched ones; their influence in the estimation of the parameters of the regression equation was much greater and their exclusion more serious.

8.160 These studies, although based on consumer price data, demonstrate how serious sample degradation can occur and how unmatched excluded commodities may be quite different from included ones. Two procedures for dealing with such situations will be considered: the use of hedonic price indices instead of the partial hedonic patching discussed above and chaining. Both rely on a data set of a representative sample of commodities and their characteristics in each period. A checklist of structured commodity characteristics to be completed each reporting period is one way changes in quality characteristics can be prompted and monitored: this is especially useful in high-technology industries. If a new commodity is introduced and has or is likely to have substantial sales, then it is included as a replacement or even an addition. Its characteristics are marked off against a checklist of salient characteristics. The list will be developed when the sample is initiated and updated as required. Alternatively, web pages and trade associations may be able to provide lists of models and their prices; however, the need for transaction prices as opposed to list prices is stressed.

G.2 Hedonic price indices

8.161 It is important to distinguish between the use of hedonic regressions to make adjustments for quality differences when a noncomparable substitute is used, as in Section E, and their use in their own right as hedonic price indices, which are measures of quality-adjusted price changes. Hedonic price indices are suitable when the pace and scale of replacements of commodities are substantial. There are two reasons for this. First, an extensive use of quality adjustments may lead to errors. Second, the sampling will be from a matched or replacement universe likely to be biased. With new models being continually introduced and old ones dying, the coverage of a matched sample may deteriorate and bias may be introduced as the price changes of the new or old models differ from those of the matched ones. A sample must be drawn in each month, and price indices must be constructed, but, instead of being controlled for quality differences by matching, they will be controlled for, or partialed out, in the hedonic regression. Note that all the indices described below use a fresh sample of the data available in each period. If there is a new commodity in a period, it is included in the data set and its quality differences controlled for by the regression. Similarly, if old commodities drop out, they are still included in the data for the indices in the periods in which they exist. In Section E.4.4 of this chapter, the need for caution was stressed in the use of hedonic regressions for quality adjustments owing to theoretical and econometric issues, some of which will be considered in the appendix to Chapter 22. This need for caution extends to the use of the results from hedonic indices and is not repeated here for the sake of brevity.

8.162 In Chapter 18, theoretical price indices will be defined and practical index number formulas considered as bounds or estimates of these indices. Theoretical index numbers will also be defined in Chapter 22 to include goods made up of tied characteristics, so that something can be said about how such theoretical indices relate to different forms of hedonic indices. A number of forms will be considered in Chapter 22, and the account is outlined here.

G.2.1 Hedonic functions with dummy variables on time

8.163 The sample covers the two time periods being compared—for example, t and t + 2—and does not have to be matched. The hedonic formulation regresses the price of commodity i, pi, on the k = 2. . . . K characteristics of the commodities zki. A single regression is estimated on the data in the two time periods compared, the equation also including a dummy variable Dt+2 being 1 in period t + 2, zero otherwise:

lnPi=β0+β1Dt+2+Σk=2Kβkzki+ϵi.(8.28)

8.164 The coefficient β1 is an estimate of the quality-adjusted price change between period t and period t + 2. It is an estimate of the change in (the logarithm of) price, having controlled for the effects of variation in quality via Σk=2Kβkzki.. Note that an adjustment is required for β1: the addition of one-half (standard error)2 of the estimate as discussed in Teekens and Koerts (1972). Two variants of equation (8.28) are considered. The first, the direct fixed-base version, compares period t with t + 2 as outlined: January–February, January–March, and so forth. The second is a rolling chained version evaluated for period t with t + 1; then again for t + 1 with t + 2 and so on, the links in the chain being combined by successive multiplication. A January–March comparison, for example, would be the January–February index multiplied by the February–March one. There is also a fully constrained version. This entails a single constrained regression for a period of time—January to December, for example—with dummy variables for each month. However, this is impractical in real time because it requires data on future observations.

8.165 The approach just described uses the estimated coefficient on the dummy variables on time to compare prices in period t with prices in each subsequent period. In doing so, the β parameters on the quality variables are constrained to be constant over the period being compared. A fixed-base, bilateral comparison using equation (8.28) makes use of the constrained parameter estimates over the two periods compared and, given an equal number of observations in each period, is a form of a symmetric average. A chained formulation would estimate an index between periods 1 and 4—represented here as I1,4—as I1,4 = I1,2 × I 2,3 × I3,4

8.166 There is no explicit weighting in these formulations; this is a serious disadvantage. In practice. cutoff sampling might be employed to include only the most important commodities. If sales data are available, a weighted least squares (WLS) estimator should be used, as opposed to an OLS estimator. It is axiomatic in normal index number construction that the same weight should not be given to each price comparison because some commodities may account for much larger sales revenues than others. The same consideration applies to these hedonic indices. Diewert (2005b) has argued that sales values should form the basis of the weights over quantities. Two commodities may have sales equal to the same quantity, but, if one is priced higher than another, its price changes should be weighted higher accordingly for the result to be meaningful in an economic sense. In addition, Diewert (2005b) has shown that it is value shares that should form the weights, because values will increase—over period t + 2, for example—with prices, the residuals, and their variance thus being higher in period t + 2 than in t. This heteroscedasticity is an undesirable feature of a regression model resulting in increased standard errors. Silver (2002) has further shown that a WLS estimator does not purely weight the observations by their designated weights. The actual influence given is also due to a combination of the residuals and the leverage effect. The latter is higher because the characteristics of the observations diverge from the average characteristics of the data. He suggests that observations with relatively high leverage and low weights be deleted and the regression repeated.

G.2.2 Period-on-period hedonic indices

8.167 An alternative approach for a comparison between periods t and t + 2 is to estimate a hedonic regression for period t + 2 and insert the values of the characteristics of each model existing in period t into the period t + 2 regression to predict, for each item, its price. This would generate predictions of the prices of items existing in period t based on their zit characteristics, at period t + 2 shadow pricesp^it+2(zit) These prices (or an average) can be compared with the actual prices (or the average of prices) of models in period t, pit(zit) as a, for example, Jevons hedonic base-period index:

PJHB=[Πi=1Ntp^it+2(zit)]1/Nt[Πi=1Ntpit(zit)]1/Nt=[Πi=1Ntp^it+2(zit)]1/Nt[Πi=1Ntpit]1/Nt[Πi=1Ntp^it+2(zit)]1/Nt[Πi=1Ntp^it]1/Nt.(8.29a)

8.168 Alternatively, the characteristics of models existing in period t + 2 can be inserted into a regression for period t. Predicted prices of period t + 2 items generated at period t shadow prices, pit(zit+2), are the prices of items existing in period t + 2 estimated at period t prices, and these prices (or an average) can be compared with the actual prices (or the average of prices) in period t + 2, pit+2(zit+2); a Jevons hedonic current-period index is

PJHC=[Πi=1Nt+2pit+2(zit+2)]1/Nt+2[Πi=1Nt+2p^it(zit+2)]1/Nt+2=[Πi=1Nt+2pit+2]1/Nt+2[Πi=1Nt+2p^it(zit+2)]1/Nt+2[Πi=1N+2tp^it+2]1/Nt+2[Πi=1Nt+2p^it(zit+2)]1/Nt+2.(8.29b)

8.169 For a fixed-base, bilateral comparison using either equation (8.29a) or (8.29b), the hedonic equation is estimated only for one period, the current period t + 2 in equation (8.29a) and the base period t in equation (8.29b). For reasons analogous to those explained in Chapters 16, 17, and 18, a symmetric average of these indices would have some theoretical support. It would be useful as a retrospective study to compare the results from both approaches (8.29a) and (8.29b). If the discrepancy is large, the results from either should be treated with caution, similar to the way a large Laspeyres and Paasche spread would cast doubt on the use of either of these indices individually. It would be evidence for the need to update the regressions more often.

8.170 Note that a geometric mean of equations (8.29a) and (8.29b) uses all of the data available in each period, as does the hedonic index using a time dummy variable in (8.28). If in (8.28) there is a new commodity in period t + 2, it is included in the data set and its quality differences controlled for by the regression. Similarly, if old commodities drop out, they are still included in the indices in the periods in which they exist. This is part of the natural estimation procedure, unlike using matched data and hedonic adjustments on noncomparable replacements when commodities are no longer available.

8.171 With the dummy variable approach, there is no explicit weighting in its formulation in equations (8.29a) and (8.29b), and this is a serious disadvantage. In practice, cutoff sampling might be employed to include only the most important commodities or, if value of output data are available, a WLS—as opposed to OLS—estimator used with value of output shares as weights, as discussed in Appendix 22.1.

8.172 The indices ask counterfactual questions. Asking what the price of a model with characteristics z would have been if it had been on the market in a period ignores the likelihood that the appearance of that model would in turn alter the demand for other computers, thus altering the coefficients of the hedonic regression as well. The matter is particularly problematic when backcasting, that is, using a current period’s specification in some previous period’s regression as in equations (8.29a) and (8.29b). If the specifications increase rapidly, it may not be sensible to ask the value of some high-tech model when such technology was in an earlier stage of development. It should be kept in mind that hedonic coefficients may as much reflect production technology as demand (see Chapter 22), and old technologies simply may not have been able to produce goods to the standards of later ones. The question reversed—what would be the value of a previous period’s specification in a subsequent period’s regression—while subject to similar problems, may be more meaningful. In general, the solution lies in estimating regressions as often as possible, especially in markets subject to rapidly changing technologies.

8.173 There are alternative formulations to those in equations (8.29a and b). Some of the sampled varieties in period t + 2 and t will be matched, and there is a case for using the actual prices for these comparisons and predicted prices when unmatched and unavailable, as outlined by de Haan (2007).

8.174 There is also a formal analysis as to why the results from time dummy and the period-on-period (imputation) indices described above differ in Diewert, Heravi, and Silver (2009).

G.2.3 Superlative and exact hedonic indices (SEHI)

8.175 In Chapter 18 Laspeyres and Paasche bounds will be defined on a theoretical basis, as will superlative indices, which treat both periods symmetrically. These superlative formulas, in particular the Fisher index, are also seen in Chapter 17 to have desirable axiomatic properties. The Fisher index is supported from economic theory as a symmetric average of the Laspeyres and Paasche bounds, and it was found to be the most suitable such average of the two on axiomatic grounds. The Törnqvist index will be shown to be best from the stochastic viewpoint and also not require strong assumptions for its derivation from the economic approach as a superlative index. The Laspeyres and Paasche indices are found to correspond to (be exact for) underlying Leontief aggregator functions with no substitution possibilities, whereas superlative indices are exact for flexible functional forms including the quadratic and translog forms for the Fisher and Törnqvist indices, respectively. If data on prices, characteristics, and quantities are available, analogous approaches and findings arise for hedonic indices (Fixler and Zieschang, 1992 and Feenstra, 1995). Consider a theoretical index for an export price index from the resident producer’s perspective, but now only defined over products defined in terms of their characteristics. The prices are still of products, but they are wholly defined through p(z). An arithmetic aggregation for a linear hedonic equation finds a Laspeyres lower bound (as quantities supplied are increased with increasing relative prices) is given by

R(p(z)t,S(v)t1)R(p(z)t1,S(v)t1)Σi=1Nxit1p^itΣi=1Nxit1pit1=Σi=1NSit1(p^itpit1)(8.30a)

where R(.) denote the revenue at a set of output prices; input quantities, v; and technology, S, following the fixed-input output price index model. The price comparison is evaluated at a fixed level of period t – 1 technology and inputs. sit1 are the shares in total value of output of product i in period t – 1, sit1=xit1pit1/Σi=1Nxit1pit1 and

p^itpitΣk=1Kβkt(ziktzikt1)(8.30b)

are prices in period t adjusted for the sum of the changes in each quality characteristic weighted by their coefficients derived from a linear hedonic regression. Note that the summation is over the same i in both periods because replacements are included when a product is missing and equation (8.30b) adjusts their prices for quality differences.

8.176 A Paasche upper bound is estimated as where

R(p(z)t,S(t)t)R(p(z)t1,S(v)t)Σi=1Nxitp^itΣi=1Nxitpit1=[Σi=1NSi*t(pit1p^it)]1(8.31a)
wheresi*t=xitp^it/Σi=1Nxitp^itandp^it1pit1+Σk=1Kβkt1(ziktzikt1),(8.31b)

which are prices in period t – 1 adjusted for the sum of the changes in each quality characteristic weighted by its respective coefficients derived from a linear hedonic regression.

8.177 In Chapter 18 an economic theory of index numbers will be outlined and a theoretical output (export) defined for which the Laspeyres PL and Paasche PP form bounds on their “true” economic theoretic indices,

PLP(p0,p1,α)PP.(8.32)

8.178 The SEHI approach, thus, first utilizes the coefficients from hedonic regressions on changes in the characteristics to adjust observed prices for quality changes. Second, it incorporates a weighting system using data on the value of output of each model and their characteristics, rather than treating each model as equally important. Finally, it has a direct correspondence to formulation defined from economic theory.

8.179 Semi-logarithmic hedonic regressions would supply a set of β coefficients suitable for use with these base- and current-period geometric bounds:

Πi=1N(pitp^it1)SitR(p(z)t,q,T)R(p(z)t1,q,T)Πi=1N(p^itp^it1)Sit1,(8.33a)
p^it1pit1exp[Σk=1Kβkt1(ziktzikt1)]p^itpitexp[Σk=1Kβkt1(ziktzikt1)].(8.33b)

8.180 In equation (8.33a) the two bounds on their respective theoretical indices have been shown to be brought together. The calculation of such indices is no small task. For examples of its application see Silver and Heravi (2001a and 2003) for comparisons over time and Kokoski, Moulton, and Zieschang (1999) for price comparisons across areas of a country.

8.181 Note that unlike the hedonic indices in Sections G.2.1 and G.2.2, the indices in both parts of equations (8.30), (8.31), and (8.33) need not be based on matched data. Silver and Heravi (2001a and 2003) used scanner data for the universe of transactions via a two-stage procedure whereby, first, cells were defined according to major price-determining features, such as all combinations of brand, outlet type, and (for television sets) screen size—much like strata. There may be a gain in the efficiency of the final estimate because the adjustment is for variation within strata, much in the way that stratified random sampling improves on simple random sampling. The average price in each matched cell could then be used for the price comparisons using (8.30a), (8.31a), or (8.33a), except that—to ensure that each cell’s quality differences from characteristics other than these major ones did not influence the price comparison—adjustments would be made for quality changes using (8.30b), (8.31b), or (8.33b). This allowed all matched, old unmatched, and new unmatched data to be included because, if the average price in, say, a cell of (8.30a) was increased because of the inclusion of a new improved product, (8.30b) would be used to remove such improvements, on average. Consider, for example, a brand X, 14-inch television set without stereo sound, assembled by establishments in a given elementary aggregate industrial group. In the next period there may be matched cells: a 14-inch television set for brand X, but which also includes stereo. The new model may have to be grouped in the same cell with the brand X, 14-inch television sets with and without stereo and the average price of the cells compared in (8.30a), (8.31a), or (8.33a), with a quality adjustment for the stereo of the form (8.30b), (8.31b), or (8.33b) undertaken. There may be a gain in the efficiency of the final estimate because the adjustment is for within-strata variation, much in the way that stratified random sampling improves on simple random sampling. The estimated coefficient for stereo would be derived from a hedonic equation estimated from data of other television sets, some of which possess stereo.

8.182 The above has illustrated how weighted index number formulas might be constructed using data on prices, quantities, and characteristics for a product when the data are not matched. This is because continuing with matched data may lead to errors from (1) multiple quality adjustments from products no longer produced and their noncomparable replacements and (2) sample selectivity bias from sampling from a replacement universe as opposed to a double universe.

G.2.4 Difference between hedonic indices and matched indices

8.183 In previous sections, the advantages of hedonic indices over matched comparisons were referred to in terms of the inclusion by the former of unmatched data. This relationship is developed more formally here. Triplett (2004) argued and Diewert (2003) showed that an unweighted geometric mean (Jevons) index for matched data gives the same result as a logarithmic hedonic index run on the same data. Consider two periods so T = 2 and assume that the models are matched in each of the two periods so that the set of models in period 1 is equal to that in period 2, S(1) = S(2), and the number of models N(1) = N(2) = M so that the same M models are available in each period. Hence the model characteristics are the same in each, that is, we have zmtk = zmk say, for t = 1,2; m = 1, . . . , M; and k = 1, . . . K. Silver and Heravi (2005) have shown the following formula for the log of the hedonic price index:

lnP2/P1=1N(12)ΣmϵS(12)ln(pm2/pm1)+1N(12)ΣmϵS(2¬1)[lnpm2Σk= 1Kzm2kβk*α2*]1N(12)ΣmϵS(1¬2)[lnpm1Σ= 1Kzm1kβk*α1*].(8.34)

8.184 The first set of terms on the right-hand side of equation (8.34) is the matched-model contribution to the overall index, ln P2/P1. The next two sets of terms are, respectively, the change in price due to unmatched models existing in period 2 but not in 1, and unmatched models existing in period 1 but not in 2. These expressions are not captured in a matched-models index. If the second set of terms, [1/N(12)]×ΣmϵS(2¬1)[ln(pm2Σk=1Kzm2kβk*α2*], is positive, then the matched-model price index is too low and must be adjusted upward. Consider a new model m introduced in period 2. If (the logarithm of) its price (ln pm2) is above that predicted from a period 2 hedonic regression (Σk=1Kzm2kβk*α2*), then this will raise the overall price index, and a matched-model index would be too low if it ignored such new models. Similarly, consider the last set of terms in equation (8.34) and an unmatched old model, introduced in period 1 but no longer available in period 2. If it was priced in period 1 above its period 1 predicted price then the matched-model price index would be too high (note the negative sign). The extent and nature of the bias depend on the pricing strategy of new and old models. The hedonic dummy variable approach, in its inclusion of unmatched old and new observations, can be seen from equation (8.34) to possibly differ from a geometric mean of matched price change. The extent of any difference depends, in this unweighted formulation, on the proportions of old and new commodities leaving and entering the sample and on the price changes of old and new ones relative to those of matched ones. If the market for commodities is one in which old quality-adjusted prices are unusually low while new quality-adjusted prices are unusually high, then the matched index will understate price changes. Different market behavior and changes in technology will lead to different forms of bias.

8.185 If sales weights replace the number of observations in equation (8.34), then different forms of weighted hedonic indices can be derived as explained in Chapter 22, Section A.5, and formally derived in Silver and Heravi (2005).

G.3 Chaining

8.186 An alternative approach for dealing with commodities with a high turnover is to use a chained index instead of the long-term fixed-base comparison. A chained index compares prices of items in period t with period t + 1 (Indext, t+1) and then as a new exercise, studies the universe of commodities in period t + 1, and matches them with items in period t + 2. These links, Indext, t +1 and Indext+1,t+2, are combined by successive multiplication continuing to, say, Indext+5,t+6 to form Indext, t +6: Only items available in both period t and period t + 6 would be used in a fixed-base trade price index. Consider the five commodities 1, 2, 5, 6, and 8 over the four months of January to April as shown in Table 8.2. The price index for January compared with February (J:F) involves price comparisons for all five commodities. For (F:M), it involves commodities 1, 4, 5, and 8; for (M:A), it involves commodities 1, 3, 4, 5, 7, and 8. The sample composition changes for each comparison as commodities die and are born. Price indices can be calculated for each of these successive price comparisons using any of the unweighted formulas described in Chapter 22. The sample will grow when new commodities appear and shrink when old commodities disappear, changing in composition through time.

8.187 Sample depletion may be reduced in long-run comparisons by the judicious use of replacement items. However, as discussed in the next chapter, the replacement sample would include a new commodity only when a replacement was needed, irrespective of the number of new commodities entering the market. Furthermore, the replacement commodity is likely to be either of a similar quality, to facilitate quality adjustment and thus have relatively low sales, or be of a different quality with relatively high sales but requiring an extensive quality adjustment. In either case, this is unsatisfactory.

8.188 Chaining, unlike hedonic indices, does not use all the price information in the comparison for each link. Commodities 2 and 6, for example, may be missing in March. The index makes use of the price information on commodities 2 and 6, when they exist, for the January–February comparison but does not allow their absence to disrupt the index for the February–March comparison. It may be that commodity 4 is a replacement for commodity 2. Note how easily it is included as soon as two price quotes become available. There is no need to wait for rebasing or sample rotation. It may be that commodity 7 is a replacement for commodity 6. A quality adjustment to prices may be required for the February–March comparison between commodities 6 and 7, but this is a short-run, one-off adjustment, The compilation of the index continues for March–April using commodity 7 instead of commodity 6.

8.189 The chained approach has been justified as the natural discrete approximation to a theoretical Divisia index (Forsyth and Fowler, 1981; and Chapter 17). Reinsdorf (1998) has formally determined the theoretical underpinnings of the index, concluding that in general, chained indices will be good approximations of the theoretical ideal. However, they are prone to bias when price changes “swerve and loop,” as Szulc (1983) has demonstrated (see also Forsyth and Fowler, 1981; and de Haan and Opperdoes, 1997).

8.190 The dummy variable hedonic index uses all of the data in January and March for a price comparison between the two months. Yet the chained index ignores unmatched successive pairs as outlined above; nevertheless, this is preferable to its fixed-base equivalent. The hedonic approach, by predicting from a regression equation, naturally has a confidence interval attached to such predictions. The width of the interval is dictated by the fit of the equation, the distance of the characteristics from their mean, and the number of observations. Matching, chained or otherwise, does not suffer from any prediction error. Aizcorbe, Corrado, and Doms (2001) undertook an extensive and meticulous study of high-technology goods (personal computers and semiconductors) using quarterly data for the period 1993–99. The results from comparable hedonic and chained indices were remarkably similar over the seven years of the study. For example, for desktop central processing units, the index between the seven years of 1993: Q1 and 1999: Q4 fell by 60.0 percent (dummy variable hedonic), 59.9 percent (chained Fisher), and 57.8 percent (chained geometric mean). The results differed only in quarters when there was a high turnover of commodities, and, in these cases, such differences could be substantial. For example, for desktop central processing units in 1996: Q4, the 38.2 percent annual fall measured by the dummy variable hedonic method differed from the chained geometric mean index by 17 percentage points. Thus, with little model turnover, there is little discrepancy between hedonic and chained matched-models methods and, for that matter, fixed-base matched indices. It is only when binary comparisons or links have a high model turnover that differences arise (see also Silver and Heravi, 2003 and 2005).

8.191 There is a possibility that the introduction of new models and exits of old ones instantaneously affects the prices of all existing models. In such a case, the price changes of existing models will suffice. They will reflect the price changes of new entrants and old departures not part of the sample. This argument is used for the case that direct matched-models comparisons, chained matched-models comparisons, and hedonic indices should give the same results. It is an empirical matter, and its plausibility will vary among industries. It is more likely to apply to fast-moving goods with few to no development costs or barriers to entry.

8.192 It is possible to make up for missing prices by using a partial, patched hedonic estimate as discussed above. Dulberger (1989) computed hedonic indices for computer processors and compared the results with those from a matched-models approach. The hedonic dummy variable index fell by about 90 percent from 1972–84, about the same as for the matched-models approach where missing prices for new or discontinued commodities were derived from a hedonic regression. However, when using a chained matched-models approach with no estimates or imputations for missing prices, the index fell by 67 percent. It is also possible to combine methods; de Haan (2007) used matched data when available and the time dummy only for unmatched data—his double-imputation method.

H. Long-Run and Short-Run Comparisons

8.193 This section outlines a formula to help quality adjustment. The procedure can be used with all of the methods outlined in Sections D and E. Its innovation arises from a possible concern with the long-run nature of the quality-adjusted price comparisons being undertaken. In the example in Table 8.2, prices in March were compared with those in January. Assumptions of similar price changes are required by the imputation method to hold over this period for long-run imputations. This gives rise to increasing concern when price comparisons continue over longer periods, such as between January and October, January and November, and January and December, and even subsequently. In this section, a short-run formulation outlined in Sections C.3.3 and D.2 is more formally considered to help alleviate such concerns. Consider Table 8.5, which, for simplicity, has a single commodity A that exists throughout the period, a commodity B that is permanently missing in April, and a possible replacement C in April.

H.1 Short-run comparisons: Illustration of some quality-adjustment methods

8.194 A comparable replacement C may be found. In the previous example, the focus was on the use of the Jevons index at the elementary level because it is shown in Chapter 21 that this has much to commend it. The example here uses the Dutot index, the ratio of arithmetic means. This is not to advocate it but only to provide an example using a different formulation. The Dutot index also has much to commend it on axiomatic grounds but fails the commensurability (units of measurement) test and should be used only for homogeneous items. The long-run Dutot index for April compared with January is

PD[Σi=1NpiApr/NΣi=1NpiJan/N],

which is 8/5 = 1.30, a 30 percent increase. The short-run equivalent is the commodity of a long-run index up to the immediately preceding period and an index for the preceding to the current period, that is, for period t + 4 compared with period t:

PD[Σi=1Npit+3/NΣi=1Npit/N]×[Σi=1Npit+4/NΣi=1Npit+3/N],(8.35)
Table 8.5.

Example of Long-Run and Short-Run Comparisons

article image
Note: n/a = not available. Figures in bold are estimated quality-adjusted prices described in the text.

or, for example, using a comparison of January with April:

PD[Σi=1NpiMar/NΣi=1NpiJan/N]×[Σi=1NpiApr/NΣi=1NpiMar/N],

which is of course 65×86=1.30

article image
as before.

8.195 Consider a noncomparable replacement with an explicit quality adjustment: say C’s value of 6 in April is quality-adjusted to be considered to be worth only 5 when compared with the quality of B. The quality adjustment to prices may have arisen from an option cost estimate, a quantity adjustment, a subjective estimate, or a hedonic coefficient as outlined above. Suppose the long-run comparison uses an adjusted January price for C, which is B’s price of 3 multiplied by 6/5 to upgrade it to the quality of C, that is, 6/5 × 3 = 3.6. From April onward, the prices of the replacement commodity C can be readily compared with its January reference period price. Alternatively, the prices of C in April onward might have been adjusted by multiplying them by 5/6 to downgrade them to the quality of B and enable comparisons to take place with commodity B’s price in January: for April the adjusted price is 5/6 × 6 = 5; for May, the adjusted price is 5.8; and for June, it is 6.67 (see Table 8.5). Both procedures yield the same results for long-run price comparisons. The results from both methods (r ounding errors aside) are the same for commodity B.

8.196 However, for the overall Dutot index, the results will differ because the Dutot index weights price changes by their price in the initial period as a proportion of total price (Chapter 21, equation (21.1)). The two quality-adjustment methods will have the same price changes but different implicit weights. The Dutot index in May is 9/5.6 = 1.607 using an adjustment to the initial period, January’s price, and 7.8/5 = 1.56 using an adjustment to the current period, May’s price. The short-run indices give the same results for each adjustment:

85.6×98=1.607 using an adjustment to the initial period, January’s price, and

75×7.87=1.56 using an adjustment to the current period, May’s price.

8.197 The overlap method may also take the short-run form. In Table 8.5, there is a price for C in March

of 5 that overlaps with B in March. The ratio of these prices is an estimate of their quality difference. A long-run comparison between January and April would be (6×45+2)/5=1.36. The short-run comparison would be based on the commodity of the January to March and March to April link: (6.86×65)=1.36

8.198 At this unweighted level of aggregation, it can be seen that there is no difference between the long-run and short-run results when commodities are not missing, comparable replacements are available, explicit adjustments are made for quality, or the overlap method is used. The separation of short-run (most recent month-on-month) and long-run changes may have advantages for quality assurance to help spot unusual short-run price changes. But this is not the concern of this chapter. The short-run approach does, however, have advantages when imputations are made.

H.2 Implicit short-run comparisons using imputations

8.199 The use of the short-run framework has been considered mainly for temporarily missing values, as outlined by Armknecht and Maitland-Smith (1999) and Feenstra and Diewert (2000). However, similar issues arise in the context of quality adjustment. Consider again Table 8.5, but this time there is no replacement commodity C, and commodity A’s prices have been changed to trend upward. Commodity B is again missing in April. A long-run imputation for commodity B in April is given by 3.52×3=5.25. The price change is thus (5.25 + 3.5)/5 = 1.75, or 75 percent. One gets the same result by simply using commodity A (3.5/2 = 1.75), because the implicit assumption is that price movements of commodity B, had it continued to exist, would have followed those of A. However, the assumption of similar long-run price movements may in some instances be difficult to support over long periods. An alternative approach would be to use a short-run framework whereby the imputed price for April is based on the (overall) mean price change between the preceding and current period, that is, 3.52.5×4=5.6 in the above example. In this case, the price change between March and April is (5.6 + 3.5)/(2.5 + 4) = 1.40. This is combined with the price change between January and March: (6.5/5) = 1.30, making the price change between January and April 1.30 × 1.40 = 1.82, an 82 percent increase.

8.200 Consider why the short-run result of 82 percent is larger than the long-run result of 75 percent.

The price change for A between March and April of 40 percent, on which the short-run imputation is based, is larger than the average annual change of A, which is just over 20 percent. The extent of any bias from this approach was found in the previous section to depend on the ratio of missing values and the difference between the average price changes of the matched sample and the quality-adjusted price change of the commodity that was missing, had it continued to exist. The short-run comparison is to be favored if, as is likely, the assumption of similar price changes is considered more likely to hold than the long-run one.

8.201 There are data on price changes of the commodity that is no longer available—commodity B in Table 8.5—up to the period preceding the period in which it is missing. In Table 8.5, commodity B has price data for January, February, and March. The long-run imputation makes no use of such data by simply assuming that price changes from January to April are the same for B as for A. Let the data for B’s prices in Table 8.5 (second to last row) now be 3, 4, and 6 in January, February, and March, respectively, instead of 3, 3, and 4. The long-run estimate for B in April is 5.25 as before. The estimated price change between March and April for B is now a fall from 6 to 5.25. A short-run imputation based on the price movements of A between March and April would more correctly show an increase from 6 to (3.5/2.5) × 6 = 8.4.

8.202 There may, however, be a problem with the continued use of short-run imputations. Returning to the data for A and B in Table 8.5, consider what happens in May. Adopting the same short-run procedure, the imputed price change is given in Table 8.5 as 4/3.5 × 5.6 = 6.4 and for June as (5/4) × 6.4 = 8. In the former case, the price change from January to May is

[(6.4+4)(5.6+3.5)]×[(5.6+3.5)(3.2)]=2.08

and in the case of June

[(8+5)(6.4+4)]×[(6.4+4)(3+2)]=2.60

against long-run comparisons for May:

[((4/2)×3+4)(3+2)]=2.00

and long-run comparisons for June:

[((5/2)×3+5)(3+2)]=2.50.

8.203 A note of caution is required here. The comparisons use an imputed value for commodity B in April and also an imputed one for May. The price comparison for the second term in equation (8.35), for the current versus immediately preceding period, uses imputed values for commodity B. Similarly, for the January to June results, the May to June comparison uses imputed values for commodity B for both May and June. The pragmatic needs of quality adjustment may demand this. If comparable replacements, overlap links, and resources for explicit quality adjustment are unavailable, an imputation must be considered. However, using imputed values as lagged values in short-run comparisons introduces a level of error into the index that will be compounded with their continued use. Long-run imputations are likely to be preferable to short-run changes based on lagged imputed values unless there is something in the nature of the industry that cautions against such long-run imputations. There are circumstances when the respondent may believe the missing commodity is missing temporarily, and the imputation is conducted under the expectation that production will subsequently continue. A wait-and-see policy is adopted under some rules—three months, for example—after which it is deemed to be permanently missing. These are the pragmatic situations that require imputations to extend over consecutive periods. These circumstances promote lagged imputed values to compare against current imputed values. This is cautioned against, especially over a period of several months. There is an intuition that the period in question should not be extensive. First, the effective sample size is being eaten up as the use of imputation increases. Second, the implicit assumptions of similar price movements inherent in imputations are less likely to hold over the longer run. Finally, there is some empirical evidence, albeit from a different context, against using imputed values as lagged actual values. (See Feenstra and Diewert’s 2000 study using data from the U.S. Bureau of Labor Statistics for its International Price Program.)

8.204 The short-run approach described above will be developed in the next section, where weighted indices are considered. The practice of estimating quality-adjusted prices is usually at the elementary commodity level. At this lower level, the prices of commodities may subsequently be missing, and replacements with or without adjustments and imputations are used to allow the series to continue. New commodities and varieties are also being introduced; the switching of sales between sections of the index becomes prevalent. The turmoil of changing quality is not just about the maintaining of similar price comparisons but also about the accurate reweighting of the mix of what is produced. Under a Laspeyres framework, the bundle is held constant in the base period, so any change in the relative importance of commodities produced is held to be of no concern until the next rebasing of the index. Yet capturing some of the very real changes in the mix of what is produced requires procedures for updating the weights. This was considered in Chapter 6. The concern here is with a higher-level procedure equivalent to the short-run adjustments discussed above. It is one particularly suited to countries where resource constraints prohibit the regular updating of weights through regular household surveys.

H.3 Single-stage and two-stage indices

8.205 Consider aggregation at the elementary level. This is the level at which prices are collected from a representative selection of establishments across regions in a period and compared with the matched prices of the same commodities in a subsequent period to form an index for a good. Lamb is an example of a good in an index. Each price comparison is equally weighted unless the sample design gave proportionately more chance of selection to commodities with more sales. The elementary price index for lamb is then weighted and combined with the weighted elementary indices for other commodities to form the XMPI. A Jevons elementary aggregate index, for example, for period t + 6 compared with period t is given as

PJ=ΣiN(t+6)N(t)(pit+6/pit).(8.36)

Compare this with a two-stage procedure:

PJ=ΣiN(t+5)N(t)(pit+5/pit)×ΣiN(t+5)N(t+6)(pit+6/pit+5).(8.37)

8.206 If a commodity is missing in period t + 6, an imputation may be undertaken. If equation (8.36) is used, the requisite assumption is that the price change of the missing commodity, had it continued, is equal to that of the average of the remaining commodities over the period t to t + 6. In equation (8.37), the missing commodity in period t + 6 may be included in the first stage of the calculation, between periods t and t + 5, but excluded in the second stage, between periods t + 5 and t + 6. The requisite assumption is that price changes between t – 1 and t are similar. Assumptions of short-run price changes are generally considered to be more valid than their long-run counterparts. The two-stage framework also has the advantage of including in the worksheet prices for the current period and the immediately preceding one, which, as will be shown in Chapter 10, promotes good data validity checks.

8.207 Feenstra and Diewert (2000) applied a number of mainly short-run imputation procedures to price comparisons for the U.S. Bureau of Labor Statistics International Price Program. Although such price indices are not the direct interest of this Manual, the fact that about one-quarter of the individual commodities tracked did not have price quotations in any given month makes it an interesting area to explore the results from different imputation procedures. When using the two-stage procedure, they advise against carrying forward imputed prices as if they were actual values for the subsequent price comparison. The resulting price relatives for the subsequent period based on prior imputations had a standard deviation about twice that of price relatives where no imputation was required, leading them to conclude that such a practice introduced a significant amount of “noise” into the calculation. Feenstra and Diewert (2000) found more variance in price changes in the long-run imputation method than the short-run method. They also found from both theory and empirical work that when actual prices are available in a future data set and they are used to interpolate back on a linear basis the missing prices, such estimates lead to much lower variances than the short-run imputation approach. However, such linear interpolations require the statistical agency to store past information until a price quote becomes available, interpolate back the missing price, and then publish a revised XMPI.

Appendix 8.1 Data for Hedonic Regression Illustration

article image
article image