6. Price Collection
- International Monetary Fund
- Published Date:
- September 2004
6.1 This chapter gives an overview of price collection issues. It describes a range of options for each aspect of collection, but it is not prescriptive, since different solutions can be used depending on individual country circumstances. Price collection is a vital part of the overall PPI compilation process. Without good quality price collection procedures, it is difficult or impossible to produce accurate and reliable results, regardless of how rigorous the subsequent processing is throughout the remaining steps of producing the PPI. chapter 12 on organization and management of the PPI also provides guidance on price collection within the framework of the whole PPI system.
B. Timing and Frequency of Price Collection
6.2 Calculating the PPI entails collecting prices from businesses relating to particular products and time periods. Businesses can be both sellers or buyers of products, so that prices may be collected for sales of goods and services for use in an output price index or purchases of goods and services used in the production process for use in an input price index. Both output and input PPIs are often needed, particularly for use as deflators.
6.3 The frequency of collection is often monthly, although a number of countries operate a quarterly collection system. For the purposes of this chapter, it is assumed that price collection is monthly, which is the most common practice. When collecting prices for a particular period, there are two basic choices of collection period: point-in-time or period averages.
B.1 Point-in-time prices
6.4 Point-in-time prices relate to the price of a product on a particular date in the month—for example, first day, first Monday, the nearest trading day to the fifteenth of the month, etc. This approach makes the collection date straightforward, and it should be well understood by the business establishment that prices provided relate to transactions on that date.
6.5 The main advantage of point-in-time pricing is that comparisons from month to month will be consistent, which is particularly important when there are step changes in prices taking place during the month, such as a general price increase or duty changes. One of the disadvantages of a set point in time for producer price indices is that a transaction may not have taken place on the specified date. If this happens, respondents can be asked to provide details of a transaction that occurred as near as possible to the specified date. Another problem is that point-in-time estimates are more susceptible to short-term external influences (for example, extreme weather, labor stoppages) that could affect the price on the particular day of price collection. They may also miss short-term price changes (for example, rise and fall) that occur between pricing dates.
B.2 Period prices
6.6 Period prices are an estimate of the price across the month and so are an average price for the month. A period price should take into account when price changes occurred during the month. For example, if a product was priced at 10 for the first 10 days of a month, and then increased to 15 for the remaining 20 days, then the average price would be 13.33 (that is, [10 X 10 + 20 X 15]/30). This averaging is usually done by the statistical office and requires the exact date of the price change to be supplied by the respondent.
6.7 This approach usually yields a smoother time series and is less susceptible to the timing of price increases. The method is also easier for respondents since they can select a transaction and specify the relevant transaction date within the period. A key feature (compared with point-in-time estimation) with this method is that when a price changes partly through the month, the full effect of the price change is not included in the index until the following month. This is appropriate for an index used for deflation purposes as well as for an inflation measure.
6.8 Often a single price quotation is taken to represent the average price over the particular reference period. A more accurate measure of an average transaction price is the unit value price. In theory, unit value prices, which are total sales divided by the total number of units sold in a period, are the most complete method of pricing.
6.9 If this method is used, the commodity must be either homogeneous or able to be expressed in terms of some common physical unit. A homogeneous commodity can be distinguished by
Its point of purchase (outlet effect),
The various competing brands or product lines of the commodity being sold at an outlet (brand effect), or
The various package sizes at which the commodity is sold (packaging effect)
6.10 The time period in which unit values are calculated should be the “longest period which is short enough so that individual variations in price within the period are regarded as unimportant” (Diewert, 1995a.)
6.11 Unfortunately, this method is very problematic and is not generally recommended, since any change in product quality, product mix, or timing can seriously distort the average unit price. In limited circumstances—for example, for a highly volatile but narrowly defined and homogeneous product like petroleum—this method can be used.
6.12 Often the index will be less timely when compared with point-in-time estimates, since the average cannot be calculated until the end of the period. Further, care must be taken to ensure that the average prices relate to a narrowly defined product of constant quality, rather than a broad commodity group.
6.13 Transportation costs should be excluded from the unit value calculation because the pricing basis for output PPIs is the basic price—that is, the amount received by the producer, exclusive of any taxes on products and transport and trade margins.
In other words, the pricing point is ex-factory, ex-farm, ex-service provider, and so on.
B.3 Choice of point-in-time or period prices
6.14 The choice of collection period is influenced by a number of issues, such as the frequency of collection, the practicalities of price collection, and the uses of the index. The choice of collection method becomes less important the more frequent the collection; thus, the choice is more important for quarterly collection than monthly collection, although it is still an important consideration for monthly collection. The use of the index is an important consideration. Since PPIs are used to deflate sales data, ideally the index should relate to the time period of the sales flows. Most economic statistics relate to a period rather than a point in time, and so again, in principle, the price index should do the same.
6.15 A distinction can be made between the frequency of collection and timing of observations. Monthly prices can be observed quarterly, for example.
6.16 The choice of collection frequency is determined by issues such as costs and the periodicity needed for deflation of output or sales data. In the EU, members are required to provide monthly data to the Statistical Office of the European Communities (Eurostat) under the short-term indicators regulation. Normally, prices will be collected from every producer in the sample for each time period. This will ensure that all price changes are captured by the PPI.
6.17 While collecting prices for every period is appropriate for most industries, there may be industries where prices are generally stable, products take a long time to produce, or prices change at predetermined times—for example, each January. Collecting a price every period in such a situation may be an unnecessary burden on businesses. For respondents in these industries, a price for each period is still required, although it may be possible to reduce the periodicity of collection; this is a case where using “carryforward” price imputation is desirable. In some exceptional circumstances, respondents may be allowed to give forward prices, but care must be taken to avoid complacency. In these cases, usually involving long-term contracts, respondents can make a commitment that the price will not change in the defined forward period. If this turns out to be incorrect, the frequency of reporting would be changed.
B.5 Definition of a price observation
6.18Chapter 5 on sampling explains how a representative sample of products should be selected. This section describes how prices for these selected products should be determined and collected. A price observation is defined as the price of a specific product at the point in time or for the period of price collection and its terms of sale. To ensure consistency in the final index, the price observation should compare like with like for each period. The product should be defined as tightly as possible so that the returned price is consistent from period to period and changes in quality can be identified (see Chapter 7, Section B). The price should be one that a customer has paid for the specified product and include all available discounts and special offers—that is, a real transaction price. (See Section D.4.1 below.)
6.19 If the product specification changes from one period to another, the price needs to be adjusted to ensure consistency. For instance, the quantity per order may increase, resulting in a lower unit-selling price. If the new quantity sold was available at the same unit price last period as it is this period, this is not a genuine price decrease and should not be reflected in the index. Rather, the comparison should be made between the same quantity purchased in both periods so that the index compares the same specifications (that is, like with like).
6.20 The returned price should be provided in a consistent currency, but even for domestic sales this may not necessarily be the currency of the home country (for example, price could be provided in euros or U.S. dollars). In that case procedures should be in place to convert all returned prices to home currency values. However, it must be clear what coverage is intended—either production for the home market or for the home and export market.
B.5.1 List prices
6.21 The PPI’s aim is to measure actual prices paid to or received from producers for goods or services. These are commonly referred to as transaction prices. By definition, these prices include all discounts or rebates given.
6.22 The price of goods or services as quoted in a catalogue or advertisement is often referred to as the list price, book price, or recommended retail price. These prices are typically higher than transaction prices, as discounts or rebates apply to transaction prices.
6.23 In most areas of the economy, the prices actually paid or received for goods or services are not the list prices. Typically, negotiations between the producer and purchaser result in some form of discount or rebate, particularly to large purchasers. In most cases they are substantial reductions off the list price and will vary over time. PPI compilers should ensure that actual transaction prices are obtained rather than list prices.
6.24 It is usually easier for a respondent to provide a list price rather than a transaction price. For the reasons already stated, this is not appropriate. Because it is difficult to price a transaction, to achieve constant quality, compilers should ensure that the product priced is the same as that priced in the previous period.
B.6 Issues for high inflation or hyperinflation
6.25 During periods of high inflation or hyperinflation, the timing of price collection takes on significant importance. Prices may well change substantially during the collection period.
6.26 The frequency of collection also becomes more important, so quarterly collection may be inadequate for policymakers in a hyperinflationary period. Even in times of low inflation, it is important that early signals of upstream inflationary pressure are captured. Validation of the data may also prove to be more difficult, because it is likely that every price quote would fail validation checks set during “normal” inflation, and it would be more difficult to spot erroneous returns.
6.27 On a wider scale, there is also a potential problem of feedback or circularity fueling inflation. Some companies may use the PPI to fix their prices (as part of a contract with the customer), which could then feed into the calculation of future PPIs. There is always a risk of this in detailed indices, but the risk would be higher in periods of very high inflation.
C. Product Specification
6.28 The PPI price collection survey is unusual compared with most business surveys, since there is a requirement to get a detailed product specification from respondents before the routine monthly collection can begin; this process is often called initialization or recruitment.
6.29 A separate set of processes and survey forms is required for the initialization procedure. The collection method used can also be different for the initialization period—for example, it may be possible and desirable to make a personal visit to each new respondent at initialization, but subsequent routine price collection would be done via postal collection. The initialization form should put more emphasis on explaining the purpose of the survey and contain more details about the product specification requirements. This form should also contain a product list for the respondent to identify which products they produce from the list. The initialization process may also be conducted by specialist staff such as field officers (see Section D.6)
6.30 The following section on product specification can apply to both the initialization process and routine collection procedures.
C.1 Purpose of product specification
6.31 For each product group or service, prices for a set of specific representative products need to be fully specified for pricing. These products should be typical of the price movements of the range of individual products within the product group or service under consideration. The selection of products from within the product range of each producer would ideally be undertaken from a complete census of the relevant transactions. Obviously, in most cases this information is not available. In some cases there can be a trade-off between having infrequent data on a more complete and detailed basis and more frequent product-updating procedures that rely on the respondents to the price collection to self-select products that are representative of their output and, hence, product group. The sample selection aspects of product selection are covered in Chapter 5.
C.2Aspects of product specification
6.32 There are a number of different aspects of product specification. For example, simply giving a product name will not be sufficient if the size of the package changes, which would, in turn, affect the price received. The essential purpose of a good product specification is to ensure that a consistent price is collected from period to period, relating to a consistent product with the same terms of sale in each period.Table 6.1 lists the main criteria that could affect the price of a product and could form part of a specification.
6.33 The above details combine to give a tighter specification for the product than just the description alone. Specifying a product in this way also supports the adjustment of the price associated with any changes in the product quality or the terms and conditions of sale. Some respondents object to providing full specification details because of concerns about confidentiality, and in these cases the specification can be held in detail by the statistical office, but a shorter encoded specification can be used on printed material such as forms. If this is done, it is essential to review the specification regularly.
C.3Other forms of description
6.34 For some industries, a specification for a particular product may not be appropriate. For example, some industries produce goods or services on a made-to-order basis, and the same product is not produced in successive periods. Examples of this could be furniture manufacturers, shipbuilders, and accounting services. In these instances, a generic specification may be more appropriate. This would be a specification, as described previously, but for a standard product, rather than for a specific product. This could be a product that the company has made at some point in the past, or a basic model that it customizes individually for each customer. See Section D.5.2 below for more details on this type of pricing.
D. Collection Procedures
D.1 Survey collection techniques
6.35 The aim of all survey collection techniques is to facilitate the transmission of price data from businesses to the statistical office in a secure and cost-effective manner, while minimizing the administrative burden on the respondent. A range of approaches to PPI data collection are discussed below—postal survey, automated telephone response, personal interview, telephone interview, and Internet data provision. All of these methods rely on good questionnaire design, good respondent relations, or good interviewing techniques. The highly sensitive and confidential nature of the price data provided by businesses may necessitate extra security requirements in data collection and processing.
|Product name||Company’s name for the product within the specified product group. This should ideally contain information on the model/variety of the product.|
|Serial number||For the company’s reference. This allows for changes in product name.|
|Description||In addition to the product name, this gives an opportunity for the company to specify what (if any) enhancements or add‐ons are included in the product. For example, with cars, a number of options are usually available (metallic paint, sunroof), all of which could affect the price of the product.|
|Size of transaction||The amount of the product sold in the transaction and whether volume discounts apply.|
|Units of sale||Units used in describing the product.|
|Class of customer||Some companies may have different pricing structures for different customers (for example, retail and trade). A reference number can be used to maintain customer confidentiality.|
|Discounts||Many companies offer trade, volume, competitive, or preferred customer discounts. All applicable discounts should be described.|
|Payment terms||Companies may have different prices for different payment or credit terms.|
|Carriage terms||Whether transport costs are included and what type of transport.|
|Currency||Currency the price will be provided in.|
D.2 Questionnaire design
6.36 Regardless of which data collection method is used, good questionnaire design is essential for the successful collection of prices. The questionnaire should be designed to make it easy for the producer to use and understand what is required.
6.37 The layout should facilitate the extraction of data and should contain detailed descriptions of the products to be priced. Detailed descriptions not only help the producer but also help in validation and the identification of quality changes. Quality adjustments cannot be made in the absence of detailed product specifications (see Chapter 7 for more details on quality adjustment techniques). Detailed product descriptions also ensure that the same products are priced each month, which gives important continuity and enables the statistical office to validate the data.
6.38 The questionnaire should be designed to help the respondent extract information quickly and to enable speedy and accurate processing in the office. To meet these objectives, the questionnaire should
Provide clear instructions on what the respondent is required to do;
Define why the establishment has been chosen, what the survey is, and how the data are collated or published;
Enable respondents to complete the form quickly and accurately;
Ensure supporting notes are available for each item of data to be collected;
Use plain and clear language;
Clearly identify the organization from which the survey has been sent and give a contact point and telephone number so respondents can get in touch to resolve any problems;
Request reasons for price changes; and
Ask whether the products are still representative or sold in volume.
6.39 Different designs can be used to make the questionnaire easier to complete for certain classes of respondents. For example, different designs could be used for production sector and service sector questionnaires. Also, a questionnaire with a checklist design that provides all the important specifications and price-determining characteristics will help respondents and data collectors. They will be able to verify the transaction and provide any new specifications or changes to the price basis that may apply when a previous transaction is no longer available and a replacement is selected.
6.40 One way to make the form easier for businesses to complete is to put the last recorded price on the questionnaire using a “tailored form” with unique product descriptions for each respondent. This will require the statistical agency to have much better form design and printing capabilities. It is controversial in terms of the impact on the results, however; while it is easier for the producer to complete, there is a greater risk that less care is taken in the completion of the survey, and the producer is more likely to repeat last periods price even if a price change has taken place. There is also the risk that confidentiality will be breached if the form goes astray or even to the wrong part of the organization.
6.41 For help in validation and to reduce recontact with the producer, it is useful to provide a comments block to allow the respondents to explain any unusual movements in their prices. It is also important to emphasize to the respondent that any change in specification must be reported. An example of a PPI postal survey price collection form that takes many of these issues into consideration is provided in Figure 6.1.
Figure 6.1.Example of PPI Collection Form
D.3 Medium of data collection
6.42 The following section outlines a range of survey collection methods. The principles of questionnaire design outlined above apply to each of these methods.
D.3.1 Self-completion: return of data by postal survey
6.43 Key points of good practice that should be followed in questionnaire design are outlined below.
6.44 The form should be clearly addressed to the company in question. It should
Display on the front page the name of the institution from which it has been dispatched;
Explain why it has been sent, how the results will be used, and from whom the end product can be obtained;
Include the name and number of the direct contact in the office should respondents require assistance in completing the return; and
Include any statutory obligations respondents are under to complete the form and the penalties for not doing so.
6.45 Within the form, sufficient descriptions and explanations should be included for the respondent to follow, including
Guidance notes for each section requiring data,
Clear definition of the product requiring data,
The time period or point in time to be covered by the return,
Instructions on how to change the description of the product,
Information on linking the product description to industry tariff codes, and
A period of back prices for amendment, if necessary.
6.46 Allow for changes to administrative information on the form, including
Space to record comments,
The name and contact number of the person completing the form,
Changes to the mailing address of the company, and
Notes on how to return the form (prepaid envelope).
6.47 The main advantage of the postal survey approach is that it is inexpensive, particularly when coupled with modern data-handling technology, which reduces the need for operators to physically type data into systems. Large and dispersed geographical areas can be covered for minimum extra cost. This assumes, of course, that the postal system in the country is accurate and dependable with deliveries.
6.48 Disadvantages of the postal survey approach include the difficulty in achieving a high level of response from respondents because the collection mode is not interactive. This can be mitigated if there is legislation in place to penalize for nonresponse. Potential quality problems can arise when respondents do not pay adequate attention to the notes and complete the form incorrectly. For this reason, it is wise to explain the requirement through a meeting or telephone conversation when the respondent is originally selected to participate in the survey; personal contact with the respondent, even by phone, should be encouraged as a general method for improving the quality of data returns.
6.49 Nonresponse follow-up contacts and resolving queries about reported data can add a significant cost to the postal approach.
D.3.2 Automated telephone data submission
6.50 Usually the PPI price survey collects price details for a small number of products from each respondent. The brevity of the questionnaire makes the PPI ideal for telephone-based data entry systems, in which the respondent reports the information directly over the telephone by following voice prompts and entering data using a touch-tone telephone. The prerecorded dialogue in such systems enables the respondents to report their monthly data quickly and accurately. Usually a letter is dispatched asking the respondent to make a telephone return. This approach has the advantage of making it possible to program the dialogue to allow validation of the data to take place during the telephone call. This can be done by asking the respondent to leave a voice message or by switching the call to a data collection analyst. Generally, this method is beneficial to the statistical office by reducing desk processing and hence reducing costs. Some on-line validation can take place that may benefit respondents by saving them from being recontacted by the statistical office.
6.51 The possible disadvantages of this system are confusion to some users caused by the technology, and, because respondents can leave a voice message without discussion, some further clarification contact may be needed. This method is also less useful when there are complex product specifications, which need to be updated frequently. For example, the United Kingdom does not use this collection method for the PPI computer price index.
D.3.3 Personal interviewer
6.52 This involves a face-to-face meeting with each respondent on a regular basis (for example, monthly, quarterly) by a trained interviewer to obtain the data necessary for the survey. The main advantage of this approach is that the data can be validated at the source, and problems and differences of understanding can be resolved during the discussion.
6.53 However, the big disadvantage is the cost of employing interviewers and the high travel costs, particularly where long distances are involved. There is also a disadvantage to the respondent, who would have to spend more time in face-to-face meetings with statistical office representatives. Field collection for the PPI is not as viable as for the CPI because
(i) Outlets are not clustered in population centers and are often in decentralized industrial areas;and
(ii) Inspection of products cannot be carried out, leading to less quality control of specifications.
6.54 A further variation here is to use another collection method (for example, postal questionnaire) on a regular basis and have a less frequent personal interview to clarify details such as the product range and representativity. For example, some statistical offices visit each respondent on a rotating basis over a five-year period. This also gives the statistical office the chance to “train” the respondent to provide good quality data. This approach can be particularly beneficial if used at the point where the business is initially brought into the sample, since many of the problems can be dealt with in a face-to-face meeting.
D.3.4 Telephone interviews
6.55 Each respondent is called during the collection period and asked for the data required for the survey, with the interviewer validating the form when speaking to the respondent. The data collection staff can be assisted with a set dialogue or through computer-aided telephone interviewing. It is important to provide adequate training to deal with questions that arise during the call. The main advantage of this approach is the data validation during the telephone call, but, again, this is costly in terms of staffing, and there can be difficulty getting through to the respondent to get the information. Telephone interviewing is becoming more difficult with technical developments that allow respondents to answer telephone calls only from selected people (via voicemail). Also, the respondent may not have the data immediately at hand, which could lead to guessing rather than the correct data. The main con cern with this method is that it is likely to lead to bias caused by respondents repeating previous observations—that is, stating that there has been no change.
D.3.5 Internet data provision
6.56 This method of data collection offers great potential in terms of efficiency and economy. Respondents can be given the questionnaire and reminded to respond through this channel. Systems to validate the data in real time are also possible. This is a benefit to the business respondent since it reduces recontact time (although the benefits of a one-to-one dialogue are lost compared with other methods). Since the returned data are in electronic format, it is efficient for further processing by the statistical agency and the response times are quicker than postal-based collection.
6.57 There are, however, a number of issues related to Internet collection. To be effective, a very large proportion of the businesses in the country must have access to the Internet. Also, Internet security is vital, given the commercially sensitive nature of producer prices.
D.3.6 Electronic capture of data from disc
6.58 This method involves the supply of a floppy disc containing an electronic questionnaire. The respondent loads the disc and completes the information before returning the disc to the statistical office. Data are then transferred to the statistical office database. This method allows on-line validation techniques to be built into the questionnaire to save recontact time. However, the procedures for dealing with floppy discs are onerous, and for short surveys such as the PPI (with few data items to be collected), the benefits are limited.
D.3.7 Electronic data trasfer
6.59 This method of collection involves the transfer of data files directly from the establishment’s systems and allows a large volume of data to be collected with a minimum ongoing collection burden to the respondent. The initial setup procedures can be quite burdensome, but the regular collection costs are reduced. The statistical office has to clearly define the data format and information system protocols. It is possible that this type of collection could allow full unit value data to be collected, which can be beneficial for tightly defined and homogeneous products.
D.3.8 E-mail collection
6.60 The use of e-mail is another collection method, which allows the survey form to be delivered and returned electronically. This approach isless efficient than some of the other electronic methods outlined above; however, it could be useful where postal services are less reliable. It is also useful as a reminder technique, since it offers speedy contact with respondents. Again, security is a key issue, and, since E-mail can be less secure than some of the other forms of electronic collection, the legal issues should be examined carefully.
D.3.9 Alternative sources
D.3.9.1 Published sources
6.61 Some data items are available from public sources such as trade publications. Examples include prices for metals that are traded on financial markets, which are reported in the financial press and international journals such as Metal Bulletin. Published sources provide a high-quality source of price data for these products. Their advantage is that they are readily available and relatively inexpensive; they also reduce the respondent burden. Before using published source data, the statistical office must be sure that the source is reliable and that the prices reported are genuinely independent market prices. It is important to verify that the prices are actually based on business transactions of these products. It is also a good idea to become as familiar as possible about the methodology used by the compiling organization.
6.62 A further subset of published sources that is becoming increasingly important is data collection from company websites. Many companies have created extensive websites allowing customers to search by product specification and in some cases allowing customers to set their own product configuration. It is then possible to buy the product directly from the site. This type of website offers tremendous potential for PPI price collection and also for independent validation of prices received through the more conventional channels. There are a few issues to consider, such as the extent to which the Internet prices are list prices instead of transaction prices and whether large buyers attract lower negotiated prices under long-term contract, for example.
6.63 Another important issue is that the Internet-advertised prices may be retail prices. But in some circumstances retail and producer prices are the same. For example, one of the best sectors to use Internet advertising for price collection and validation is in personal computers (PCs). In this case, major manufacturers have set up websites that enable the public and businesses to buy directly from the manufacturer (again, it is important to be aware that bulk discounts could be offered).
6.64 It is worth noting that even if published sources are not used as direct inputs to the compilation of the index, they can provide valuable information for editing, external verification, and the preparation of analyses of the main index movements.
D.3.9.2 Regulatory data sources
6.65 For some products or services it is possible to get data from government regulators. It can be difficult to get access to this information if confidentiality constraints apply, but where it is possible to get this type of data, it can be of very high quality. Telecommunication and rail fare data are two examples of service prices collected in this way in several countries. The potential overlap between retail or producer price can also be an issue when using this type of data, but in many cases there are different tariffs for business and consumer use.
D.4 Field procedures
6.66 The following section outlines practical field practices adopted by many statistical offices
D.4.1 Price discounts
6.67 Producer prices should be transaction prices, not list prices. This means that all discounts should be taken into account. Discounts can be given for a variety of reasons, such as prompt payment, volume of the purchase, competitive price-cutting, and so on. It is important that this be made clear during the survey collection process. Discounts arising from high-volume transactions can cause particular difficulty. Problems occur if the volume sold to the representative customer changes from period to period, which could lead to changes in the discount rate in each period. In such a case the price index would be seen to move simply because of changes in the volume mix, rather than as a result of a pure price change. This type of problem commonly occurs in quarry products, such as road stone and railway ballast. A possible approach in such circumstances is to seek prices for the same specific, typical transaction volume each month.
6.68 A common form of discounting is to provide a larger quantity of the product for the same price, sometimes for a limited period. The product specification should include details on quantity to enable an adjustment to be made to include this type of discount. Retrospective discounts based on sales volumes are an important feature of the manufacturing sector but are difficult to collect with normal survey techniques, so these tend not to be included. An example is the bonus paid to car dealers by manufacturers based on sales volumes, which is separate from the original sales transaction.
6.69 Rebates are a form of discount where the discount is (generally) paid after the purchase and are normally based on the cumulative value of purchases over a specified time—for example, a rebate is given at the end of the year based on the customers total purchases over that year.
6.70 The collection of discounted prices and the identification of discounts are complicated in practice by a number of factors. First, the pricing structure used by the company may be complex and the conditions under which discounts apply may be described in nonstandard terms. Second, differences in pricing and discounting procedures among companies require that data collection be tailored to each company. Third, in a number of areas, the level of discounts and rebates is commercially very sensitive information, and senior company officials may know only the full level of discounts offered to major customers. Taken together, these three factors mean that identifying and keeping track of discounts constitute the major tasks facing PPI compilers.
6.71 Rebates in PPI indices pose major practical problems in that they are often determined by future events—for example, the buyer receives a rebate at the end of the year on the basis of how much he or she purchased during the year. Thus, at the start of the year, although it is known that the buyer will receive a rebate, the precise amount is unknown. The special problem posed by rebates of this sort is that the final price to be paid may not be known until the end of the period concerned, when the total value of purchases will be known and hence the level of rebate can be calculated. This type is often referred to as a retrospective price fall.
6.72 Often a rebate paid to the buyer in the form of a reduction in the cost of the purchase over a year occurs in a particular month. This can lead to reported prices showing dramatic price falls for that particular period. PPI compilers should take care to ensure this does not occur.
D.4.1.2 Treatment of rebates in PPI
6.73 The question arises as to how such rebates should be treated. Should the price paid each month be shown in the index as the price for the item If so, how should the rebate be treated—as a retrospective price reduction? If so, should the previous prices be revised?
6.74 On balance it is considered that where the rebate is already in existence, the rebate should be treated as a discount and deducted from the monthly price and not treated as a retrospective price reduction. The basis for calculating the rebate should be the buyers normal volume of purchases (if the buyer is a new customer, then the basis for calculating the rebate should be the average quantity purchased by that category of buyer).
6.75 Changes in the level of rebates should be reflected only when the actual rebate for the same quantity purchased or sold changes. Changes in the rebates paid to a particular customer for changing the volume of purchases should not be reflected as a price change.
6.76 As price indices are designed to measure price changes for a constant quantum of purchases or sales, the rebate collected should be the rebate applicable to that constant quantity and clearly specified in the pricing basis.
6.77 Where rebates are specified in terms of a monetary value of purchases or sales, it is important to realize that because of inflation a monetary value does not represent a constant real quantum. As a consequence, the monetary value should, if possible, be converted to a quantity. If this is not possible, then the dollar value should be updated each year according to the change in the price of the item concerned.
6.78 If the quantity or value of a respondents purchases or sales changes significantly, the pricing basis should be changed to reflect this. The change in rebate associated with this should not be allowed to affect the index.
6.79 Where a number of levels of rebates are offered, it is necessary to ascertain the importance of each level of rebate and to price those that are significant.
6.80 Caution must be adopted when dealing with retrospective price falls. Revisions to previously published indices can create major problems for users, who use them to negotiate contracts.
D.5 Other variables
D.5.1 Quality or specification changes
6.81 If any variable in the product specification changes, the respondent should be questioned about the change and whether new features have been added. If it is a simple change that has no effect on the price, then the specification should be updated and a marker placed on the product description to indicate that it has been changed. If the quality increases (in terms of producers cost), then this should be reflected in the index as a price fall. If the quality decreases, then this should be reflected as a price rise.
6.82 If the price of the product changes and it is solely or partly because of the change in the specification of the product, then the data collection form returned to the respondent should contain the new price, but for index purposes a quality adjustment should be made to record no price changes for the product on the price database (since it is not an inflationary pressure causing the price to move). If, for example, the price of the product moves 10 percent and the respondent assesses that only 5 percent of the movement was related to pure price change, with the remaining 5 percent owing to a different product with changed specification being quoted (a quality improvement), then the price relative should move by only 5 percent.
D.5.2 Unique products
6.83 A unique product is a product that is only manufactured once to the specification of an individual customer. Within a group of products, each product will be different from the others—for example, industrial furnaces, ships, or an audit contract. In these cases the price cannot be observed over multiple periods. There are a number of approaches to solving this problem, as follows:
(i) Model Pricing: Ask the respondent to provide a notional product with a basic range of characteristics, based on recent orders. For each period the respondent is asked to supply a hypothetical price quote based on this hypothetical product. It is important to update the product specification at regular intervals. Described below are three methods of model selection that can be used. Refer to Chapter 10. for further discussion.
(a) An actual product sold in some recent period, which is representative of the respondents output, can be selected and specified in detail as the model to be priced;
(b) A hypothetical model that is representative of the types of products produced by the respondent can be established. While this model may never have been (or never will be) produced, it must represent an item that could be readily produced; and
(c) A component model can be established. These are used in those cases where no single model can represent the output of the respondent. In such cases a number of models can be selected or a notional model incorporating the key components from the various items produced can be established that is,—incorporating the different types of materials used and different production techniques. In the latter case the model would be purely hypothetical in that it might never be built, but it would nevertheless be representative for measuring price changes.
(ii) Repeat Recent Real Sale: Ask the respondent to provide a price quote for a recent real sale and to provide a hypothetical price for this exact product design for the subsequent months. If the order is not repeated again after a reasonable interval—for example, six to eight months—then a replacement product is sought.
(iii) Specification Pricing: A base model of the product or service is agreed upon with the company, and then in each subsequent month the company supplies the price for each individual part of the model—for example, one hour of an accountants time or a ton of steel and so on. When the data are returned to the office, they are collated using a formula agreed upon with the company to arrive at a price each month.
(iv) Component Pricing: This approach entails collecting prices for a selection of component parts and using them as inputs to produce a final output price. It is important to include the prevailing margin achieved by the producer, and it is also important to have a dialogue with the producer to ensure that the components remain representative and constitute a very high proportion of the total inputs. For example, a recent EU task force report on large capital equipment suggested that this approach would be sufficient if the value added for the assembly of the components did not exceed 10 percent of the total product value.
6.84 These approaches are much more burdensome on respondents, since they cannot simply look at recent sales data to provide price quotes. To accurately supply price quotes using these approaches would lead to the respondent incurring substantial costs, so in practice there is an element of estimation in this process.
6.85 In all the above unique product cases, the main difficulty is persuading the respondents of the value of this approach, because they do not produce this specific product as described. To resolve these issues, a field officer visit may be necessary. It is important to include base-level products such as these in the PPI since they are often associated with high-value goods, which would not be included otherwise. It is all too easy to select a related, but simple, product—for example, in shipbuilding to ignore large, unique ships and to concentrate on only small, regularly produced products such as dinghies.
6.86 A further approach to solving the unique product problem could be to build a hedonic model. This would enable the unique model to be valued from its characteristic set. This method is more normally associated with quality adjustment but could be extended where sufficient data are available. See Chapter 7 and 12 on quality adjustment for more details on this approach.
D.5.3 Unit values
6.87 In some circumstances it is possible to use unit value indices to overcome the problems of unique products, but this is recommended only for products with a very narrowly defined product group—for example, for road stone. See Section B.2 above for a discussion of the limitations of unit value indices.
D.5.4 Transfer prices
6.88 Transfer prices are defined in the 1993 SNA (paragraph 3.79) as
affiliated enterprise may set the prices of transactions among themselves artificially high or low in order to effect an unspecified income payment or capital transfer.
Transfer prices should be used with caution because they often do not fully reflect the true value of the goods or services being transacted. It is important to aim to collect market prices, or real transaction prices. The best way to get real transaction prices is to ensure that the price recorded is to a third party, not to another part of the same business. In some circumstances it might not be possible to get real market prices without losing too many price quotes to produce a representative index. If the only price available is to another part of the same business, which is potentially but not necessarily a transfer price, careful attention must be paid to market price movements for similar products to ensure that the interenterprise sales reflect market conditions. It is important to avoid recording stable prices for these transactions over the long term if market prices are changing. Obtaining good weight information on the value of output or sales will also be difficult because the revenue weights should reflect market prices.
D.5.5 Sampling issues
6.89 The objective of price indices is to measure pure price change over time—that is, to measure the extent to which the cost of an identical basket of products changes over time, not affected by changes in quality or quantity or the terms of sale. This is often referred to as pricing to constant quality; it is not a simple objective to achieve because the characteristics of products being sold in the marketplace, including their terms of sale, change over time. Frequently, the precise commodity priced in one period is no longer available in the next period either because there has been some change in the characteristics of the commodity or something new has taken its place.
6.90 Sampling issues, such as sample loss relevance, avoidance of making quality adjustments, selecting noncomparable replacements, and inadequate matching procedures, greatly affect the representativeness of the sample over time; ignoring such issues could bias the index. Refer to Chapter 7 for further detail.
6.91 PPI compilers must devise techniques to minimize differences and eliminate their effect on the index.
D.5.6 Price discrimination
6.92 Price discrimination refers to the situation where the same product sells at different prices in different markets. An example would be the same grade of wheat sold at different prices in different markets, or a different volume of wheat sold at the same price to different markets. When such price discrimination occurs, the average price of wheat can change over time because of changes in the proportions of wheat sold to each market.
6.93 How should such price changes be reflected in the PPI? The answer will depend on the reason for the price discrimination and requires an examination of the various forms of price discrimination. There are four main forms of price discrimination (listed below). All four factors (or any combination) could apply to a particular specification.
6.94 In cases of price discrimination, it is extremely difficult to determine the price change when the destination of the purchaser changes. In these instances, PPI compilers should specify only the purchasers of product and the quality or conditions of sale differences between purchasers.
D.5.6.1 Differences in selling terms and provision of credit
6.95 The conditions under which goods are sold often vary between markets (buyers). For instance, prices may be lower in one market because the goods are paid for on delivery, while prices in another market may be higher, reflecting the fact that goods are sold on credit. In these situations it seeming reasonable to argue that identical goods are not being sold in each market. What is in fact being sold in the second market is a mixture of the good and credit. It follows that, in these cases, shifts in destination should not be reflected as price changes.
D.5.6.2 Differences owing to timing of contracts
6.96 Where goods are sold on a long-term contract basis, price differences may arise between different markets simply because of differences in the period when the contracts in respect of these markets were signed. In these cases it seems clear that changes in prices owing to changes in destination should be reflected in the index. Failure to do so would run the risk of missing out on long-term price changes for products where the destination is changing over time.
D.5.6.3 Competitive Presurres
6.97 In some markets, goods may have to be sold at lower prices because of competition from other countries (for example, dumping of EU agricultural products), while in other markets producers may be able to achieve higher prices because of the absence of such competition. In these cases, shifts between markets represent pure price changes and should be reflected as such.
D.5.6.4 Hidden quality differences
6.98 For some items, such as tinplate, respondents supply prices only for broadly specified products. In these cases, destination may serve as a defacto quality specification—for example, the quality of tinplate shipped to destination A is different from that shipped to B. In this situation, changes in destination should not be reflected as price changes.
D.6Field officer visits
6.99 Field officer visits serve two broad purposes. First, field officers are often used in the initialization or recruitment process to identify representative products from within the respondents product range and to discuss the exact reporting requirements for the PPI. (This approach is used by the United States, Australia, and France; the French use qualified engineering staff to visit companies.) In some regions (for example, in Europe, where adetailed survey on products by industry of origin, PRODCOM, is conducted by member states) it isknown that a group of products are manufacturedby a company in a particular sector. In such a case, there are two options when selecting products: either let the business pick the most representativeproduct (that is, the one that accounts for the largestpercentage of the respondents turnover for theclass of product), or let the field officer select theproduct with the respondent. Each approach has advantages and disadvantages. For example, only the respondents know the best or easiest product forthem to supply data for; however, it is also important to get a price quote for several products that are representative of the respondents range of products. This may require collaboration between the respondent and the field officer. For more information on product selection, see Chapter 5.
6.100 The second main purpose of the field officer visit is to assist respondents with problems incompleting returns—for example, in the case ofunique products or late responses. The field officercontacts and visits the company to understand its specific concerns and problems in completing the form and to work with the company to overcomethem. This is largely a reactive activity used tosolve problems, but another alternative is to have arolling program of visits so that each respondent (or key respondent) is visited over a set period. Thishelps to keep the respondent educated and ensuresthat problems do not linger unnoticed for longperiods.
D.7 Industry specialists
6.101 The role of the industry specialist is similar to that of the field officer, but the industry specialistconcentrates on a narrow range of industries. Sinceprices are required from very specialized industries such as chemicals and semiconductors, it is difficult for the statistical office to ensure the quality of datareturns and have a meaningful dialogue with respondents unless the organization includes analystswith a more detailed knowledge of these complex industries. A small team becomes experts in a certain field—for example, computers. The teams are fully up to speed on changes in the market, their respondents activities within the market, and specific problems relating to completion of survey data. These experts then analyze returns in line with theindustry intelligence and support respondents whenthey provide data.
D.8 Delinquency follow-up
6.102 It is important to achieve high responserates; to achieve this, procedures should be established to follow up with nonresponders. Problemswith maintaining adequate response rates will occurin price surveys that do not have delinquency follow-up procedures, even when the surveys havestatutory penalties.
6.103 Delinquency follow-up can be done usingany of the data collection methods outlined above A reminder telephone call is an effective technique, since this enables the respondents to discuss any difficulties they have with the survey at the same time, and it is often possible to take data over the telephone (although a price taken over the telephone should be marked for later verification). This technique has the advantage of generating quick results, but it does require the statistical agency to maintain an up-to-date list of contacts and their telephone numbers. Given the labor-intensive nature of reminder telephone calls, it is possible to target these calls on key responders, which are usually the responders with the largest weights.
6.104 A follow-up letter is often effective, particularly if the country has a legal penalty for nonresponse. In such a case, it is possible to make thewording of the follow-up letter stronger, with moreemphasis on the legal penalty. It is usual to follow aset procedure including the use of a recorded delivery letter if formal legal proceedings are to be conducted for nonresponse.
E. Respondent Relations
6.105 Respondents are very important to statistical offices, because without them there are no data. Therefore, developing good relations with and gaining the trust of data providers is an integral part of producing good estimates.
E.1 Dealing with refusals
6.106 On occasion, you will come across a respondent who states, “I’ve had enough” or “I'm not doing it anymore.” In general, providing respondents with relevant information related to their concern will ensure they will continue to provide data. In the examples above, refusals could reflect issues with confidentiality, lack of importance, and too many forms.
6.107 Possible ways of dealing with these concerns are as follows:
(i) Confidentiality: When dealing with issues of confidentiality, a statistical agency that is independent of other government agencies has an advantage over those that are not. The IMF (2003) provided a framework for assessing the data quality of the PPI. This framework identified the importance of independence and confidentiality to ensure trust of respondents. Reassurance that the data will not be released to any other agency or person is much easier with legislation backing the statistical office.
(ii) Objective of the Collection: Reassure the provider that the prices are aggregated into an index that is published monthly or quarterly, and inform the provider of the importance of contributing and the use of the statistics produced.
(iii) Respondent Workload: Talk with the respondent to ensure that the data are easy to obtain and that the current specifications are still relevant; see if the provider can get rotated out of the sample if the company has been contributing data for many years.
E.3 Reducing respondent workload
6.108 Apart from the ongoing use of tailored forms, which significantly simplify the collection process for respondents, PPI compilers can actively reduce provider workload by
(i) Identifying commercially available data that meet the methodological requirements of price indices and using these as a substitute for data collected from respondents. The cost incurred in purchasing these data is compared with expected collection costs, with the benefit of reducing provider load taken into account; and
(ii) Identifying administrative data sources of prices that meet the methodological requirements of price indices and using these as a substitute for data collected from respondents.
F.1 Verification and validation of prices
6.109 Verification aims to identify potentially incorrect prices as early in the process as possible, consult with the respondent, and amend the data if necessary. Three key checks are required:
Data reported were accurately entered into the processing system,
All requested data were provided, and
Data reported were valid (outlier detection).
6.110 This section covers only the first two bulletpoints above—that is, simple data checks at thepoint that data enter into the PPI system. Validation assesses whether the data returned by respondents are credible in relation to other data for the same industry or commodity, and the treatment of data that are not credible are covered in Chapter 9.
F.2 Verification tolerance
6.111 The first stage in the verification process is to determine that the data entered into the system for further processing are an accurate reflection of the data returned. This can be achieved through either a manual audit or an automated system. These checks should determine whether
(i) All data fields required have been completed,
(ii) The data entered in the database agree with those reported, and
(iii) All data fields are completed within an expected parameter range.
When the data have been accurately recorded by the statistical office but basic data checks are not passed, the analyst will need to contact the respondent to verify the information or to get the correct data. Returned prices may be compared with those received for the previous period. If the price change is outside a specified range, then the price should be marked for further investigation. Respondents providing dubious prices can then be contacted to check that the large change is correct and to provide a reason for the large change. Large price changes fall into two main categories: those that are erroneous and those that are correct but genuinely unusual. The second category is more difficult to deal with because they could be outliers, which might result in the need for special treatment within the estimation procedure. Outlier treatment is discussed in Chapter 9.
F.3 Setting tolerances
6.112 The tolerances for data verification checks should be set so that any changes outside the boundaries of expectation are flagged for the data reviewer.
6.113 Tolerances may need to be set independently for each product group. For products that have volatile prices, such as oil or seasonal items, it may be appropriate to have quite wide verification tolerances. Other products may have more stable prices, and so narrower tolerances would be more appropriate. To set verification tolerances for a particular product, price changes over a period of time, say two or more years, need to be analyzed. The range of price changes can then be considered, and the top and bottom 10 percent, for example, can be used to set the tolerances for verification.
6.114 In addition to checking for large pricemovements, another check looks for prices thathave not moved for a considerable period. Mostcompanies will review their prices on a regular basis, often annually. If a companie’s price has notmoved for 15 months, for example, they may be returning the same price out of habit. In these casesthe company should be contacted to see if the trueprice is being returned. In periods of low inflation, the periods of stability between price changes mightget longer and the number of companies not reporting price changes for considerable periods could increase.
G. Related Price Issues
G.1 Lagged prices
6.115 In some cases it is not possible to get prices in time for the current-periods compilation. To ensure that the PPI is published in a timely manner, the previous available price or lagged price can be used. These lagged prices often come from administrative sources, and the delays result from the time required to collate data by the external supplier. Examples include financial intermediation service charges and insurance premiums.
G.2 Seasonal products
6.116 Some products are available for only part of the year—for example, items associated with religious festivals and certain fresh fruits or vegetables. The practice commonly adopted in such cases is to carry the last reported price forward, until the next seasons trading starts and a new price can be collected. This procedure tends to dampen the index movements when the product is out of season and causes upward and downward movements in the product index when the product is back in season. One solution is to impute the missing prices based on the short-term movements of prices for similar products. An alternative solution is to have variable weights for each period, so that the product has a zero weight when not in season. The disadvantage of this is that it makes index point effect analysis less straightforward, since care has to be taken to ensure that the correct weights are applied in each month. See Chapter 10 and 22 on seasonal products and Chapter 7 and 9 on imputation techniques for more details.