Taking account of hyperinflation

6.23 Special arrangements may need to be put into place where there is hyperinflation. In these circumstances it becomes even more important that the prices of individual items in individual shops are collected at precisely the same time each month, otherwise misleading figures may result. Consideration should be given to the more frequent collection of prices and correspondingly a more frequent compilation of the index. Where prices are normally collected on a quarterly basis, it may be sensible to collect them more frequently. If this is not feasible, it may be appropriate to up-rate prices proportionately by some relevant indicator to provide an approximation to a monthly index. If this is done, however, great care needs to be taken in choosing the appropriate comparator, particularly as relativities between prices can change dramatically in periods of hyperinflation.

6.24 In some circumstances, rapid or frequent price changes may be associated with certain items only and action should be taken accordingly. By way of example, food prices may rise disproportionately because of a bad harvest and it may be sensible to increase the frequency of the index for food items only. Alternatively, a simpler way of dealing with this situation may be to monitor a small number of relevant prices on a regular basis without producing a full price index. Such sub-indices could be published separately or used to up-rate the later prices collected in the period, as mentioned above. These items may be chosen according to their importance for the family budget and whether they are particularly susceptible to big price increases.

Price collection techniques

6.43 For many items, prices will be collected locally by price collection agencies employed by the national statistical institute, or their own employees, visiting retail outlets and recording current prices for an agreed selection of items. But some prices may be collected centrally from catalogues, by retailers providing list prices covering a range of outlets, by telephone, fax, letter, emails or from Internet sites. All these methods may be cost-effective or necessary to represent different aspects of consumer purchasing behaviour and so it is not surprising that many statistical offices use a variety of data collection techniques. In addition, such price collections can allow for the implementation of specific methodological procedures by head office staff (for example, quality changes). Either local collection agents or head office staff can use these varying collection methods. Examples of price collection techniques include the following:

• Prices may be obtained from mail catalogues to represent a certain type of retail outlet, or where high street catalogue stores have nationwide coverage with uniform pricing policies. Increasingly in some countries mail order suppliers are offering their own Internet services. In the case of both mail order and Internet shopping, care has to be taken to treat delivery prices and sales taxes consistently and correctly.

• Prices may be obtained over the Internet either for convenience (where major stores offer the same prices on the Internet as in the shops) or through necessity in order to maintain a representative sample where this type of retail outlet is increasingly used (for example, for books).

• Some retailers have national pricing policies with no individual pricing discretion, even for sales and special offers. In these cases, a single store can be visited or the retailer’s head office may agree to supply a single price list (covering all items or prices for the specific selected items).

• Prices may be obtained over the telephone or by fax where there will be no ambiguity in price because the item being priced is standard and the contractor will quote a standard charge (for example, electricians may be telephoned for charges for providing a new single electricity socket). In addition, obtaining a price over the telephone will reflect what the consumer will often do in practice. A further factor is that many service providers (such as plumbers or window cleaners) tend not to work from retail outlets and it would be difficult to visit due to their variable working hours off-site at the customer’s own premises.

• Prices may be obtained by letter, fax or email, accompanied by relevant head office forms for completion and return in cases where central office collection is deemed more efficient or where local price collection is not possible (for example, tariff prices). Examples include prices collected from a sample of local authorities, insurance companies, public utilities, and telephone companies.

• Prices may be obtained from other government agencies or regulatory authorities, which can act as intermediaries in the price collection process. In some countries, for example, this would be the case for electricity prices.

• In some cases, secondary sources can provide data on specific goods. Two examples, taken from the United States CPI but by no means unique to that country, are airline fares and used vehicles. A sample of scheduled airline flights is selected using detailed ticket data from the United States Department of Transportation. Monthly pricing is then carried out by online reference to a private-sector computerized fare database widely used by travel agents and others. In the case of used cars and trucks, both sampling and pricing employ published data from a dealer trade association. The benefits of using secondary data may include larger sample sizes, faster or less expensive access to data, or the avoidance of particularly difficult collection problems.

View Full Size

Price collection procedures

• Download Figure
• Download figure as PowerPoint slide

6.44 When using other sources such as catalogues or the Internet for prices, special care must be taken to ensure that they are correctly recorded with or without sales taxes, or with or without delivery charges. In these cases, procedures should include a check that the prices are relevant for the index period.

6.45 It is important to remember that all the usual price collection principles and quality assurance concerns remain relevant for prices collected from the Internet (including the need for detailed descriptions, immediate availability of the item for purchase, treatment of special offers, and possibility of substituting comparable or new items).

6.46 Where prices are taken over the telephone it is recommended that the retailer be visited occasionally, where practicable, to maintain personal contact and response rates and to ensure that no misunderstandings of items or pricing are occurring. As far as possible, prices collected by telephone should also be confirmed in writing to provide confirmation for quality assurance procedures (see Chapter 12).

6.47 Many households may be unable to access the Internet, and Internet shopping provides additional services, such as home delivery. This means that collecting prices from the Internet can be considered either as the introduction of a new outlet or a new item. In both instances action should be taken as part of the procedure for maintaining a representative sample at the time of the regular updating of the item and location selections, usually at the time of chain linking. It should be noted that consideration will also need to be given to whether a move to Internet shopping involves a quality change. For example, in the case of food shopping, free delivery may be included for payment over a certain level, or the average “use by” date may differ from that found in traditional outlets.

6.48 The scope for improving the efficiency of data collection can increase with the arrival of technological advances in the marketplace. New collection methods are continually becoming available, particularly in the technologically advanced countries. Methods for future collection include touch-tone dialling facilities and scanner data, both of which have the advantage of offering new ways for businesses to reduce the burden or inconvenience of supplying data.

6.49 It should be remembered that to keep the index representative, it might be appropriate to collect prices for an item in more than one way. For example, people may buy books from catalogues, from a variety of shops (bookshops, newsagents, supermarkets, department stores, and so on) and through the Internet. In these circumstances, it is appropriate to collect prices through all types of outlet where transactions are significant.

Questionnaire design

6.50 Good design of the questionnaire form (or its electronic equivalent) is essential for the successful collection of prices. Not only is it important that the price collectors find it easy to use, but the format and layout should facilitate the extraction of data (price, item description, comments, and so on) by head office for effective quality assurance.

6.51 The first step in designing a questionnaire is to define the information that needs to be gathered and how it will be collected. Different forms will be appropriate for each of the collection methods that are deployed, for instance visiting retailers compared with collecting by post. There will, however, be a number of common principles. The questionnaire should be practical for the price collector to use in the field and should also facilitate basic quality assurance. It is for the latter reason that it has been argued that the price recorded last time the item was surveyed should be shown on the questionnaire, as this will prompt the collector to ask questions if the previous price is very different from the current price. The converse to this argument is that the recording of the last price may mistakenly lead price collectors to identify the item to be priced by reference to price rather than item description, or at the extreme to estimate the price or repeat the previous price without actually visiting the shop.

6.52 It should be remembered that, at the time an index is chained, the questionnaire will need to list all items included in the old as well as the new basket. For example, an index chained annually on January prices requires both the old sample of locations and items and the new sample of locations and items for the base month.

6.53 An example of a price collection form is given in Appendix 6.1. This example is of a form used by the collector for recording prices when visiting an outlet and could be either a paper or electronic version. It is also possible to ask the shopkeepers concerned to complete the form themselves and to send it to the national statistical institute. Such a form may therefore serve for reporting as well as for collecting. If the form has space for recording prices over a whole sequence of months, the collector may keep the form and transcribe the prices from it each month onto a separate report form, which is sent to the national statistical institute. Where the form used for collection is also used for reporting, there are two main possibilities: either the form has space for recording prices over a whole sequence of months, and the form is shuttled backwards and forwards monthly between the collector and the office; or new forms for collection and reporting are printed out by the computer each month. In the latter case, if considered desirable, the form may contain the prices recorded in the previous month alongside the spaces for recording the current month’s prices. It should be noted that the transfer of the prices to another form or system, whether done by computer or manually, may lead to transcription errors.

6.54 Increasingly, the use of an electronic version of the questionnaire on a hand-held computer or “personal assistant”, with built-in validation checks, is seen as advantageous for local price collection by price collectors. The data can then be transferred electronically from collector to head office via a variety of intermediate steps for further validation checks by the price collection agency.

6.55 It is recommended that price collectors be required to provide full descriptions of the items being priced. This enables checks to be put in place to ensure that collectors are properly following instructions, particularly on the selection of items to be priced. It also ensures that any changes, including changes in the quality of the items, are being properly identified, with enough detailed information to enable decisions to be taken on quality adjustment. Price collectors should be given a checklist or set of codes to record relevant information on changes relating to outlets, items or prices. The information needs to be systematically collected. For instance, codes to help with quality adjustment need to reflect those characteristics that most influence price. Prior research, for example, based on the hedonic method, can help to predetermine these (see Chapters 7 and 21).

6.56 Codes for managing the sample of outlets may include:

• –closed down: outlet permanently shut or closed down;

• –temporarily unavailable: outlet temporarily closed, but likely to be open next month;

• –refusal: owner or staff refuses to cooperate;

• –change of details: change of ownership or name, or change of purpose.

6.57 Continuity is one of the most important principles of price collection. As the index measures price changes, it is vital that the same item is priced every month in order to establish a true picture of price changes. So if, for example, a jar of a supermarket’s own brand strawberry jam has been selected, that particular brand and flavour should continue to be collected; if it is out of stock, another brand and flavour should not be used without further investigation to establish whether this is a temporary situation or likely to be permanent. In the latter case, and if another flavour of the same brand, size and quality is available, then this item should in normal circumstances be chosen as a “comparable” item and the item description suitably amended. If a different brand, size or quality product is available then this should be selected as a “new” item, but only where no comparable items are available. The same principles apply to other items, such as clothes, and fresh fruit and vegetables. With clothes, it may be important that colour, fabric, country of origin, logos and size are specified to ensure that the same item is priced each month. For fresh fruit and vegetables, useful attributes to record may be “country of origin”, “class” and variety. For electrical equipment, it may be the specifications and features given in the manufacturer’s catalogue that are important.

6.58 It is not possible to be prescriptive because the concept of equivalence will vary between different countries; but for practical purposes it is important that a detailed description of the items being priced is recorded. Item descriptions will assist the price collector and head office in choosing or confirming the suitability of a replacement for an item that has been withdrawn and will also help identify changes in quality. The focus should be on recording price-determining characteristics.

6.59 Should the regular price collector, for whatever reason, be unable to carry out the normal collection, full and accurate descriptions will enable a relief collector to carry out the collection without any doubt as to the correct items.

6.60 Most of the time, the item will be exactly as collected the previous month and all that will be recorded is a new price. However, should there be a change or uncertainty in the item, then it will be necessary for price collectors to use their own judgement and to inform head office, bearing in mind that head office staff are responsible for making the final decision. A pre-coded specification will be less time-consuming and will provide better guidance to the price collector on what information should be reported. The codes might include:

Comparable (C): The original item is no longer stocked but a similar alternative has been collected that does not differ in terms of major attributes. The price is likely to be in a similar range although this may not always be the case.

New (N): The item has been replaced by something new that is not really comparable but is equally representative of that commodity group. If possible the collector should try to find out the price of the “new” item in the chain link or base period.

Sale or special offer (S): A price decrease because of a genuine sale or special offer, with a sale or discount sticker present. This does not include damaged or out of date stock or clearance goods. The latter should never be included. A price reduction where there is no notice of a sale or special offer is not a “sale”; the item should still be priced, but without the S indicator code.

Recovery (R): A return to the normal selling price, for example after a sale or special offer. This need not be a return to the same price as before the sale or special offer.

Temporarily out of stock (T): Guidance will need to be given to the price collector concerning the meaning of “temporarily” (in terms of expected duration, which may vary for different items). It may be advisable to replace items immediately (for example, fashion clothing, if it is unlikely that the identical item will come back into stock). Typically, T indicators should not be used for more than two consecutive months—in the third month, a replacement should be chosen. In food outlets, it is very unusual for items to go permanently out of stock. The collector should always try to check future availability with the retailer.

Missing (M): Used where the outlet has never stocked or no longer intends to stock an item and there is no appropriate alternative item. In these circumstances it is recommended that the item is checked at subsequent collections to ensure that a suitable replacement item has not come into stock.

Weight (W): A permanent weight or quantity change to the product.

Query (Q): Such a code may be used to supply extra retail information to head office (for example, “10 per cent extra free”, “3 for the price of 2”, or a strange price difference that is not covered by one of the other indicators, such as a bumper issue of a magazine at an increased price). Arrangements need to be in place for head office to respond to these comments and to treat the price quotes accordingly.

6.61 The use of these codes is illustrated in Appendix 6.1. Even if the retailer says there have been no price changes since the previous month, the price collector should confirm prices anyway. This will require some diplomacy, but it is important because it is easy for the shopkeeper to overlook a small number of price increases, forget when the last increase occurred or even deliberately mislead the price collectors. The use of codes is important for operational reasons. For example, if an item is unlikely to remain available the next month, then a substitute can be selected in advance and an overlap price collected.

6.62 As a general rule, a price should be recorded only if the exact product being priced is on display and immediately available for sale. No price should be recorded if a product is temporarily out of stock. For certain large items such as furniture, however, where the item must normally be ordered, the price should be recorded as long as the retailer confirms that it is available for delivery within an “acceptable” time period.

6.63 Some food items, such as meat, fish and cheese, can be sold in variable weights, so it is sensible to collect prices per unit of weight. This should be taken from the package labelling or calculated directly by the collector. Roughly the same package size and type should be used each month, as the unit price might be lower for larger pack sizes or differ between package types. Other items, such as eggs, are often sold in specified quantities. For these, it is essential that collectors record prices for the specified quantity, as total and unit prices usually depend on the number bought. If X eggs are to be priced and the price for the number is not quoted directly, then the price of one egg can be obtained and multiplied by X to get the required price. Care does need to be taken, however, to ensure that unit price does not decrease with quantity. Another example is mint. This herb is often sold in bunches of variable size, so a number of bunches should be weighed and priced to obtain a price per kilogram.

6.64 Certain food items, such as fruit or vegetables, are more difficult to price as some outlets might price items per number purchased while others might price by weight. For example, peppers may be priced by weight or by unit no matter what the size. Garlic may be priced per bulb, clove or by weight. Various types of berries may be priced by weight or by punnet, which may differ in size or how full they are. In these instances, care has to be taken with the product descriptions. Collectors need to be aware of the importance of collecting the same thing from one month to another, so that genuine price changes are recorded and not quantity or quality price changes.

6.65 The use of hand-held computers for local price collection provides more scope for quality assurance both in the field and at head office, without some of the disadvantages associated with paper forms. Price collection using hand-held computers is discussed in more detail below. Use of electronic forms on floppy disks or by email, for example for central price collection from the head offices of large retail chains, may be more cost-effective than sending price collectors into individual outlets. But in these circumstances, care needs to be taken to check that there are no price variations between different outlets in a chain and that any special offers given locally are covered. Where there are such local factors, adequate account will need to be taken of them, otherwise the price recorded for the index may be misleading.

6.66 A decision needs to be taken over whether large retail chains should be placed in separate strata (treating the chain rather than an individual outlet as the sampling unit) or whether a sample of outlets from each chain is taken (thus taking as the sampling unit an outlet from a particular chain). As a general rule, a retail chain with no national pricing policy cannot be treated as a single sampling unit, but it may be possible to visit only a few of their outlets if it can be established that each outlet visited reflects the chain’s prices over a wide area. In these situations, it is usual to approach the management of the chain’s head office to confirm their pricing policies and obtain permission for the collection. Each year, when approached again for permission to continue the price collection, the management should be asked to confirm that their regional pricing policy remains unchanged. The prices collected are then given a weight to reflect the market share they represent, in the same fashion as the weights applied to prices collected centrally for a chain where there are no price variations between outlets. Issues relating to central collection of prices from local businesses and outlets (for instance, over the telephone by staff located in the national statistical institute) and the collection of prices for retail chains from their corresponding head office are discussed further below.

Field procedures

6.67 Adequate field procedures are required to ensure that the quality of the price index is not compromised through errors in price collection. Price collection needs to be carefully planned and managed, and effective instructions and training given to price collectors. Most prices are likely to be collected through price collectors visiting individual outlets. Guidance on the organization and management of field procedures relating to local price collection is given in Chapter 12.

6.68 In some circumstances it may be more efficient for prices to be obtained from one source rather than through surveys in the field. These are covered in the section that follows.

Central and head office collection

6.69 One form of central and head office price collection is where price data representing a number of shops are collected from one single source. This can take place when chains of shops have proven national pricing policies, with no local variations between stores in terms of either price normally paid or special offers and discounts. In these cases, the chains* outlets should be excluded from local price collection and the prices collected should be weighted according to the market share of sales.

6.70 The selection of this type of central collection and calculation is usually dependent on one or more of the following considerations: national or local pricing policies; available sources of data (including willingness of chains to assist in this way and forward commitment by them to provide data centrally); data presentation and format (advertised prices or average transaction prices provided by email, on floppy disk or paper); reference point of available data (price lists match the collection day or period); and frequency of price changes.

6.71 Central price collection may also be appropriate for some service prices. These could include:

• –fees set by a professional or trade association or union;

• –charges for public utilities or services provided by deregulated (and regulated) bodies or government (such as: water, gas and electricity tariffs; bus and train fares; birth, marriage and death registration fees);

• –prices centrally determined by government (for example, fees to be paid for services, such as health care and education, that may be partially or fully funded by government);

• –taxes and licence fees paid to government (for example, television licences and vehicle excise duties).

In some instances, data may need to be requested from regional authorities, for example where there are regional utility providers.

6.72 Data may be requested in writing, or by telephone or electronically. Where letters are sent, consideration should be given to using office automation for the generation of data requests (for example, mail merge facilities), logging responses, monitoring progress and sending reminders to non-respondents. Useful categories for informing progress might include: return received; return being checked; query sent and awaiting resolution of query; figures finalized.

6.73 The greatest gains from electronic reporting of centrally collected prices are likely to be efficiency gains from automation, better work monitoring and fewer problems arising from transcription errors. The risk—one that is associated with all central price collection—is that the impact of an undetected error can be compounded because of the relatively large weight that may be placed on one price or set of prices. Clearly, this factor should be reflected in the quality assurance procedures as well as in the sampling procedures. It has been observed that national statistical institutes can be slow in reviewing their quality assurance procedures following a move to greater central price collecting. This can lead to a disproportionate amount of effort at head office being focused on the checking of local prices. This is particularly so if local prices have been rigorously scrutinized in the field; any individual error will not have a noticeable impact on the index unless it is part of a systematic bias, for example arising from inadequate instructions to collectors.

6.74 Providers of goods and services may send either a full price list or a tariff from which an appropriate sample of prices and weights can be extracted or just those prices required for compilation of the index. In some instances, for example a regional transport authority, it may be acceptable for data to be provided in the form of a price index. In these cases it is clearly important to ensure that the index has been calculated accurately and in accordance with the requirements of the consumer price index, using agreed methodology, and that the central office exercises strict quality control. The latter may be done by, for example, checking the computation once a year or more frequently against the basic data or by setting up automated systems to detect abnormal changes. Agreements on the methodology of the computation should include such things as item selection, weighting of components and timing of collection, as well as the mathematical construction of the index. The index should also be made available to the central office with supplementary briefing and explanations for price movements. Any potential problems, such as the need to resample where previously quoted items are no longer available, should be discussed with the national statistical institute in advance. A continuous quality control may take the form of a reconciliation analysis against other related data (including announced price changes) and identifying outliers when compared to previous values of the index. Data and prices published by another organization or government body may provide a useful comparator. Where prices are taken over the telephone, it is highly recommended that all price quotations are subsequently confirmed in writing to ensure that any queries may be resolved and that an audit trail is maintained for future months in case of subsequent discrepancies that cannot be resolved.

6.75 In all instances it is important to check on a regular basis that the item or service being provided has not changed in any respect, because if this is the case a quality adjustment may need to be applied. In the case of supermarkets and other large suppliers of data, confirmation should be sought from head office that code numbers have not changed to ensure that the items being priced do not change unexpectedly between one price collection period and the next.

6.76 As stated previously, the frequency of collection depends on both the range of prices being monitored and when the prices are known or expected to change. For instance, bus and rail fares may change once a year on a prespecified date. In other instances, prices may change throughout the year as different providers review their pricing structures, but the expectation may be that prices will show little volatility. For example, it may be necessary only to contact health insurance companies on a quarterly basis or local authorities for prices of school dinners only at the start of each term. Decisions on these issues will need to be based on knowledge of local circumstances, with satisfactory procedures in place to detect any change in procedures.

6.77 The number of price quotes required at each collection will depend on individual circumstances and will need to take account of the weights and homogeneity of the index as well as the underlying volatility of prices (see Chapter 5). It is also best to avoid situations where a handful of price quotes from, say, an individual retail chain represents a large weight in the index. The number of prices collected centrally should where possible reflect the importance of that item in the shopping basket and the range and volatility of the prices.

6.78 All the data collection principles above are to be followed for all central and head office price collection, regardless of whether these forms of collection have been introduced for reasons of practicality, cost-effectiveness, or special methodological concerns.

6.79 Further examples of items that may be collected centrally include: some aspects of transport, such as tolls for bridges; situations where there may be a variety of different outlets but where there is uniform pricing for all consumers; and instances where the data requirements for quality adjustment are better met by exploiting a single data source. By way of elaboration, if none of the towns or cities selected for local price collections have tolls on roads, bridges or tunnels, then these could be unintentionally excluded from the index; but by selecting a sample of these across the country - with prices collected centrally - the index remains representative of these types of expenditure. Similarly, if the prices of goods and services are the same all over the country, regardless from whom they are bought (for example, newspapers and magazines), then these prices are most cost-effectively collected centrally. The more complex methodological calculations of prices, including quality adjustments, may also be best collected centrally. Examples of these include some housing costs, and computers and cars (where information on technical specifications at the level of detail required for quality adjustment may not be available from shopkeepers).

Price reductions

6.80 One of the principles relating to consumer price indices, which is applied with few exceptions (such as owner-occupier housing costs), is that only transaction prices, that is prices actually paid by individuals or households, should be included in the index. This may differ from the advertised price if, for example, a discount is offered. In practice, however, discriminatory discounts, which are available only to a restricted group of households (as opposed to non-discriminatory discounts that are available to all), are generally excluded on principle. For example, money-off coupons and loyalty rewards for previous expenditure are normally ignored and the non-discounted price is recorded. Also, it may be difficult to obtain the price paid if this is subject to individual bargaining. It may therefore not come as a surprise that, while the general rule above may appear simple, there are a number of instances requiring special treatment either because of conceptual issues or because of practical difficulties. The following guidelines reflect practices followed by a number of countries. They do not represent a set of rules because the appropriate practice to be followed will be determined by individual circumstances, which might vary between different countries.

6.81 Discounted prices should only be taken if generally available to anyone with no conditions attached; otherwise the non-discounted or unsubsidized price is recorded. In particular, the general practice is to ignore money-off coupons and loyalty rewards. A judgement needs to be made, however, relating to the interpretation of “generally available”. For instance, reduced prices for payment by direct debit may be taken into account depending on the extent to which consumers as a whole have access to and use such a service. A judgement is required in the latter case on the threshold to be set for access, above which action is taken for inclusion in the index. Alternatively, different payment methods may all be priced individually (for example, separate data collection for electricity payments by cash, direct debit and pre-payment) and weighted together to form a single price index for that item.

6.82 Price discrimination. Discounts available only to a restricted group of households should be disregarded because they are discriminatory, unless they are significant and are available either to the vast majority of the population or to identifiable subgroups who qualify for such discounts on the basis of demographic or other characteristics not requiring action by the individuals concerned at the time of purchase. In the latter case, they should be treated as stratification or coverage issues in item sampling. Some judgement is required. Examples of allowable price discrimination may include lower prices offered to pensioners (for example, discounted travel or haircuts) and discounts for people who receive state benefits. Another example of a case where prices are not universally available to all, and where judgement is required, is where a nominal or token membership fee is required by the retail outlet. In these cases, the take-up of such membership—which is widely available to all—needs to be considered in terms of thresholds and general spending patterns of the consumers and the conditions placed on membership which may make the latter restrictive (for example, minimum levels of purchase). Ease of access to the outlets in question may be a relevant factor as well, say, if in practice the customers need to have the use of a private car.

6.83 Sale or special offer prices should be recorded if these are either temporary reductions on goods that are likely to be available again at normal prices, or are stock-clearing sales (such as January sales or summer sales). Before designating a price as a “sale” price, however, special care should be taken to ascertain that there is a genuine sale with price reductions on normal stock. On occasion, stock is continually sold below the recommended retail price or advertised as a special offer even though these prices are available all year. In such cases, prices should not be considered as sale prices, but can still be collected. Special purchases of end-of-range, damaged, shop-soiled or defective goods should not normally be priced, as they are likely not to be the same quality as, or comparable with, goods previously priced and are unlikely to be available in future. If the special offer is limited to the first customers, the item should not be priced, as the offer is not available to everyone. Introductory special offers may be included if they are available to all. In reality, however, given the need to price the same “basket” each month, such offers will not be chosen as representative items unless they are introduced at the time of an update of the “basket” or when a replacement item needs to be chosen. Discounts on goods close to expiry dates should be disregarded or treated as specification or quality changes.

6.84 Bonus offers, extras and free gifts. Prices for items temporarily bearing extra quantities (for example, 30 per cent extra free) should not be adjusted to take account of the increased quantity if it is thought that the extra quantities involved may not be wanted by most consumers, will not have influenced the decision to purchase or will not be consumed. Similarly, free items with other purchases (such as buy 2 get 1 free or free gift with every product purchased) should be disregarded. Money-off coupons for future purchases should be disregarded, as these may not be used or wanted. Free gifts such as plastic toys in cereal boxes should be ignored because they are not included in the list for price observations; it is the price to be paid to get the cereal in the box that is relevant. Collectors should be aware that temporary “special offer” weight changes (X per cent extra free) could become a permanent weight change (for example, cans of alcoholic drinks changing size from 440 ml to 500 ml) and should feed the information back to head office as they become aware of it. In this way, head offices can issue new or amended guidance to price collectors about item specifications.

6.85 Stamps. Sometimes purchasers are given special stamps, which can be accumulated and subsequently exchanged for goods and services. If a discount is available as an alternative to such stamps, then the discounted price should be recorded. Otherwise, the stamps should be disregarded.

6.86 Trade-ins. In general the price reduction obtained by trading in an old item (for example, a car) compared with the nominal full price should be ignored. This treatment follows convention, as the transaction essentially relates to a second-hand good and only the service charge levied by the outlet in buying and selling the good comes under the scope of the index. In reality, however, the situation is not so clear-cut. For instance, a garage may well give a discount which is greater than the retail value of the traded-in car and, therefore, in effect gives a genuine discount on the new car. In many cases, discounts from trade-ins are very difficult to evaluate. The trade-in value may be negotiable in each case, and the full nominal price—which is used as the benchmark against which the discount is measured—may not be known. It may therefore be best to report the list price or asking price.

6.87 Sales taxes. When an indirect tax is not included in the price of individual items in a shop, but is instead added on when the customer pays for the item, great care must be taken to record the price including tax. To make sure of this, with items for which the price is normally quoted pre-tax, and in areas where a general sales tax is added to the bill, the price collection forms should require the collector to indicate whether or not the price recorded does include the tax—as a price check—so that it can be added where necessary.

6.88 Tips for services. If a compulsory service charge is included, for example on a restaurant bill, only the compulsory amount should be included in the price, but not any additional discretionary tips. For services which are free in principle, but which in practice can rarely be obtained without what amounts to a tip, or where tipping at a standard rate is the common practice, such tips should be added to the specified price.

6.89 Regular rebates or refunds should only be taken into account when attributable to the purchase of an individual identifiable product and granted within a time period from the actual purchase such that they are expected to have a significant influence on the quantities buyers wish to buy. For example, money-back deposits on bottles should be deducted from the price if they are a sufficient incentive for returning the bottle, while money-back offers on lawn mowers after a five-year period should be disregarded. In all cases, a consistent decision for each item must be applied over time. Decisions about the treatment of rebates are not easy to recommend, as many decisions are made on an individual basis. They may reflect income rather than expenditure changes and may require different treatment for, say, national accounts uses.

6.90 Irregular rebates or refunds should only be taken into account when they apply to the purchase of an individual product and are granted within a time period such that they are expected to have a significant influence on the quantities purchasers are willing to buy. Loyalty rebates or coupons associated with previous expenditure at the outlet, to be used for similar or other purchases, should generally be disregarded, as they are discriminatory. If they are significant factors, they should be treated as stratification or coverage aspects of sampling (see Chapter 5). One-off rebates (for example, associated with privatization) should be disregarded as they do not relate to the specific time period of the consumption and are unlikely to affect levels of consumption. They can be viewed more as a source of additional income.

6.91 Credit card and other payment arrangements involving interest, service charges or extra charges incurred as a consequence of failing to pay within a specified period of time from the purchase should be disregarded. For example, zero interest as well as positive interest loans granted to finance a purchase should be disregarded when determining the price. Reductions for cash payments may be included but care should be taken to ensure consistent treatment from one period to the next.

Price bargaining

6.92 Bargaining relates to a situation where prices are individually negotiated between sellers and purchasers, and are not predetermined. The process of negotiation is a characteristic of, for example, marketplaces in many African countries where almost everything to be purchased must be negotiated to arrive at an agreed price, including a wide range of life’s daily necessities that can account for a large part of household consumption. The system of bargaining is characterized by its great flexibility in the setting of prices. Final transaction prices and quantities will vary from one transaction to another and cannot be determined until the purchase has been made. Similarly, there will be variations between transactions in the quality of the goods being purchased. Clearly, these special conditions require special methods to determine purchasers’ prices for inclusion in the consumer price index.

6.93 It can be argued from the viewpoint of the system of national accounts that bargaining is a form of price discrimination. A purchaser is not free to choose the purchase price because the seller can charge different categories of purchasers different prices for identical goods and services sold under exactly the same circumstances. It follows that “identical” products sold at different prices should be recognized as having the same quality, and their prices must be averaged to obtain a single price relative to calculating price indices. In reality, the variation in transaction price can rarely be associated with identifiable price-related categories of customers. Rather, purchasers may inadvertently buy at a higher price than may be found elsewhere or could have been finally negotiated. Notwithstanding this, collectors of prices should guard against the presumption that price differences do not relate to quality (or quantity) differences.

6.94 Where prices are determined by bargaining, standard price survey methods—which consist of collecting prices directly from sellers—can generate erratic price indices that do not reflect actual price movements in a market. For example, prices collected by enumerators depend on their ability, willingness and power to bargain, in the same way as actual prices paid by genuine purchasers. In addition, prices can vary during the course of a day as well as from one day to the next, adding an extra dimension to the concept of representativeness. A number of survey methods and price collection techniques have been developed to overcome the difficulties inherent in measuring prices that have been bargained.

6.95 Survey by purchase of products. The principle is that price collection should be carried out in conditions that simulate as closely as possible situations in which real transactions actually take place. Price collectors behave like regular purchasers by actually purchasing items to be priced and spreading their purchases over the day to ensure representativeness. In each case, the field manager will need to carry out regular checks on quantities and prices obtained by collectors. The following approaches may be taken:

• Price collectors buy items to determine the relevant price through bargaining. They should be trained to behave as normal purchasers and strive to get the lowest possible price from selected outlets and sellers. Given the high turnover of sellers, the sample of sellers should be partially renewed on a regular basis to ensure that it remains representative and chained in as appropriate.

• Price collectors buy items and, in addition, are given an incentive to get the best price. For example, a price ceiling may be set and the collector may receive a proportion of the difference between the ceiling and the bargained price. This incentive system guards against potential difficulties caused by the collector not getting the lowest price because, unlike an ordinary customer, he or she is not concerned with maximizing value for money and is not constrained by income.

6.96 Survey of purchasers. The prices purchasers have paid are collected throughout the day immediately after the purchaser leaves the outlet or market stall, together with a record of the quantity and quality of the product purchased. The extent of the haggling should be determined (for example, opening and closing prices) together with an indication of the relevant parameters determining the price. A form of incentive payment for survey participation may be needed where there is reluctance among purchasers to submit to such time-consuming questions.

6.97 For the survey by purchase of products and the survey of purchasers, all items in the basket of items used to calculate the consumer price index that are subject to bargaining should be covered. The number of prices collected needs to be sufficient both to cover all relevant items and to provide a reliable guide to average price. This may be difficult to determine beforehand, although previous price collections should provide some guidance. It is suggested that price collectors engaging in a survey of purchasers are given a form on which to record the number of quotations per stall or shop, as indicated by the various respondents. This can be used to check the number of quotes obtained against the target number set by head office. An example of such a form is given in Table 6.1.

6.98 Survey of trends in wholesale prices. A limited parallel collection of wholesale prices can be a useful supplement for problematic items where the information obtained from the above survey techniques is only partially successful, for example where there is a deficit in the number of observations obtained. Ideally, prices should be obtained from the particular wholesalers where the relevant retailers get their goods. All factors should be observed which might result in increases in the corresponding retail prices, such as changes in taxes on retail activities, licence fees and the rental for the market stall. Assuming that those factors remain constant over time, the evolution of wholesale prices may be used as a proxy for the retail price index of relevant items. The price of an item for the current period would be estimated by multiplying the price of the previous period by the corresponding evolution in wholesale price.

6.99 Determination of the prices paid by a purchaser can be problematic where the final price is for a bundle of items, for example where a stall holder gives the purchaser extra quantities as a bonus for buying a number of goods. If the bonus comprises several categories of items, including the item on which a transaction price was being directly negotiated, then the purchase has to be split into as many sub-transactions as item categories. In these cases, a commonsense approach is needed. There is a fine dividing line between this type of circumstance and the “two-for-one” offers sometimes found, for example, in Western-style supermarkets. The latter form of discount is often excluded from price calculations on the basis that the purchaser does not want or use the additional amount supplied. Additional perishable goods, for example, will become out of date and be thrown away. This argument is less relevant in market purchases in a developing country, where many consumers will be living on a subsistence income and therefore will consume all purchases. In such cases, purchasers will have actively bartered an overall price for the total basket of purchases, including any “free” goods thrown in.

Table 6.1

Example of a survey form showing the number of price quotations by shop or stall

Table 6.1

Example of a survey form showing the number of price quotations by shop or stall

Items	Targeted number of quotations (set by head office)	Actual number of quotations
		Shop/Stall 1	Shop/Stall 2	…	Shop/Stall n
Item 1	5	0	3		5
Item 2	4	4	5		4
Item 3	8	5	8		8
…
Item k	5	7	2		6

View Table

Table 6.1

Example of a survey form showing the number of price quotations by shop or stall

Items	Targeted number of quotations (set by head office)	Actual number of quotations
		Shop/Stall 1	Shop/Stall 2	…	Shop/Stall n
Item 1	5	0	3		5
Item 2	4	4	5		4
Item 3	8	5	8		8
…
Item k	5	7	2		6

6.100 The method for determining the price paid by the purchaser is illustrated in the following example: a purchaser wants to buy 5 kg of carrots and is offered a bonus consisting of 500 grams of carrots, 100 grams of lettuce and 200 grams of baby marrow.

6.101 Three transactions can be identified, involving: 5.5 kg of carrots; 100 grams of lettuce; and 200 grams of marrow. The bonus has to be valued at prices at which the seller would have sold and the purchaser would have bought the items. The assumption made is that prices, in local currency units (LCU), would have been determined through bargaining on the same conditions as the price of the item needed (carrots). If the opening value of 5 kg of carrots is LCU 15,000 and the closing value LCU 12,000, whereas the opening values of other foodstuffs included in the bonus are LCU 990 for a bunch of 264 grams of lettuce and LCU 4,620 for a heap of baby marrow of 4.4 kg, the actual closing price of carrots will be determined as shown in Table 6.2. The actual purchaser’s price of carrots is found to be LCU 2.0967 per gram or LCU 2,096.7 per kilo.

6.102 If the price collector does not know the closing price at which lettuce and baby marrow would have been sold by the seller of the carrots, then it can be estimated. This is done by collecting opening values and standard quantities from a sample of sellers in the same market or at different outlets in the same area. The average opening price of an item is equal to the sum of opening values of the item divided by the sum of relevant standard quantities. For each bonus item (lettuce and baby marrow), the resulting average opening price will be divided by the bargaining ratio calculated on the item needed (carrots) to estimate a closing price for that bonus item. The value of each bonus item is obtained by multiplying the closing price by the quantity offered. If the packet of bonus items contains an item of the same quality as the requested item, that bonus item will be valued on the basis of the closing value of the requested item.

Forced replacements, product substitution and quality adjustment

6.103 A difficulty which confronts both local and central price collections occurs when an item that was being priced is no longer available and a substitute needs to be found. This is briefly discussed here because it relates to real decisions facing price collectors in the field, but the issues are covered in more depth in Chapters 7 and 8. In cases where a replacement has to be found, the price collector should normally take the nearest equivalent product available in the outlet, taking into account those characteristics which will be most influential in determining price and purchasing habits (for example, one out-of-date or obsolete item should not be replaced with a close item which may also shortly suffer the same fate). Nevertheless, where it is considered desirable to take the opportunity provided by product substitution to update the sample, a “most representative” replacement may be chosen. In the latter case, care must be taken to ensure that sufficient controls are in place to achieve the desirable end.

Table 6.2

Example illustrating the method for determining the actual price paid by the purchaser when bargaining takes place

Table 6.2

Example illustrating the method for determining the actual price paid by the purchaser when bargaining takes place

	Requested item	Bonus items
	Carrots	Carrots	Lettuce	Baby marrow
Opening value of standard/requested quantity (local currency units)	15 000	15 000	990	4 620
Standard/requested quantity (grams)	5 000	5 000	264	4 400
Opening unit price of standard/requested quantity (local currency units per gram)	3	3	3.75	1.05
Opening unit price of bonus quantity (local currency units per gram)		3	3.75	1.05
Bonus quantity (grams)		500	100	200
Opening value of bonus quantity (local currency units)		1500	375	210
Closing value of items received (local currency units)	12 000	1 200	300	168
New price (local currency units per gram)	2.4	2.4	3	0.6
Bargaining ratio	1.25	1.25	1.25	1.25
Payment (local currency units)	12 000
Estimated closing value of bonus (local currency units)	1 668
Actual value of requested item (all carrots) (local currency units)	10 332
Quantity received of requested item (grams)	5 500
Actual purchaser’s unit price of requested Item (local currency units per gram)	2.09671
Improved bargaining ratio 1(12000–300–168) % 5500 = 2.0967. 23 % 2.0967 = 1.52.	1.522

View Table

Table 6.2

Example illustrating the method for determining the actual price paid by the purchaser when bargaining takes place

	Requested item	Bonus items
	Carrots	Carrots	Lettuce	Baby marrow
Opening value of standard/requested quantity (local currency units)	15 000	15 000	990	4 620
Standard/requested quantity (grams)	5 000	5 000	264	4 400
Opening unit price of standard/requested quantity (local currency units per gram)	3	3	3.75	1.05
Opening unit price of bonus quantity (local currency units per gram)		3	3.75	1.05
Bonus quantity (grams)		500	100	200
Opening value of bonus quantity (local currency units)		1500	375	210
Closing value of items received (local currency units)	12 000	1 200	300	168
New price (local currency units per gram)	2.4	2.4	3	0.6
Bargaining ratio	1.25	1.25	1.25	1.25
Payment (local currency units)	12 000
Estimated closing value of bonus (local currency units)	1 668
Actual value of requested item (all carrots) (local currency units)	10 332
Quantity received of requested item (grams)	5 500
Actual purchaser’s unit price of requested Item (local currency units per gram)	2.09671
Improved bargaining ratio 1(12000–300–168) % 5500 = 2.0967. 23 % 2.0967 = 1.52.	1.522

6.104 When a replacement is made, it is important for the price collector to provide a detailed specification of the new item so that head office can identify any associated quality change. This is to ensure that the consumer price index continues to reflect the cost of buying a fixed, constant quality basket of goods. Head office should then use the information collected to decide on any relevant quality adjustment to be applied.

6.105 When such a situation occurs, a nominal price in the base month (which for some indices will be the previous month) is needed for the new or replacement item. The latter may be obtained from the shopkeeper or one of three methods can be applied to take account of quality differences, which can then be used to estimate a new base price. These are direct comparison (that is, when there is no change in quality), direct (explicit) quality adjustment, or indirect (implicit) quality adjustment. When a new rather than a comparable replacement item is priced, it may be necessary for the new item to be kept out of the index for a short period until there is sufficient evidence of its longer-term availability and price stability.

6.106 In some countries, a table of quality coefficients is used to adjust prices. In one North African country, for example, the item “green tea” should be represented by Minara tea; however, if this is unavailable an alternative tea may be collected and that price scaled by the relevant coefficient (for example, Oudaya tea x 1.20). More detailed guidance on direct and indirect quality adjustment is given in Chapter 7.

6.107 If an outlet closes or refuses to allow further price collection, then another similar outlet should be selected from the same location and the indirect quality adjustment approach used to calculate new base prices. See Chapter 5 on sampling for replacing outlets within locations.

Electronic reporting

6.108 Electronic reporting for centrally collected prices and use of hand-held computers for local price collection can introduce greater efficiency into price collection and processing, as well as providing more scope for effective auditing, but both are dependent on the introduction of effective quality control procedures. Electronic reporting through the use of electronic point of sale (EPOS) or scanner data is also likely to increase over time.

6.109 Electronic reporting for centrally collected prices. Centrally collected data can be collected electronically in a number of ways. Once initial contact has been made with data suppliers, a mutually convenient electronic data collecting procedure can be initiated. Options include:

• –emailing data collection spreadsheets between the national statistics institute and the retailer;

• –emailing of price lists at agreed times by retailers;

• –touch-tone dialling facilities for data to be supplied in an agreed format;

• –use of the Internet (supplemented if necessary by telephone calls to clarify definitions and availability).

6.110 Hand-held computers. The greatest gains from the use of hand-held computers for local price collection are likely to be drawn from efficiency in data transmission, better quality data as a result of the additional editing facilities available in the field, and the elimination of transcription errors. In addition, hand-held computers can generally speed up timetables.

6.111 The validation checks made during local price collection using hand-held computers will generally differ very little from those that should be carried out at central office when paper forms are received under the more traditional methods of collecting price data. The advantage of hand-held computers is that they provide the opportunity to validate prices in the field and in consequence correct errors at the time of price collection, rather than attempting to do so afterwards. In practice, it may be expensive and very difficult to check prices after collection. For example, prices may have changed in the intervening period and the price collector may have to rely on the shopkeeper’s memory.

6.112 The choice of hand-held computer will depend on a number of factors, including price, reliability, maintenance and ease of use. The computing functions of data transfer, including back-up and downloading of data, as well as compatibility with office systems, are also important. Other considerations of particular concern to the price collector include ergonomic aspects, size and weight, editing facilities, and expected battery life. The risk of theft and other security matters will also play a part.

6.113 The introduction of hand-held computers can involve a significant initial outlay associated with purchasing the computers, developing the software and training price collectors. In addition there will be ongoing maintenance costs. These costs can sometimes be reduced or spread either by using the machines for other data collection in the national statistical institute, for example a household budget survey, or by contracting out to another organization that may already deploy these machines for other statistical surveys. These costs can be offset, at least in part, by more efficient working by price collectors and savings generated by less transcribing and inputting of data by hand, and a reduction in data editing by head office staff.

6.114 Careful planning is required in moving from a paper-based collection system to one using computers in order to avoid the risks inherent in such a change. National statistical institutes planning a move to price collection using hand-held computers should embark on extensive pilot testing and should also consider some limited double-running in parallel with the old paper collection system to ensure the robustness of the new method and that it is producing the same numerical results.

6.115 The additional facilities offered by hand-held computers, including local editing of prices, plus the elimination of the need for data transcription, may necessitate a general reorganization of the process for producing the consumer price index, and a redefining of roles and interaction between different members of the production team and between head office and price collectors.

6.116 It is important that clear rules and procedures are set out controlling the changes that can be made in the field by the price collector and the changes that should be made centrally. For example, replacement outlets could be pre-programmed in the event that outlets close down or refuse entry. Flexibility should allow price collectors to select and key in the new attributes for replacement items subject to procedures controlled centrally.

6.117 EPOS or scanner data. Electronic point of sale (EPOS) data usually refers to data obtained directly from a retailer’s electronic point of sale, while scanner data usually refers to a commercial database that collates individual EPOS data. National statistical institutes are increasingly looking towards EPOS or scanner data as a convenient method of obtaining up-to-date and accurate information, not only on the quantity and prices of goods sold but also on their specification. The latter can be used to control the representativity of the sample and also to measure changes in quality. The advantage of this is that data are collated electronically without the necessity of sending price collectors into the field.

6.118 When considering the use of scanner data, account needs to be taken of such matters as the representativity of outlet and product coverage, and also whether the average prices given in scanner data accurately reflect actual transaction prices in the outlets themselves. In addition, it cannot be assumed that the geographical and population coverage or the treatment of goods and transactions matches the scope of the index. Scanner data are also likely to be of little use in collecting prices of services, which in many countries comprise an increasing share of transactions and thus of weights in consumer price indices. On a practical front, the unique identification of products can sometimes be problematical, as one item might be covered by more than one code number, and code numbers may not be uniquely assigned to one product and may be recycled as items disappear.

Purchasing power parities

6.119 Purchasing power parities are used to deflate major economic aggregates, such as gross domestic product, to enable intercountry comparisons of real income levels to be made in terms of real volume, that is, adjusted to account for local prices and different consumption patterns. Purchasing power parities consist of intercountry comparisons of prices for a basket of goods and services that is both representative of and comparable between the countries involved. The underlying price data therefore differ from those used in consumer price indices in so far as the latter basket is designed to be representative solely of private household consumption in the economic territory of an individual country.

6.120 It would be attractive in principle to construct consumer price indices and purchasing power parities from the same basic set of price data. In practice, the scope for this may be limited because of the differing objectives of the two exercises. In particular, the additional need for prices collected in the context of purchasing power parities to be comparable between countries will generally result in a more tightly defined basket compared with that likely to be available and used for a consumer price index.

6.121 An investigation of the potential overlap between the two baskets may, however, identify potential areas where one price collection may suit both purposes. This may be the case particularly for unbranded goods and locally produced fresh fruit and vegetables, for example, where a locally produced dessert apple of a standard quality may be compared across countries without recourse to a reference to the variety concerned. In contrast, branded items—whether food or non-food—may be more problematical because of differences in availability and specification between countries.

6.122 In some cases, scanner data might provide a useful common source of price data for at least some elements of the purchasing power parities calculation, notwithstanding the drawbacks mentioned above. Annex 4 goes into more detail on issues relating to purchasing power parities and the International Comparison Program (ICP).

Data quality and quality assurance

6.123 Checks should be carried out to ensure the accuracy of the data on prices and that the index itself has been compiled according to the proper methodology. Checks to ensure that data are complete and correct should be carried out as early as possible in the collection and compilation processes. A return to the shop to re-input prices becomes increasingly less feasible as time goes on, and there is a greater risk that the prices in the shops will have changed since the initial collection. It is not possible to prescribe the type and range of checks that should be carried out. The checks will depend on individual circumstances, including sample design and the medium used for the collection of prices. For example, the use of hand-held computers by price collectors facilitates much more detailed checking at the time of the initial collection of prices in the shop than the equivalent paper system. Further guidance on quality assurance is given in Chapter 12.

Documentation

6.124 The importance of good documentation cannot be overemphasized. Documents are needed to explain what is to be done, when it should be done, how it should be done and why it should be done. Preparing such documents provides a useful opportunity to ensure the quality of current procedures used to collect prices and compile the index. It also provides an opportunity to review and improve these procedures. Once in place, documentation serves two purposes in the context of producing the index. First, it enables somebody to take over the work if the person responsible falls ill or leaves. Second, it provides a quality check to ensure that the procedures that should be carried out are indeed being carried out in practice. More generally, documentation can provide a useful reference for users of consumer price indices. Documentation is discussed in more detail in Chapter 12.