12 ORGANIZATION AND MANAGEMENT
- International Monetary Fund
- Published Date:
- August 2004
12.1 Consumer price indices (CPI) are one of the most important and widely used of macroeconomic indicators. As well as informing economic policy, they are used for indexation of welfare benefits, pensions, gilts and securities, and also for escalation clauses in private contracts. Accuracy and reliability are paramount for a statistic as important as a CPI.
12.2 The process of producing a consumer price index needs to be carefully planned. Individual circumstances vary to such an extent that this manual cannot be too prescriptive about timetables or critical path analysis of all the steps involved. Figure 12.1 nevertheless provides an outline of the kind of schedule of activities that should result from a detailed examination of the logistics of the whole periodic operation of data collection and the computation of the index.
12.3 The guidance given in this chapter, which is based on the experiences of a number of national statistical institutes, presents a range of organizational options. As individual circumstances can vary, the examples given of good practice may be for some offices aspirational.
12.4 In reviewing these options, this chapter covers the relationships between the field and central office (which kind of work is carried out in central office, the flow of information between each part of the organization, etc.). The size, frequency, cost or complexity of the collection of prices as the basis of the index may mean that in some countries not all these operations and relationships will be appropriate. The use of both a central and a local collection, or outsourcing of certain elements of the collection, may not always be effective. If the index is compiled infrequently, from a relatively small number of outlets, or concentrates on only specific location types, different circumstances will demand different solutions.
12.5 A local price collection involves collectors visiting individual outlets to collect prices for a variety of goods and services. This is the predominant method of price collection in most countries. The range and number of outlets visited and the types of goods and services priced will vary between countries.
12.6 Although the precise method of local price collection will vary, each price collector will usually be responsible for collection from a certain location or from certain types of outlet. Collectors will visit the same outlets in each collection period to attempt to price the same items. Through this type of arrangement, price collectors are able to build up effective relationships with retailers and specialist knowledge.
12.7 There are a number of important criteria relating to the conduct of the collection, whether the national statistical institute uses its own staff or contracts out the collection (as discussed below). These criteria include:
Collectors should always be smartly dressed and polite–whoever employs them, they are representing the national statistical institute.
They should carry identification to confirm their role and status.
They should make themselves known to the retailer or store manager when they arrive, and before they begin collecting prices.
They should comply with any request from the shopkeeper whenever possible, for instance, if the store is very busy and the shopkeeper asks the collector to return later in the day.
The collection should be carried out as quickly as possible, causing minimal disruption to store business.
12.8 Collectors should also follow rules of common sense in preparing for a collection. These may include making sure that they have: spare pens; the appropriate forms; a clipboard; a local map; spare batteries (if the collection is computerized); money for shopping centre car-parks; and wet weather clothing, if appropriate. In some circumstances a mobile telephone will also be useful.
12.9 One of the decisions facing any statistical agency carrying out a price collection is whether to use in-house staff or to tender the collection to an external organization, such as a private market research company, another part of the agency, or another governmental department that specializes in surveys.
12.10 The nature of the price collection and the distribution and profile of statistical staff may help to determine whether the collection is suitable for contracting out. Where price collection is continuous, or involves complicated decision-making (such as quality adjustment), or where prices are collected from a small number of locations, it may be advantageous to keep the collection in-house. However, if the collection takes place over a small number of days per month, from a large number of locations, is relatively straightforward and involves routine or simple decision-making (perhaps selecting from a prespecified list of codes), then contracting out may be considered if there are enough market research companies with suitable skills existing in the country.
Figure 12.1Price collection procedures
12.11 Contracting out local collection can lead to reduced costs. Where price collection is carried out electronically, the responsibility for purchase and maintenance of data capture devices may be transferred to the contractor.
12.12 Contracting out may also allow statistical staff to spend more time analysing data rather than collecting them. By divorcing the role of data collector and data checker, statistical staff may feel more comfortable questioning the validity of price data. Accuracy of data collected can be directly linked to the performance of the contractor through performance measures which drive incentive payments (and penalties if targets are not achieved).
12.13 Central shop-collected prices are prices obtained from the central offices of major retailing chains with national pricing policies. Branches of these chains may be excluded from the local collection if data can be collected more effectively centrally. Data suppliers may provide information on paper forms, or by entering price data on spreadsheets and forwarding them to the national statistical institute by email, CD-ROM or on floppy disks. Mail order catalogues can also be treated as central shops: prices are recorded as and when the catalogues are issued. These prices are then combined with those for the same items from the local collection.
12.14 Price data for services or fees may be collected centrally from organizations such as trade associations, national or local governmental departments and so on. Whenever possible, these prices are obtained from one central source, although there will have to be contact with regional or competing companies if there are local variations. Data may be requested in writing or by telephone, or may come automatically because the national statistical institute is on a provider’s mailing list. Providers may send either a full price list or tariff sheet, from which the relevant prices will be extracted by the CPI staff, or just the prices of those items specified in the data request. All price quotations should be confirmed by some form of written documentation. Frequency of enquiry varies across the range of items and depends on when prices are known or expected to change. The most common frequencies are monthly or quarterly, but there are also instances of collecting as and when necessary, although in these cases checks must be in place to ensure that all price data are reported. For instance, this may be the case where tariffs for gas, electricity and water change once a year on a predetermined date.
Quality in the field
12.15 Quality is an important aspect of price collection. A high-quality price collection enables a statistical agency to have confidence in the index it produces, and ensure that observed price changes are genuine and not the result of collector error. It is important that procedures are developed to ensure that the standard of collection is maintained at a high level for every collection period. These procedures will form the basis of collector training and should be included in any training material developed for price collectors. Guidance to collectors should cover price index principles, organizational issues and validation procedures.
12.16 Accurate item descriptions are vitally important in ensuring item continuity. Collectors’ descriptions should be comprehensive enough to ensure that collectors are able to price the same item in each collection period. It is therefore important that contributors record the attributes which uniquely define the item they are pricing. For example, for clothes it will be important that colour, size and fabric composition are specified to ensure that the same item is priced each month. For fresh fruit and vegetables, useful attributes to record may be country of origin, class and variety.
12.17 Accurate item descriptions will assist the price collector and head office in choosing a replacement for an item that has been withdrawn and will also help to identify changes in quality. Head office staff should be encouraged to spend some time, each collection period, going through collectors’ descriptions to ensure that the correct items are being priced. Collectors should also be encouraged to review their descriptions to ensure that they contain all of the relevant information. It may be useful to ask collectors occasionally to switch collections with another collector so that they understand the importance of good descriptions.
12.18 Continuity is one of the most important principles of price collection. Because a price index measures price changes, it is vital that the same item is priced every month in order to establish a true picture of price changes. So if, for example, ajar of a supermarket’s own brand of strawberry jam has been selected, that particular brand and flavour should continue to be collected. If it is out of stock in the collection period, another brand and flavour should not be used. If, however, in subsequent collection periods, the selected jam continues to be out of stock, but another flavour of the same brand and price is available, then this item should be chosen as a comparable item and the item description suitably amended. If no comparable item exists, then a new item must be chosen, and the description amended. Thus a new price chain will begin. It is not possible to be prescriptive because the concept of equivalence will vary between different countries; but for practical purposes it is important that a detailed description of the items being priced is kept.
12.19 As continuity is so important in the compilation of an accurate price index, collectors should be encouraged to check with the retailer that an item is out of stock before replacing it. Some guidelines may be drawn up by the head office of the national statistical institute to cover different items. Food items, for example, will usually come back into stock in the following collection period, and so should not be replaced immediately, whereas fashion clothing will rarely come back into stock once a “season” has ended or the stock has been exhausted, and so should be replaced immediately in the collection.
12.20 Collectors should also be encouraged to plan their route for price collection to take account of outlet opening and closing times and any special retailer requests. Collectors may find it useful to compile a route map, listing the order in which outlets should be visited. This is particularly useful when the collection in a location has to be undertaken by a different collector, for example to cover for sick leave. Collectors should be encouraged to try to collect prices at similar times within each collection period. This is particularly important when pricing volatile items, such as petrol and oil, where there can be sharp fluctuations.
Data entry queries
12.21 Once the price data are correct and complete, a series of validation checks may be run. In deciding on what checks should be carried out, account should be taken of the validation checks carried out in the field. For example, the use of hand-held computers will increase the potential for validation at time of price collection and reduce the need for detailed scrutiny at head office. In addition, it would clearly not be productive or cost-effective to repeat tests already carried out.
12.22 The range of tests carried out may include:
Price change: The price entered is compared with the price for the same product in the same shop in the previous month, and triggers a query where the price difference is outside preset percentage limits. These limits vary, depending on the item or group of items, and may be determined by looking at historical evidence of price variation. If there is no valid price for the previous month, for example because the item was out of stock, the check can be made against the price two months or three months ago.
Maximum/minimum prices: A query is raised if the price entered exceeds a maximum or is below a minimum price for the item of which the particular product is representative. The range may be derived from the validated maximum and minimum values observed for that item in the previous month expanded by a standard scaling factor. This factor may vary between items, again based on previous experience.
12.23 If a hand-held computer is used, both of these tests can be implemented easily to take place at the time of collection, otherwise they will need to be conducted in the head office as soon as possible after collection and before prices are processed on the main system. A failure in either test should not result in the collector being unable to price the item, but should prompt the collector to check and confirm the entry, and prompt for an explanatory comment.
12.24 Queries raised may be either dealt with at head office or sent to the price collector for resolution. For example, scrutiny of a form might show that a big price difference has arisen because the item priced was a new product replacing another that has been discontinued. In this case there may be no need to raise a query with the price collector unless there is evidence to suggest that to label the item “new product” is incorrect.
12.25 Where an error is discovered too late in the process to resolve, head office will need to reject it and exclude that item from that month’s index. Care should be taken that the item is also excluded from the base month so that the basket is kept constant.
12.26 Collectors should be encouraged to give feedback to head office on their experiences of price collecting. Collectors are a valuable source of information and often give good early feedback on changes in the marketplace. Collectors can often warn of size or product changes before the head office is able to derive this information from other sources such as trade magazines. Collectors’ feedback can be used to support observed price movements and to provide supplementary briefing material. It can also form the basis of a newsletter for collectors.
Quality checks in local collection: The role of auditors
12.27 The whole periodic routine of collecting prices in the field needs to be carefully planned and monitored, with arrangements in place to reflect local conditions. Circumstances vary, so it is not appropriate to be too prescriptive. It is, however, important to ensure that price collectors send in information when it is due. If they do not do so, it is necessary to find out the reason and to take appropriate action. It is also important to check that the information sent in is accurate and complete.
12.28 One way of monitoring the work being carried out by price collectors is to employ auditors to occasionally accompany collectors during the field collection, or to carry out a retrospective check on the data that have been collected.
12.29 If an auditor intends to accompany a price collector, he or she will need to inform the collector in advance in order to arrange meeting details. In general, the auditor will not accompany the whole price collection but will spend a few hours observing the price collection in a specific location. For example, it may be desirable to observe the collection of certain items or in particular outlets where collection might be problematical, and the price collector may need to rearrange his or her route accordingly.
12.30 Prior to monitoring, the auditor will need to carry out preparation work–a pre-monitoring check. Such a check might involve looking at descriptions, prices, price history and indicator codes of the items collected in the chosen location. This type of check will enable the auditor to have a good idea of the standard of collection prior to going into the field and may suggest on which areas of the collection the auditor should concentrate his or her efforts.
12.31 An auditor’s main duty is to ensure that the price collector is following the procedures and instructions laid down for price collection and that the collection is being performed competently. While the auditor may not have the role of a trainer, the opportunity may be taken to give some coaching when errors are noted. There should also be the opportunity for the collector to ask the auditor relevant questions during the monitoring exercise.
12.32 Auditors may undertake other duties at a location besides accompanying the collector. For instance, they may enumerate outlets or carry out an item review. Following a monitoring visit, the auditor should compile a report detailing the observations made while accompanying the collector. This report should include a summary of findings, a list of points for action, and a recommended course of action. Auditors may advise that a collector receive extra training on certain aspects of the price collection; head office (or the contractor, if the collection has been outsourced) should act on this. This report will then be used as a starting point on the auditor’s next visit. In other instances, general problems may arise where solutions need to be disseminated to all price collectors, perhaps by issuing revised instructions or through a newsletter.
12.33 Another approach to monitoring the standard of price collection is to carry out a backcheck, a retrospective check of a proportion of the prices recorded during the collection.
12.34 Backchecks can be used to:
–assess the standard of competence of individual price collectors;
–audit the overall standard of price collection;
–identify general training needs or the specific needs of an individual;
–highlight any key issues including, for example, problems with documentation or instructions issued by head office;
–identify areas where collection is problematical; for example all collectors may have problems in certain types of outlets, prompting the need for more detailed head office instructions.
12.35 Backchecking should be done by an expert independent of the process (preferably employed by the national statistical institute). Backchecking is carried out by visiting the selected outlet and re-collecting the prices and other relevant information, such as attribute or description codes. This activity should be carried out close to the original collection period to avoid problems of price changes occurring in the interim. It is important that backcheckers seek permission from the shopkeeper beforehand and follow the general criteria of conduct for local collection, as described in paragraphs 12.5 to 12.12.
12.36 For a backcheck to be a useful exercise it is important that performance criteria are determined to which all backcheck results can be compared. These criteria should set, for example, the acceptable number of price errors per number of items checked. Well-defined criteria will enable easy identification of a poorly performing collector or location following a backcheck.
12.37 A backcheck may include a range of tests to identify the following:
price difference–if the price is different, the auditor should check with shopkeepers to see if there has been a price change since the original collection took place;
insufficient item description–each item should be uniquely defined so that another collector can step in to cover the absence of the usual collector, for example in the event of illness;
wrong item priced–such as incorrect size being chosen;
items wrongly recorded as missing or temporarily out of stock.
12.38 A report should be sent to head office for scrutiny once the backcheck has been completed. Head office will then need to take appropriate action, which may include, for example, retraining or sending out supplementary instructions.
Other auditor functions
12.39 The range of tasks that an auditor carries out will vary from one statistical agency to another. Monitoring the standard of price collection will always be the main focus of the auditor. There are a number of other areas, however, in which auditors can be called upon to contribute.
12.40 Auditors may be required to help with the sampling of locations and items. Auditors can check that proposed collection locations contain an adequate range of shops. They can also advise on economic conditions in these locations and on any dangerous areas. Auditors can carry out commodity work. For example, if a particular item seems to be causing difficulty for price collectors, auditors can speak to collectors and retailers with a view to determining reasons for these difficulties. Auditors can also advise on changes to basket composition. They can ensure that products suggested by head office are available across the country, and can suggest item descriptions and weight bands. Furthermore, auditors can provide reports on collection in existing locations. For example, head office may raise a query about a particular outlet in a particular location; auditors can visit this outlet to find the answer to the question or to persuade a retailer to continue with the survey.
Quality checks in head office
12.41 Four kinds of regular checking are necessary in head office:
to ensure that the price collectors’ reports are sent in when they are due. If this is not done, it is necessary to find out the reason and to take appropriate action to obtain the reports;
to confirm that the reports contain what they are supposed to contain, i.e. that fields which must be filled in have not been left blank, that numeric fields contain numbers and that non-numeric fields do not;
to review and edit each return. Substitutions may have to be made centrally or those made by the collectors may have to be approved. Unusual (or simply large) price changes may need to be queried. Items priced in multiple units or varying weights may have to be converted to price per standard unit. Missing prices must be dealt with according to standard rules relating to the cause;
errors introduced when keying the numbers into the computer or transcribing them onto worksheets must be found and corrected, and preferably avoided in the first place by eliminating the need to transcribe.
12.42 It should be noted that the way the data are organized in worksheets or in the computer may differ from the way they are organized on receipt, since they will arrive at the central office organized by collector, outlet and item. Their origin should, however, be recorded so that reference to it can be made should processing disclose any problems with the data. Furthermore, even if codes provided to the collectors to list items and to describe or qualify the prices are used unchanged in the processing, other codes may have to be used for information which comes in from the collectors in non-coded form.
12.43 How the checking is organized will vary from country to country. In some cases, local or regional supervisors will do some of it; in other cases, it will be more appropriate for it all to be done centrally. Some of these tasks can be done by computer, others manually. Therefore, no general suggestion can be made about the sequence of the work or about its division into different parts.
12.44 Procedures should be in place to check that all documents, messages or files are returned from the field so that price collectors can be contacted about missing returns. Initial checks should then be carried out to ensure that data are complete and correct. For instance, checks should be run to ensure that unexpected duplicate prices (i.e. for the same item, in the same shops, in the same location) are not taken on, and that the location, outlet and item identifier codes which accompany each price exist and are valid. If any prices fail these checks, a query should be raised with the price collector for clarification. Since some of the checking may require reference back to the price collectors (or to their supervisors or respondents when direct mail questionnaires are used), the timetable for producing the index must allow for this communication to take place.
12.45 Following the checks that the price data are correct and complete, a series of validation checks may be run. In deciding on what checks should be done, account should be taken of the validation checks carried out in the field. The use of hand-held computers will increase the potential for validation at the time of price collection and reduce the need for detailed scrutiny at head office. It would clearly not be productive or cost-effective to repeat all the tests already carried out locally, except as a secondary audit or random check that those checks have been completed.
12.46 The range of checks that might be carried out is covered in paragraphs 12.21 to 12.25. In addition, it is possible for the head office to use the price data received that month to identify outliers.
12.47 Reports should routinely be generated for most representative items, to help the analyst pick out particular prices for which the level or change stands out as different from that reported for similar varieties elsewhere, or simply where the change lies outside certain specified limits. Thus, a computer printout can list all prices which either fall well outside the range of prices obtained last time for that representative item, or for which the percentage change from last time for the same item in the same outlet falls outside a specified range. The limits used will vary from item to item and can be amended in the light of experience. The analyst can then work through the printout, first ascertaining whether there has been a keying-in error, and then examining whether any explanation furnished by the collector adequately explains the divergent price behaviour or whether a query should be sent back to the supervisor or collector. The timetable should allow for this, and anomalous observations should be discarded where an acceptable explanation or correction cannot be obtained in time.
12.48 Other reports may be produced regularly on the basis of reports for several periods (e.g. several months) to detect accumulated patterns, thus enabling broader problems to be detected. For example:
One collector’s reports might show many more “outlet closed” remarks than those of other collectors, perhaps indicating either a motivational or training need on the part of that collector, or a change in retail trade patterns in a particular area.
Variety substitution for a particular representative item might become more numerous than hitherto, suggesting a possible need for revision of the specification or the choice of another representative item.
Where tight specifications list a number of brands and models of which one is to be chosen, but a large number of prices are for items not specified in the original list, this will suggest that the specified brands and models are no longer appropriate and a review of the list is required.
The dispersion of price changes for a particular representative item might be much larger than it used to be, raising the question of whether it has been appropriately specified.
12.49 Routine computer-generated reports should enable those in charge of the index to detect the existence of all such problems. Two types of reports are particularly useful: index dispersion reports, and price quote reports.
12.50Index dispersion report. This is a list of items indicating the current index for each item, the number of valid quotes for each item, and the number of price relatives (the ratio of current price to previous valid price) in each of a series of pre-selected ranges (for example, less than 40, 40–49, … 190–199, greater than 199). The index dispersion reports can be used to identify quotes with price relatives that fall outside the range of the main bulk of quotes. These quotes can be identified from quote reports for the item, then investigated and appropriate action taken if necessary.
12.51Price quote report. This consists of a range of information on an item that the index dispersion report has highlighted as warranting further investigation. Information listed may include current price, recent previous prices and base price, together with locations and types of shop. The report can be used to identify the quotes that require further investigation and also to investigate rejected prices.
12.52 Algorithms can be created which may be used to identify and invalidate price movements that differ significantly from the norm for an item. For some seasonal items for which price movements are erratic, it may be more appropriate to construct an algorithm to look at price level rather than price change.
12.53 An example is the Tukey algorithm. This operates as follows:
The ratio of current price to previous valid price (the price relative) is calculated for each price. (In the case of items tested by price level rather than price change, this stage is omitted.)
For each item, the set of all such ratios is sorted into ascending order, and ratios of 1 (unchanged prices) are excluded. (In the case of items tested by price level rather than price change, the prices themselves are sorted.)
The top and bottom 5 per cent of the list are removed (this 5 per cent is parameter 1).
The “midmean” is the mean of what is left.
The upper and lower “semi-midmeans” are the mid-means of all observations above or below the median.
The upper (lower) Tukey limit is the midmean plus (minus) 2.5 times the difference between the midmean and the upper (lower) semi-midmean. This figure of 2.5 represents parameters 2 and 3. The upper and lower values can be set independently if desired but are currently set to be equal.
If the upper limit is negative, it is set to zero. (If price levels are used, the lower limit is set to zero.)
Price relatives, or price levels, outside the Tukey limits are flagged as unacceptable and requiring amendment or further investigation.
12.54 The Tukey algorithm has a number of advantageous characteristics. In particular, it produces intuitively reasonable results; is consistent from month to month; is robust in the presence of outliers (in other words, adding in one or two rogue observations does not affect very much the limits set by the algorithm); and is robust as data volume changes (i.e. limits calculated from a subset of the data do not vary much from those calculated on the full data set).
12.55 Whilst algorithms can be an efficient way of highlighting problematical data, a word of caution should be expressed about using them. Analysts will want to assure themselves that their use does not result in systematic bias in the index. This is a matter that may also need to be taken into account in any editing routines, although it is less likely to be problematical in the context of manual editing.
Producing and publishing the index
12.56 In regard to producing and publishing the index, there are a number of organizational models that could be adopted for effective working. Considerations to be taken into account in deciding on the appropriate organizational structure include:
–the need for clarity of reporting lines;
–the need for a clear division of responsibilities;
–centralized or decentralized management of fieldwork (see above discussion on local collection, and the outsourcing of fieldwork, paragraphs 12.6 to 12.14);
–production management versus technical development;
–compatibility with corporate structures in the national statistical institute, for example, in relation to quality management, methodological research, and dissemination.
12.57 In some cases, for instance where little in-house expertise in fieldwork practice exists, it may be advantageous for fieldwork to be conducted by a different organization in either the public or private sector. In these circumstances it is important that an effective contractual relationship exists with regard to the data. There should also be agreed delivery targets and performance measures to cover such things as data delivery timetables, response rates and levels of accuracy. Consideration should also be given to the independent auditing of the contractor’s work on a sample basis.
12.58 The system used for the regular computation of the index must be sufficiently flexible to allow for changes in the kind of data obtained. For example, local price collection for purposively sampled products from the branches of a large supermarket chain may be superseded by centrally collected prices for a statistical sample drawn from complete sales data made available by the head office of the chain. In these circumstances, a modular approach may be seen as an advantage.
12.59 Analytical computations provide comparisons between the published index, or one or more sub-indices, and what they would have been using different methods or data. They help to explain why the index has moved as it has and they allow methodological experimentation. The following examples of such investigations serve to make clear some of the computational capabilities and data that are required:
–alternative aggregations of sub-indices;
–the effects of different weights; the effects of introducing newly significant product categories; and price-updating of weights;
–number and duration of missing observations; how a different method of estimating them would affect the index;
–comparison of indices computed with various sub-samples of the data as a means of estimating variance; variances of price ratios;
–computation of a standard reference index (one with no explicit quality adjustments) so that an implicit quality index is obtained;
–numbers of sampled products; rates of forced replacements; and lengths of time products remain in the sample;
–frequency distributions of quality adjustments.
12.60 To examine such matters, the database must contain not only prices but detailed descriptions of product replacements, explanatory remarks attached to observed prices, and so on. Generally, it will be found that historical databases will be too large to be stored live on the system and therefore will need to be archived. Detailed documentation relating to the archived material will need to be kept to guard against loss of vital information caused by changes in computing staff or computers. Consideration should also be given to appointing a data custodian with responsibility for all archived records.
12.61 Spreadsheets may be used for compiling sub-indices that require special procedures, or where data are collected centrally, or on an uncertain timetable or to a different timetable from that of other data collection, but effective control procedures need to be put in place. Examples of types of prices for which the use of separate spreadsheets may be useful include: air fares, hotel accommodation, newspapers and car rental. Such use of a spreadsheet has the advantage of additional flexibility and scope for combining responsibility for data collection, data input and computation. The compiler’s specialized knowledge about the markets or outlets where these prices may be observed, combined with analytical tools applied to the spreadsheet, will help the compiler to detect any irregularities in the data, facilitate investigation of whether these reflect reporting or input errors, and allow for rapid rectification. The ability to jump between numerical data entry and a chart displaying, for example, current-month and previous-month entries, helps the rapid and simple detection of anomalies. The same person can then follow this up with the data supplier.
12.62 As time passes, the resolution of problems that have arisen and adaptation to new circumstances will result in changes in the spreadsheet. Unless quality management controls are put in place, there is a danger that the spreadsheet will be understandable only by the person responsible and that it will not be properly documented. If so, two unfortunate consequences can arise:
If that person is absent, retires or moves to another job, his or her successor will find it very difficult to maintain the continuity and quality of the sub-index.
New procedures introduced to deal with new circumstances may be inconsistent with procedures used for other sub-indices for which other people are responsible.
12.63 Good documentation and good communication with colleagues will diminish these risks. At a minimum, there should be an insistence that the spreadsheets and changes in them are made understandable by the provision of adequately explicative row and column headings or of notes attached to headings. Furthermore, changes in procedures or formulae, rebasing and the application of new weights should always be introduced by moving computation over to a new sheet within the workbook, not by modifying the old sheet. The new sheet and the old sheet will then exist side by side so that they can be compared.
12.64 Inadvertent changes may be prevented by using passwords to cells containing formulae and by locking cells containing input data once editing is completed. Passwords should be known only to a limited number of people with authority to edit the spreadsheets. Regular back-up by copying the whole workbook to another disk is also essential.
12.65 Various checks should be carried out when introducing changes. These may include a comparison of the old and new basis using data from parallel running of collections (e.g. when handing over to a new collection contractor) or re-estimating backwards–for example, when new base prices are being imputed for a complete range of goods or services. Any anomalies can then be investigated further.
12.66 A consumer price index will arguably be the most important and highest-profile statistic that a national statistical institute produces and can affect the widest range of users. There is often a legal obligation for the CPI to be published within a short time period after the end of the month to which the data refer. For example, in the European Union, there is a legal requirement to publish within 30 days of the reference period the Harmonized Index of Consumer Prices (HICP), which uses the CPI data sets from member States (although the Eurostat timetable is for publication two weeks earlier than this). Any delay in publication can have a significant impact on subsequent months, threatening future publications. Significant delays could take months to catch up, in order to return to the existing tight publication timetables. It is critical, therefore, that national statistical institutes develop a robust and tested disaster recovery plan, however unlikely the need to implement it.
12.67 There are a number of possible causes of disaster:
–failure of an external contractor to fulfil obligations to supply information;
–failure of the computer system;
–major natural disaster or other event (e.g. terrorist activity) affecting the operations centres or head office of the national statistical institute.
12.68 Where the collection is contracted out, one of the most important requirements of a disaster recovery plan is to recruit an alternative permanent service supplier as soon as possible. It is probable that, on termination of a contract with an external provider, the national statistical institute could arrange to have the services supplied by a third party, but only on a temporary basis prior to re-letting the contract through a competitive tendering process.
12.69 Additional money may need to be obtained for implementing a computing disaster recovery plan. Consideration needs to be given as to whether outsourcing the disaster recovery plan to a company specializing in the provision of back-up support or maintaining an in-house capability is the best option. This will, in part, depend on the number of sites and locations at which the national statistical institute operates. If the organization has a number of sites, some distance apart but linked by modern communications infrastructure, then there is less likelihood that they will all be affected by a natural disaster.
12.70 The managers of disaster recovery plans will also need to consider:
–full specification of accommodation and associated requirements (e.g. personal computers, telephones) associated with each site;
–allocation of specific officers to specific duties for the disaster recovery period and identification of each individual’s training needs;
–investigation of practicalities and associated expenses for matters such as access to shared drives and systems, including communication and the quality management systems, from other sites;
–confirmation of costs, arrangement of site visits and liaison with procurement units in negotiating contracts.
Quality management and quality management systems
12.71 Statistical offices are faced with the continuous challenge of providing a wide range of outputs and services to meet user, i.e. customer, needs. Thus a key element of quality is customer focus and the effective dissemination of relevant, accurate and timely statistics. In addition, it can be argued that quality management should include effective customer education on the use of such statistics. In these terms, success can be measured by the achievement of a high level of satisfaction amongst well-informed users.
12.72 For the quality management of a CPI, it can be argued that the priority area is quality control of the production process itself. For most national statistical institutes, quality control of production will be an area which represents a high risk, given the complexity of the process and the financial implications of an error in the index.
12.73 If the principles of organizing and managing the collection of data, and subsequent processing of information to produce a consumer price index, are to be adopted, then it is vital that a system is in place to ensure that the data obtained, the processes involved in achieving the specified outputs, and the formulation of the policies and strategies that drive them are managed in an effective, consistent manner. The processes should, wherever possible, be open to verification; and mechanisms should be put in place to ensure that outputs meet requirements–in other words, customer satisfaction. Taken together, these elements form the basis of a quality management system.
12.74 There are varying perceptions about the meaning of quality but an important common thread is the requirement to react to and serve users of the CPI and to ensure continuous improvement in that service. Thus the implementation of an effective quality management system requires a high level of understanding of customer needs and the translation of this into a coherent statistical and quality framework. Such a framework is also necessary for putting together criteria for judging success. User needs can be canvassed either formally through negotiation of contractual obligations which may or may not be legally binding, or less formally through talking to customers on a one-to-one basis or through customer surveys.
12.75 In many countries, issues relating to the governance of the national statistical institute are set down in a “framework” or similar document. This defines the functions and responsibilities of the national statistical institute, and generally guides and directs the work of the office. For instance, an objective stated in the framework document “to improve the quality and relevance of service to customers–both in government and the wider user community” provides a powerful statement for determining workplans.
12.76 This recognition of the importance of quality can be further endorsed by a published vision of the national statistical institute as a key supplier of authoritative, timely and high-quality information. Such a vision can be encapsulated by publishing objectives in an annual business plan. These objectives can include improving quality and relevance, thereby increasing public confidence in the integrity and validity of outputs.
12.77 Performance can be measured against a combination of a number of factors, including accuracy, timeliness, efficiency and relevance. There are a number of practical examples and case studies of quality systems, illustrating how different models may be applied.
Quality management systems
12.78 Various standards of best-practice standards can be exploited to help organizations to improve quality management. Some of these standards have the added advantage of being internationally recognized.
12.79Total quality management. Total quality management (TQM) is more closely identified with a management philosophy rather than a highly specified and structured system. The characteristics associated with TQM and an effective culture of quality in an organization include:
–clearly defined organizational goals;–strong customer focus;
–strategic quality planning;
–continuous quality improvement.
12.80Benchmarking. Benchmarking is a process of comparing with others, and learning from them about what you do and how well you do it, with the aim of bringing about improvements.
12.81 There are already a number of benchmarking partnerships operating within national statistical institutes, some specifically considering the CPI. The Australian Bureau of Statistics has been particularly active in this area, and undertook an exercise in 1998-2000 in partnership with the United Kingdom. Benchmarking projects have also been undertaken in New Zealand, the Scandinavian countries and the United States.
12.82 Areas that can be considered when benchmarking a CPI collection may include:
–timelines, accuracy and coverage of collection;
–benefits of index methodologies for various items, e.g. geometric mean as against average of relatives;
–frequency of collection and publication;
–cost of collection per unit of commodity, etc.
12.83European Foundation for Quality Management Excellence Model. The Excellence Model (1994) constructed by the European Foundation for Quality Management (EFQM) is a diagnostic tool for self-assessment. The model is widely used by governmental organizations across Europe to improve quality and performance. It may be described as a tool that drives the philosophy of TQM.
12.84 The EFQM Excellence Model focuses on general business areas and assesses performance against two sets of criteria–the first consists of five criteria covering what the business area does (the enablers: leadership; people; policy and strategy; partnership and resources; and process), and the second consists of four criteria on what the business area achieves (the results: people results; customer results; society results; and key performance results). Evidence based on feedback from focus groups, questionnaires and personal interviews is used to score performance, and a resulting action plan for improvement is introduced which is then included in the business plan.
12.85 Underlying the EFQM Excellence Model is the realization that business excellence–measured through customer satisfaction–is achieved through effective leadership which drives policy and strategy, allocates resources compatible with that policy, and manages employees in such a way as to enable them to manage the processes.
12.86 In the case of national statistical institutes, where some procedures are governed by statute or regulation, the use of the EFQM Excellence Model enables continuous improvement to be taken forward across a range of processes and functions. To work effectively, it needs the commitment of senior managers, who must be responsible for leading any self-assessment. However, unlike ISO 9000, where assessment is carried out by qualified auditors often from outside the work area (see below), the EFQM Excellence Model relies on the input from all staff.
12.87ISO 9000. The International Standard ISO 9000 is an international quality standard for management systems (ISO, 1994). A quality system is a commonsense, well-documented business management system that is applicable to all business sectors. It helps to ensure consistency and improvement of working practices, including the products and services produced.
12.88 The ISO standards were fully revised as ISO 9001 in November 2000 to match current philosophies of quality management and views regarding the structures that need to be in place to ensure that continuous improvement is maintained (ISO, 2000).
12.89 The revised standards give users the opportunity to add value to their activities and to improve their performance continually by focusing on the major processes within the organization. They will result in a closer alignment of the quality management system with the needs of the organization and reflect the way the organization runs its business activities. By meeting the ISO 9000 standard, an organization will come more into line with TQM and the EFQM Excellence Model.
Scope for greater use of quality management techniques
12.90 Both ISO 9000 and the EFQM Excellence Model have received a great deal of international recognition over recent years. At the same time, the use of benchmarking networks has also grown in prominence. It is therefore pertinent to ask whether more coordinated use should be made of these and other quality management techniques at a strategic level in fields of statistics where the focus is on international comparability. This is particularly so with statistics that are compiled for treaty purposes, for example by member States of the European Union following detailed methodological guidelines laid down in law.
12.91 The arguments are fivefold:
It is paramount that such important non-optional statistics whose production and uses are enshrined in legislation have the full trust of users.
The quality of international comparisons is dependent on the weakest link, thus good-quality statistics from one country may be of little value if not matched by statistics of equally good quality from other countries.
There is a potential for misleading analysis and conclusions arising from differences in the application of standard methodology.
Empowerment in ensuring the establishment of adequate control processes is reduced when production is delegated to member States.
There is limited scope for centralized validation and quality management when production is decentralized.
Performance management, development and training
12.92 An effective performance management system for individuals is just as important as applying such a system to management structure. Performance management can be seen as a continuous process designed to improve work outputs by focusing on what people actually achieve rather than the amount of effort they put into the work. It should provide the link between the objectives of the individuals, those of their team and those of the wider organization, so that workplans are coherent across the organization, and everybody knows what they are doing and why they are doing it. The performance management system should provide clear objectives for monitoring and evaluation, to enable feedback on performance and also to assist with the identification of the development needs of individuals. Performance management should be continuous.
12.93 Effective training will help motivate staff and equip them to deliver a good-quality CPI. At its simplest, training will give a background understanding of the nature and uses of the index and how it is compiled. Training and development takes many different forms and may include:
–tutoring by the line manager or supervisor;
–attending an induction course or reading a manual;
–accompanying an experienced price collector.
12.94 A written training plan is useful in identifying training and development needs in relation to the organization’s goals and targets. It can also be used to identify the resources required to deliver the training to meet these needs, and to evaluate whether the training has been delivered effectively and objectives have been met.
Specific training for compilers and collectors
12.95 Further training will be required for specific skills, depending on the roles of the individuals and their jobs. Training should continue beyond the induction stage to cover changed procedures, and retraining where performance is unsatisfactory.
12.96 Price collectors will need to be trained specifically in field procedures, including relations with shopkeepers, the selection and definition of a valid price, special rules for certain individual items (including seasonal items), how to complete forms and, where appropriate, how to use hand-held computers. Compilers of the index will need to be trained specifically on validation procedures and consistency checking, the calculation of centrally collected indices, weighting procedures and how to aggregate prices, as well as on the treatment of seasonal items and special procedures relating to some sections (e.g. housing). It may also be beneficial to provide training in local or national trading or statistical regulations, economics, and commodity information.
12.97 Significant benefits may result from the interaction between price collectors and index compilers. Benefits will also be gained from liaison between the national statistical institute and commodity experts from industry. Such experts can advise on issues such as how to identify quality features on particular items, for example electrical goods, personal computers, or clothing and footwear.
12.98 It may be beneficial if statisticians from headquarters are personally responsible for supervising price collection in the area where the head office is situated, so that they can have first-hand experience of the problems involved. This will put them in a position to provide assistance where difficulties arise. Equally, it is a good idea to arrange for regular visits to headquarters by groups of collectors and their supervisors. It is good for morale. Price collectors will, arguably, do a better job if they feel that they belong to a team, if they can see that their work is appreciated and if they feel that their problems are understood. Visits to headquarters will help convey that the accuracy and conscientiousness of their contribution is recognized as being crucial to the quality of the index. Visits to head office by price collectors also will help the statisticians to keep in touch with conditions in the field and, for example, to obtain more information about new goods and aspects of quality change.
12.99 Similarly, compilers of the index may wish to visit the field occasionally and participate in or simply observe the price collection. This will provide them with a better appreciation of the practical problems associated with price collection and a better feel for data (and in consequence for the quality of the index), together with the skills required to help with price collection in the event of an emergency.
12.100 A manual and other documents such as desk instructions may serve for initial training. Later on such documents should enable the collectors and compilers to remind themselves of all the relevant rules and procedures. The documentation should be well organized and well indexed so that answers to problems can quickly be found.
12.101 Documentation should be checked by all concerned and updated regularly. The pile of pieces of paper containing amendments should never grow large, but should be replaced by a new consolidated version. One way of achieving this is to have a loose-leaf manual so that individual pages can be replaced whenever necessary. Another option is to keep an electronic version that can be updated by nominated individuals. It is important that the updating of documentation is done in a systematic and controlled way. A variety of software is available to help the statistician to do this.
12.102 The benefits of using standard electronic software for documentation are threefold:
–more efficient production of documentation, because the software helps with the initial compilation of information and reduces the need to print and circulate paper copies;
–better-informed staff, because they have immediate electronic access to the latest documentation, including desk instructions, with a search facility by subject and author;
–better quality control, because authors can readily amend and date-stamp updates and because access to non-authors is restricted to “read only”.
12.103 Training may be seen as an essential part of continuous quality improvement. Staff may be invited to operational reviews where all team members have the opportunity to raise concerns and, where appropriate, tackle specific issues through individual or group training.