Chapter

12. Organization and Management

Author(s):
International Monetary Fund
Published Date:
September 2004
Share
  • ShareShare
Show Summary Details

A. Introduction

12.1 The producer price index is used for many purposes by government, business, labor, universities, and other kinds of organizations, as well as by members of the general public. Aprefix-worduracy and reliability are paramount for a statistic as important as the PPI. Whether the PPI is used as a deflator of national aprefix-wordount values, an indicator of inflation, in escalation of contracts, in revaluation of fixed assets or stocks, or in other economic analyses, the process of producing the PPI needs to be carefully planned.

12.2 Individual circumstances vary to such an extent that this Manual cannot be too prescriptive about timetables or critical path analysis of all the steps involved. However, the description in this chapter provides an outline of the kinds of activities that should result from a detailed examination of the logistics of the whole periodic operation of compiling the index.

12.3 The following guidance seeks to present some options in the organization of data collection. The examples given are based on experience and provide an indication of goals a country may seek. In recognizing these options, this chapter, which talks about organization and management of PPI procedures, covers the relationships between the price collectors (who may be stationed at regional offices in large countries) and PPI staff at the central office (covering the work carried out in the central office, the flow of information among each part of the organization, and related activities for coordinating collection and processing data). Because of the size, frequency, cost, or complexity of the collection of prices as the basis of the index, in some countries not all these operations and relationships will be appropriate.

B. Initiation of the Price Collection Process

12.4 This process involves PPI staff visiting individual businesses drawn from a sample to establish cooperation, stress the importance of the index, and receive basic information, such as the exact goods and services produced by the business, relative importance of transactions with various clients, individuals to contact on a recurrent basis, and so on. The range and number of businesses visited and the types of goods and services priced will vary among countries. In some countries these operations may be conducted by telephone. Once this process has commenced, a questionnaire for the price collection may be designed. (SeeChapter 6, Section D, for information on questionnaire design.)

12.5 Although the precise method of current price collection will vary, each price collector will usually be responsible for collecting from a certain business or from certain types of businesses. This may enable the collector to specialize in certain subject areas of the index. Collectors will contact the same businesses in each collection period to attempt to price the same transactions of goods and services. Price collection is usually done monthly or quarterly, but the frequency can change if the prices for certain transactions change at known intervals. For example, goods or services with prices usually subsidized or regulated by government will change prices when government action is taken. These prices may be collected directly by PPI staff in the main office based on external information such as contact with other government offices or through the media. In any case, checks must be in place to ensure all price data are reported. (SeeChapter 6, Section B, for information on timing and frequency of price collection.)

B.1 Mode of price collection

12.6 One of the decisions facing any statistical agency carrying out a price collection program is whether to use in-house staff or tender the collection to an external organization. For example, another part of the agency, another government department that specializes in surveys, or a private market research company could performt his function.

12.7 The nature of the price collection and the distribution and profile of statistical staff may help determine whether the collection is suitable for contracting out to another agency or even the private sector. Where price collection is continuous, involves complicated decision making (such as quality adjustment), or is collected from a small number of businesses, it may be advantageous to keep the collection in-house. However, if the collection takes place over just a few days per month from a large number of businesses, is relatively straightforward, and involves only routine or simple decision making (perhaps selecting from a list of codes), then contracting out to another agency can be considered. For example, if the statistical office does not have a dedicated data collection staff, it could contract with other agencies such as commerce, industry, and agriculture to collect PPI data. Another possibility could be a private research company, if there are market research companies with suitable skills existing in the country. The statistical office must also take confidentiality requirements into consideration when contracting with another agency to guarantee that there are no breaches in confidentiality. This may involve national statistical laws that address the issue of data collection by contractors and enforcement of penalties for breaking confidentiality requirements.

12.8 Contracting out price collection can lead to lower costs, because the statistical office is no longer responsible for overheads such as collector pensions. When price collection is carried out using electronic methods such as computer-assisted telephone interviews (CATI) or computer-assisted personal interviews (CAPI), the responsibility for purchasing and maintaining data-capture devices may also be transferred to the contractor.

12.9 Contracting out may also allow statistical office staff to spend more time analyzing data rather than collecting it. By separating the role of data collector and data checker, statistical staff can feel more comfortable questioning the validity of price data. The aprefix-worduracy of collected data can be linked to the performance of the contractor throughperformance measures, which drive incentives payments (and penalties if targets are not achieved).

12.10 The same considerations may be used when deciding whether the survey division or PPI staff should conduct the price collection. Usually, some mixed mode of operation will be in place. Staff from the survey division may handle straightforward and routine price collection; more complicated and specialized industries such as chemicals and semiconductors will require price collection by specialists, whether from PPI staff or consultants of a statistical office.

C. Quality in Field Data Collection

12.11 Quality is an important part of price collection; a high-quality price collection enables a statistical agency to have confidence in the index it produces and ensure that observed price changes are genuine and not the result of collector error. Procedures must be developed to ensure that a high standard of collection is maintained for every collection period. These procedures will form the basis of collector training and should be included in any training material developed for price collectors. Guidance should cover price index principles, organizational issues, and validation procedures. For a discussion of the components that should be in a training regimen, statistical offices should review Finkel and Givol (1999), which includes both technology and price collection methods. Additional training requirements for statistical offices appear in Section F.1 of this chapter.

C.1 Training

12.12 The statistical office should have a general training program for staff working on the price programs. There are four basic components of such a program.

12.13 First, fundamental (basic) training must provide information on how to collect data, code data elements, review and edit basic price data, and compile collected data to produce indices. In addition, the training should impart to staff information on the purposes and uses of the collected prices.

12.14 Second, the program should highlight the need for continuous training of staff at all levels. Staff should provide feedback at all levels—from respondents to data collectors and from supervisors to staff. There should be regularly scheduled meetings between staff and supervisors at all levels to assess the program and identify current and potential problems.

12.15 Third, statistical offices also need to provide professional training for staff in computer technology, economics, statistics, and even psychology (for dealing effectively with respondents).

12.16 Fourth, annual seminars or retreats for staff can be effective in discussing the strengths and weakness of the program during the previous year and in planning for the upcoming year. This is particularly true when the program undertakes major changes in index methods, new weights, new sample designs, and so on.

C.2 Transaction descriptions

12.17 Accurate price transaction descriptions are critical in ensuring price transaction continuity. Descriptions should be comprehensive to ensure that collectors or reporters can price the same transaction in each collection period. Collectors must record all information that uniquely defines the price transaction selected. So, for example, in price collection for production of clothes, color, size, and fabric composition must be specified to ensure that the same price transaction is priced each month.

12.18 Accurate price transaction descriptions will assist the price collector, respondent, and PPI staff in choosing a replacement for a price transaction that has been terminated and will also help to identify changes in quality. PPI staff should be encouraged to spend some time, each collection period, going through reported descriptions to ensure that the correct price transactions are being priced. Collectors or respondents should also be encouraged to review their descriptions to ensure that they contain all the relevant information, and it may be useful to ask collectors oprefix-wordasionally to switch collections with another collector so that they understand the importance of comprehensive descriptions.

C.3 Continuity

12.19 Continuity is one of the most important principles of price collection. Because a price index measures price changes, the same price transaction must be priced every month so that a true picture of price changes is established. It is not possible to be prescriptive because the concept of equivalence will vary among countries, but for practical purposes a detailed description of the price transactions must be kept. Some guidelines may be drawn up by the statistical office’s PPI Head Office staff to cover different price transactions. All transaction specifications, such as the same purchaser, similar delivery terms, valuation of currencies, changes in subsidies, tax laws, and so on must be met.

12.20 Collectors or respondents should report prices at similar times within each collection period. This is particularly important when pricing volatile price transactions with sharp fluctuations.

C.4 Data entry queries

12.21 Once the price data are correct and complete, a series of validation checks may be run. In deciding which checks should be carried out, take into aprefix-wordount the validation checks carried out in the field, whether by price collectors in the regional office, survey division officials in the main office, or by PPI analysts. For example, CATI will increase the potential for validation at the time of price collection and reduce the need for detailed scrutiny by PPI Head Office staff. It would not be productive or cost effective to repeat tests.

12.22 The range of tests carried out for all collection methods may include the following:

  • (i) Price Change: The price entered may be compared with the price for the same defined transaction in the same business in the previous month and queries raised where this is outside preset percentage limits. The queries may vary depending on the price transaction or group of price transactions and may be determined by looking at historical evidence of price variation. If there is no valid price for the previous month, for example, because the produced good was out of stock and no transaction could be made, the check can be made against the price two or three months ago. The price may also be compared with other transactions conducted by the same business in the current month.

  • (ii) Maximum/Minimum Prices: A query may be raised if the price entered exceeds a maximum or is below a minimum price for group of goods or services of which the particular product is representative. The range may be derived from the validated maximum and minimum values observed for that price transaction in the previous month expanded by a standard scaling factor. This factor may vary among price transactions.

12.23 If computer-assisted techniques are used, these tests can be easily implemented to take place at the time of collection; otherwise, they will need to be conducted by PPI Head Office staff as soon aspossible after collection and before prices are processed on the main system. A failure in the CATI or CAPI should not result in collectors being unable to price the price transaction, but it should prompt them to check and confirm their entries and prompt for an explanatory comment.

12.24 Queries raised may be dealt with either by staff analysts at the PPI Head Office or by the price collector or respondent contacted for resolution. For example, scrutiny of a form might show that a big price difference has arisen because the transaction priced was a new product replacing one that had been discontinued. In this case, there may be no need to raise a query with the price collector unless there is evidence to suggest that labeling the transaction as a “new product” is incorrect.

12.25 When an error is discovered too late in the process to resolve, PPI Head Office staff will need to reject it and exclude that price transaction from that month’s index. The price transaction must be excluded from the base month so that the basket is kept constant. (SeeChapter 6, Section F, for more details on the verification process.)

C.5 Feedback

12.26 When price collectors are used, they should be encouraged to give feedback to PPI Head Office staff on their experiences. Collectors are a valuable source of information and often give good early feedback on changes in the different industries. Collectors can often warn of size or product changes before the PPI Head Office staff can derive this information from other sources, such as trade magazines or the business press. Collector’s feedback can form the basis of a collector newsletter and can support observed price movements and provide supplementary briefing material. Significant changes in price transactions within a business may require an additional visit by PPI analysts to the business to update the price transaction descriptions

D. Quality Checks in Price Collection

D.1 Role of auditors

12.27 The routine of collecting prices in the field (where the “field” represents an array of collection methods;(seeChapter 6, Section D.3) needs to be carefully planned and monitored with arrangements to reflect local conditions. Circumstances vary, and it is not appropriate to be too prescriptive. Some of the measures mentioned below may be irrelevant if PPI analysts in the Head Office collect the prices centrally. However, when data are collected locally and sent into the Head Office or reported directly by the businesses in the sample, price collectors and contacts must send in data on time. If data are not timely, it is necessary to find out the reason and take appropriate action. It is also important to check that the information sent in is aprefix-wordurate and complete.

12.28 One way to monitor the work of price collectors is to employ auditors to oprefix-wordasionally aprefix-wordompany collectors during field collection—whether data are collected by phone or personal visits or to carry out a retrospective check on data that have been collected.

D.1.1 Monitoring data collection

12.29 If an auditor intends to aprefix-wordompany a price collector during a personal or telephone interview, he or she must inform the collector in advance to arrange meeting details. In general, the auditor will not monitor the entire price collection process but will spend a few hours observing the price collection in a specific location. For example, it may be necessary to observe the collection of certain price transactions or in particular businesses where collection might be problematic, based on analysis of the price trends in the past. Special workstations in a regional or main office may be set up for auditors who listen and track a price collector using a computer-assisted method such as CATI.

12.30 Before monitoring, the auditor will need to carry out preparation work—a premonitoring check. Such a check could involve looking at descriptions, prices, price history, and indicator codes of the price transactions collected in the particular business or section of the index. This type of check will enable the auditor to understand the standard of collection before going into the field and may suggest in which areas of the collection the auditors should concentrate their efforts.

12.31 An auditor’s main duty is to ensure that the price collector is following proper procedures and instruction and performing the collection competently. While the auditor may not have the role of a trainer, he or she may give some coaching to correct any errors. The collector should use this opportunity to ask the auditor relevant questions.

12.32 Following a monitoring visit, auditors should compile a report detailing their observations. This report should include a summary report of findings, issues for action, and a recommended course of action. Auditors may advise that a collector receive extra training on certain aspects of the price collection, and PPI Head Office staff (or the contractor, if the collection has been outsourced) should act on this. The auditor’s report will be usedas a starting point on the auditor’s next visit. In other instances, general problems may arise when solutions need to be disseminated to all price collectors, perhaps by issuing revised instructions or through a newsletter or other written or electronic means.

D.1.2 Backchecking and process auditing

12.33 Another approach to monitoring the standard of price collection is to carry out a “backcheck”—a retrospective check of a proportion of the prices recorded during the collection. Backchecks can be used to

  • (i) Assess the competence of overall individual price collectors;

  • (ii) Audit the general standard of price collection;

  • (iii) Identify general training needs or the specific needs of an individual;

  • (iv) Highlight any key issues, such as problems with documentation and instructions issued by PPI Head Office staff; and

  • (v) Identify areas where collection is problematic—for example, if all collectors have problems in certain types of businesses—demonstrating the need for more detailed PPI Head Office staff instructions.

12.34 Backchecking could be done by an expert independent of the process (employed by the statistical office when price collection is outsourced). It is carried out by contacting the business(es) selected and recollecting the prices and other relevant information. This activity should be carried out at about the same time as the original collection period to avoid problems of price changes oprefix-wordurring in the interim. Backcheckers must explain to the contact person at the business the reason for the check and stress the importance of his or her participation. The response by the business will usually be favorable.

12.35 For a backcheck to be useful, the results must be compared with preestablished performance criteria. These criteria should set, for example, the aprefix-wordeptable number of price errors per number of price transactions checked. Well-defined criteria will enable easy identification of a poorly performing collector or section of the index.

12.36 The need for a backcheck may be triggered by

  • (i) Price differences—if different, the auditor should see if there has been a price change since the original collection took place;

  • (ii) Insufficient price transaction description—the auditor should determine if it is uniquely defined so that another collector can replicate the process;

  • (iii) Wrong price transaction priced, such as incorrect details of the transaction being chosen; and

  • (iv) Price transactions wrongly recorded as missing or temporarily out of stock.

12.37 A report should be sent to the PPI Head Office staff for scrutiny once the backcheck has been completed. The Head Office will then need to take appropriate action, which may include, for example, retraining or sending out supplementary instructions.

12.38 Auditing and backchecking are important ways to improve quality, but there is a trade-off between this and the burden on business imposed by the audit process. For example, businesses are likely to object to being asked for the same information twice (once as a check). There are, of course, ways to audit collection without imposing extra burdens on business—for example, by monitoring telephone conversations with businesses and examining the quality of the data and supporting information received by different analysts.

D.1.3 Other auditor functions

12.39 The range of tasks auditors carry out will vary from one statistical agency to another, and monitoring the standard of price collection will always be their main focus. However, there are a number of other areas to which auditors can be called to contribute.

12.40 Auditors may be required to help with initiating the price collection process and price transaction sampling. Auditors can also carry out other work. For example, if a particular price transaction is causing difficulty for price collectors, auditors can speak to collectors and businesses and find out why. Auditors who work on CPI and PPI can also lead to more consistency among the indices, and they can advise both CPI and PPI Head Office staff on availability of goods and services and other economic activities that may be of importance.

12.41 Auditors who learn that a particular business is participating in numerous surveys of the statistical agency can inform the Head Office of this information to help the agency find ways to reduce respondent burden or better coordinate collection at the business.

D.2 Quality checks by PPI Head Office staff

12.42 Regular quality checking is recommended, as necessary, to

  • Ensure that the price collectors’ reports are sent in when they are due. If not, it is necessary to find out the reason and take appropriate action;

  • Confirm that the reports contain what they are supposed to contain—that is, that required fields have not been left blank, that numeric fields contain numbers, and that nonnumeric fields do not;

  • Review and edit each return. Substitutions may have to be made centrally, or those made by the collectors may have to be approved. Unusual (or simply large) price changes may need to be queried. Transactions priced in multiple units or varying weights may have to be converted to price per standard unit. Missing prices must be dealt with aprefix-wordording to standard rules relating to the cause; and

  • Identify and correct errors introduced when keying the numbers into the computer or transcribing them onto worksheets.

12.43 As stated above, logical checks conducted in the field by an automated process can reduce the number of checks and errors handled by PPI Head Office staff.

12.44 Note that the way the data are organized in worksheets or in the computer may differ from the way they are collected in the field. Their origin should, however, be recorded so that reference to them can be made should processing disclose any problems. Even if codes provided to the collectors to list price transactions and to describe or qualify the prices are unchanged in processing, other codes may have to be used for information that comes in from the collectors in noncoded form. How the checking is organized will vary from country to country. In some cases, local or regional supervisors will do some of it; in other cases, it will be more appropriate for it all to be done centrally.

12.45 Some of these tasks can be done by computer; others, manually. Therefore, no general suggestion can be made about the sequence of the work or about its division into different parts. Procedures should be in place to check that all documents, messages, or files are returned from the field so that price collectors can be contacted about missing returns. Initial checks should then be carried out to ensure that data are complete and correct. If any prices fail these checks, a query should be raised with the price collector for clarification. Since some of the checking may require reference back to the price collectors (or to their supervisors or respondents when direct mail questionnaires are used), the timetable for producing the index must allow for this communication to take place.

12.46 Following the price data checks, a series of validation checks may be run. In deciding which checks should be carried out, aprefix-wordount should be taken of the validation checks carried out in the field. For example, computers will increase the potential for validation at the time of price collection and reduce the need for detailed scrutiny at the PPI Head Office. In addition, it would not be productive or cost-effective to repeat the tests already carried out, except as a secondary audit or random check.

D.2.1 Reports

12.47 Reports (on paper or computer) should be generated routinely for most representative price transactions. Reports help the analyst pick out particular prices that are different from those reported for similar firms elsewhere or that lie outside certain specified limits. A computer printout can list all cases that either fall well outside the range of prices obtained earlier for that representative price transaction or that show a marked percentage change from last time for the same price transaction in the same business. The limits used will vary from price transaction to price transaction and can be amended. The analyst can study the printout, first ascertaining whether there has been a keying error, then examining whether the explanation furnished by the collector adequately explains the divergent price behavior, and finally determining whether a query should be sent back to the supervisor or collector. The timetable should allow for this step, and anomalous observations should be discarded when an aprefix-wordeptable explanation or correction cannot be obtained in time. (Also seeChapter 9, Section D, on editing data.)

12.48 Other reports may be regularly produced on the basis of several periods (or months) reporting (to detect aprefix-wordumulated patterns) that will enable broader problems to be detected. For example,

  • One collector’s reports might show many more “not available” remarks than those of other collectors, perhaps indicating either a motivational or training need on the part of that collector, or a change in retail trade patterns in a particular area;

  • Substitution for a particular representative price transaction might become more numerous than before, suggesting a possible need for revision of the specification or the choice of another representative price transaction; or

  • The dispersion of price changes for a particular representative price transaction might be much larger than it used to be, raising the question of whether it has been appropriately specified.

12.49 The routine computer–generated reports should enable those in charge of the index to detect all such problems.

D.2.2 Dispersion index report

12.50 This dispersion index report is a list of price transactions with the current index for each price transaction, number of valid quotes for each price transaction, and the range of price relatives. The index dispersion prints can identify situations with price relatives that fall outside the range of the main bulk of quotes. These quotes can be identified and investigated and appropriate action taken if necessary.

D.2.3 Quote report

12.51 The quote report consists of a range of information on a price transaction that the index dispersion report has highlighted as warranting further investigation. Information listed may include current price, recent previous prices, and base price, together with similar quotes from other reporting businesses. The report can be used to identify the quotes that require further investigation and also to investigate rejected prices.

D.2.4 Algorithms

12.52 Algorithms can be created that identify and invalidate price movements that differ significantly from the norm for a price transaction. For some seasonal price transactions for which price movements are erratic, it may be more appropriate to construct an algorithm to look at price level rather than price change.

12.53 Although algorithms can be an efficient way to highlight problematic data, a word of caution should be expressed about using them. Analysts will want to assure themselves that their use does not result in systematic bias in the index. This issue may also need to be addressed in any editing routines (as presented in (Chapter 9, Section D), although it is less likely to be problematic in the context of manual editing.

E. PPI Production and Quality Assurance

E.1 Organizational structure and responsibilities

12.54 Statistical offices could adopt a number of organizational models for effective work. In deciding on the appropriate organizational structure, statistical offices should take into aprefix-wordount the following:

  • The need for clarity of reporting lines;

  • The need for a clear division of responsibilities;

  • Centralized or decentralized management of fieldwork;

  • Production management versus technical development; and

  • Compatibility with corporate structures in the National Statistical Institute, for example, in relation to quality management, methodological research, and dissemination.

12.55 In some cases—for instance, when little inhouse expertise in fieldwork practices exists—it may be advantageous for fieldwork to be conducted by a different organization in either the public or private sector. In these circumstances, an effective contractual relationship must exist with the data collection agency. There should also be agreed delivery targets and performance measures to cover such items as data delivery timetables, response rates, and levels of aprefix-worduracy. Consideration should also be given to the independent auditing of the contractor’s work on a sample basis. It is worth noting that even if fieldwork is conducted in a different division within the organization, a contractual relationship between fieldwork and main office functions can alleviate tension and improve the quality of a PPI.

E.2 Monthly compilation

12.56 The system used for the regular computation of the index must be sufficiently flexible to allow for changes in the kind of data obtained. A modular or mix-mode approach may be seen as an advantage.

12.57 Analytical computations provide comparisons between the published index (or indices) and what an alternative index would have produced using different methods or data. They help explain the relationship with subindices, characterize the movements in the index over time, and allow methodological experimentation. The following examples of such investigations make clear some of the computational capabilities for analytical indices:

  • Alternative aggregations of subindices;

  • Alternative computations of indices for seasonal goods;

  • Effects of different weights; the effects of introducing newly significant product categories;

  • Price updating of weights;

  • Number and duration of missing observations; how a different method of estimating them would affect the index;

  • Comparison of indices computed with various subsamples of the data as a means of estimating variance; variances of price ratios;

  • Computation of a Standard Reference Index (one with no explicit quality adjustments) so that an Implicit Quality Index is obtained;

  • Numbers of sampled products; rates of forced replacements; lengths of time products remain in sample; and

  • Frequency distributions of quality adjustments.

12.58 To examine such matters, the database must contain not only prices but price transaction descriptions, details of product replacements, explanatory remarks attached to observed prices, information on the data suppliers, and so on. Generally, historical databases are too large to be stored live on the system and need to be archived. However, analysis of seasonal trends requires 12 months of data on current computer systems. Detailed documentation relating to the archived material will need to be kept to guard against loss of vital information caused by changes in computing staff or computers. Consideration should also be given to appointing a data custodian with responsibility for all archived records.

E.3 Spreadsheets

12.59 Spreadsheets may be used for compiling subindices that require special procedures or when data are collected by different methods than the main method or technique. A spreadsheet has the advantage of additional flexibility and scope for combining responsibility for data collection, data input, and computation. The compilers’ specialized knowledge (they usually are PPI analysts in the Head Office) about the markets or businesses where these prices may be observed, combined with analytical tools applied to the spreadsheet, will help them detect any irregularities in the data, facilitate investigation of whether these reflect reporting or input errors, and allow rapid rectification. A compiler can jump between numerical data entry and a chart—displaying, for example, current-month and previous-month entries—and this helps the rapid and simple detection of anomalies, and he or she can then follow-up with the data supplier. As time passes, the resolution of problems that have arisen and adaptation to new circumstances will result in changes to the spreadsheet. Unless quality management controls are put in place, the spreadsheet may be unclear and improperly documented. If so, two unfortunate consequences can arise.

  • If that developer is absent, retires, or moves to another job, the successor will find it very difficult to maintain the continuity and quality of the spreadsheet.

  • New procedures introduced to deal with new circumstances may be inconsistent with procedures used for other subindices for which other people are responsible.

12.60 Good documentation and communication with colleagues will diminish these risks. At a minimum, the spreadsheets and changes to them must be understandable with adequately explicative row and column headings or notes attached to headings. Furthermore, changes in procedures or formulas, rebasing, and application of new weights should always be introduced by moving computation to a new sheet within the workbook, not by modifying the old sheet. The new sheet and the old sheet will then exist side by side so that they can be compared. Passwords can prevent inadvertent changes to cells containing formulas and can lock cells containing input data once editing is completed. Only people with authority to edit the spreadsheets should know the passwords. Regular backup by copying the whole workbook to another disk is also essential. Working experience with spreadsheets in a PPI has revealed some other important quality management controls:

  • Spreadsheet designs should be as similar as possible,

  • PPI analysts should present these to each other on a regular basis, and

  • At least two analysts should verify the results in the spreadsheets.

E.4 Monthly consultations

12.61 PPI managers in the main headquarters may find it useful to convene monthly consultations with their analysts before publication of the index. Although various checks have been implemented throughout the process (fieldwork, editing, compilation of subindices, etc.) added value is obtained by comparing results of different economic branches in the current month. Analysts could be asked to present areas of significant change in the index (whether by percentage points or relative importance) and explain the reasons that led to these changes, such as changes in the local economy, global prices, exchange rates, institutional or government intervention, and so on. Countries may find it useful to convene all the analysts of price indices on a monthly basis to compare changes in the different markets.

E.5 Introducing changes

12.62 Various checks should be carried out when introducing changes in a PPI. These may include a comparison of the old and new basis using data from parallel running of collections—for example, when handing over to a new collection contractor, division, or individual within the organization. Checks may include reestimating backward—for example, when new base prices are being imputed for a complete range of goods or services. Any anomalies can then be investigated further.

E.6 Disaster recovery

12.63 Price indices are important and highpprofile statistics produced by a statistical office and can affect a wide range of users. There may be a legal obligation for the index to be published within a short time period after the end of the relevant month. Many contracts within an economy may be linked to the indices, whether consumer prices or producer prices. Any delay in publication can have significant effects on subsequent months, threatening future publications. With significant delays, it could take months to return to the existing tight publication timetables. It is critical, therefore, that statistical offices develop a robust and tested Disaster Recovery Plan (DRP), however unlikely the need to implement it may appear. There are a number of possible causes of disaster:

  • (i) Failure of an external contractor to fulfill obligations to supply information when the data collection is contracted out to a private company,

  • (ii) Failure of the computer system, and

  • (iii) Major natural disaster or other event (for example, terrorist activity) affecting the operations center or PPI Head Office staff.

12.64 When the whole operation of the PPI (including fieldwork) is conducted in-house by the statistical office, the DRP for the organization will already include special procedures to ensure the continuance of PPI production in times of disaster. If price collection is contracted out, the first cause of disaster mentioned above may become relevant. Therefore, the DRP may include arrangements for alternative fieldwork operations—whether by outsourcing the procedures to a third party or utilizing in-house capability (if such exists).

F. Performance Management, Development, and Training

12.65 Equally important to organizational structure in production of a PPI is the ability to ascertain an effective performance management system for individuals. Performance management can be seen as a continuous process designed to improve work outputs by focusing on what people actually achieve rather than the amount of effort put into the work. It should provide the link between the objectives of the individual and those of the team and the wider organization so that work plans are coherent across the organization and everybody knows what they are doing and why they are doing it. The performance management system should provide clear objectives for monitoring and evaluation to enable feedback on performance and also to assist with the identification of the development needs of individuals. Performance management should be continuous.

F.1 Training requirements

F.1.1 Introduction

12.66 Effective training will motivate staff and equip them to deliver a good-quality index. At its simplest, training will give a background understanding of the nature and uses of the index and its compilation. Training and development take many different forms and may include

  • Tutoring by the line manager or supervisor,

  • Attending an induction course or reading a manual, and

  • Accompanying an experienced price collector.

12.67 A written training plan can be useful in identifying training and development needs against the organization’s goals and targets. It can also identify the resources required to deliver these needs and evaluate whether training has been delivered effectively and objectives have been met.

F.1.2 Compilers and collectors

12.68 Further training will be required for specific skills depending on the roles of the individuals and their jobs. Training should continue beyond the induction stage to cover changed procedures and include retraining when performance is unsatisfactory.

  • (i) Price collectors will need to be trained specifically in field procedures, including relations with businesses, selection and definition of a valid price, special rules for certain individual price transactions (including seasonal price transactions), how to complete forms, and, where appropriate, how to use computers.

  • (ii) Compilers of the index will need to be trained specifically on the validation, consistency checking, and calculation of centrally collected indices; weighting procedures and how to aggregate prices; and treatment of seasonal price transactions and special procedures relating to some sections of the index.

  • (iii) It can also be beneficial to provide training in local or national trading or statistical regulations, culture, and commodity information.

12.69 Significant benefits can result from the interaction between price collectors and index compilers. Benefits will also be gained from a liaison between statistical offices and experts from industry, who can advise on issues such as how to identify quality features, and so on, on particular price transactions such as electrical goods, personal computers, or clothing and footwear.

12.70 It can be beneficial if statisticians from headquarters are personally responsible for supervising price collection (at least for certain parts of the index) so that they have firsthand experience of the problems involved and provide assistance when difficulties arise. It is also a good idea to arrange for regular visits to headquarters by groups of collectors and their supervisors. It is good for morale, and price collectors will, arguably, do a better job if they feel that they belong to a team. They can see that their work is appreciated and their problems are understood. It will help convey that the aprefix-worduracy and conscientiousness of their contribution is recognized as being crucial to the quality of the index. Visits to PPI Head Office staff by price collectors will help the statisticians keep in touch with conditions in the field and, for example, obtain more information about new goods and aspects of quality change.

12.71 Compilers of the index also may wish to visit the field oprefix-wordasionally and participate in or observe the price collection. This will provide them with a better appreciation of the practical problems associated with price collection, a better feel for data and index quality, and a greater understanding of the skills required to help with price collection in the event of an emergency. In a PPI, this may be of great importance, especially when collecting price transaction data in more complicated economic branches.

F.1.3 Documentation

12.72 Manuals and other documents such as desk instructions may serve for initial training and later on should enable the collectors and compilers to remind themselves of all the relevant PPI rules and procedures. Documents should be well organized and well indexed so that answers to problems can quickly be found.

12.73 All concerned should check the documentation and update it regularly; the pile of paper containing amendments should never grow large and should be replaced by a new consolidated version. One way of achieving this is to have a loose-leaf manual so that individual pages can be replaced whenever necessary, or to keep an electronic version that can be updated by designated individuals. It is important that the updating be done in a systematic and controlled way. A variety of available software programs can help the statistical office. The benefits of using standard electronic software for documentation are threefold:

  • More efficient production of documentation, because software helps with initial compilation and reduces the need to print and circulate paper copies;

  • Better-informed staff, because they have immediate electronic access to the latest documentation, including desk instructions with search facility by subject and author; and

  • Better quality control, since authors can readily amend and date-stamp updates, and access is restricted to “read only” nonauthors.

F.2 Reviews

12.74 Training can be an essential part of continuous quality improvement. Staff may be invited to operational reviews where all team members have the opportunity to raise concerns and, where appropriate, tackle specific issues through individual or group training.

G. Quality Management and Quality Management Systems

12.75 Statistical offices are faced with the continuous challenge of providing a wide range of outputs and services to meet user, that is, customer, needs. Thus, a key element of quality is customer focus and the effective dissemination of relevant, accurate, and timely statistics. In addition, a quality program should include effective customer education on the use of such statistics. In these terms, success can be measured by the achievement of a high level of satisfaction among well-informed users. The IMF has developed the Dissemination Standards Bulletin Board (dsbb@imf.org), which provides dissemination standards and a data quality reference site. The data quality reference site also includes a framework for assessing price statistics systems that is contained in Reports on the Observance of Standards and Codes (ROSC) Data Module for a number of countries. Statistical offices can use the ROSC Data Module to assess their price statistics programs.

12.76 For the quality management of a PPI, it can be argued that the priority area is quality control of the production process itself. For most statistical offices, this will be an area that represents a high risk, given the complexity of the process and the financial implications of an error in the index. If the principles of organizing and managing the collection of data, and subsequent processing of information to produce a PPI, are to be adopted, then it is vital that a quality management system be in place. This will ensure that the data obtained, the processes involved in achieving the specified outputs, and the formulation of policies and strategies that drive them are managed in an effective, consistent manner. The data systems should, wherever possible, be open to verification and mechanisms put in place to ensure outputs meet requirements—in other words, customer satisfaction.

12.77 Taken together, the above-listed elements form the basis of a quality management system. Varying perceptions about the meaning of quality exist, but an important common thread is the requirement to react to and serve user needs and to ensure continuous improvement in them. Thus, the implementation of an effective quality management system requires a high level understanding of what customers need and the translation of this need into a coherent statistical and quality framework. Such a framework is also necessary for putting together criteria for judging success. User needs can be canvassed either formally through negotiation of contractual obligations or less formally through talking to customers on a one-on-one basis or through customer surveys. In many countries, issues relating to the governance of the statistical office are set down in a framework document or similar document. This defines the functions and responsibilities of the statistical office and generally guides and directs the work of the office. For instance, an objective stated in the framework document to “improve the quality and relevance of service to customers—both in government and the wider user community” provides a powerful statement for determining workplans. This recognition of the importance of quality can be further endorsed by a published vision of the national statistical institute as a key supplier of authoritative, timely, and high-quality information. This can be encapsulated in published objectives in an annual business plan. These objectives can include improving quality and relevance, thereby increasing public confidence in the integrity and validity of outputs. Performance can be measured against a combination of a number of factors, including accuracy, timeliness, efficiency, and relevance.

12.78 There are a number of examples and case studies of quality systems in practice that illustrate how different models may be applied. Some models may be more suitable than others, depending on the exact mode of PPI operations in different countries.

G.1 Quality management systems

12.79 A number of best-practice standards can be exploited to help organizations improve quality management, some of which have the added advantage of being internationally recognized.

G.1.1 Total quality management

12.80 Total quality management, or TQM, is most closely identified with a management philosophy rather than a highly specified and structured system. The characteristics associated with TQM and an effective quality culture in an organization include

  • Clearly defined organizational goals,

  • Strong customer focus,

  • Strategic quality planning,

  • Process orientation,

  • Employee empowerment,

  • Information sharing, and

  • Continuous quality improvement.

G.1.2 Benchmarking

12.81 Benchmarking is a process of comparing with, and learning from, others about what you do and how well you do it, with the aim of creating improvements. The Australian Bureau of Statistics has been particularly active in this area and undertook an exercise in 1998-2000 in partnership with the United Kingdom. Benchmarking projects have also been undertaken in New Zealand, the United States, and Scandinavian countries. Areas that can be considered when benchmarking a PPI collection may include

  • Timelines, accuracy, and coverage of collection;

  • Benefits of index methodologies for various price transactions, for example, geometric mean versus average of relatives;

  • Frequency of collection and publication; and

  • Cost of collection per unit of commodity, staff, and so on.

G.1.3 European Foundation for Quality Management Excellence Mode1

12.82 The European Foundation for Quality Management (EFQM) Excellence Model is a selfassessment diagnostic tool that is becoming widely used by government organizations across Europe to improve quality and performance. It may be described as a tool that drives the philosophy of TQM. It focuses on general business areas and assesses performance against five criteria covering what the business area does (the enablers: leadership, people, policy/strategy, partnership/resources, process) and four criteria on what the business area achieves (the results: people results, customer results, society results, key performance results). Evidence based on feedback from focus groups, questionnaires, and personal interviews is used to score performance, and a resulting action plan for improvement is introduced, which is then included in the business plan.

12.83 Underlying the EFQM Excellence Model is the realization that business excellence measured through customer satisfaction is achieved through an effective leadership, which drives policy and strategy, allocates resources compatible with that policy, and manages employees to manage the processes.

12.84 In the case of statistical offices, where some procedures are governed by statute or regulation, the use of the EFQM Excellence Model enables continuous improvement to go forward across a range of processes and functions. To work effectively, it needs the commitment of senior managers who must be responsible for leading any selfassessment. Unlike the international standard ISO 9000 (see below), however, where assessment is carried out by qualified auditors often from outside the work area, the EFQM Excellence Model relies on input from all staff.

G.1.4 ISO 9000

12.85 The International Standards Organization (ISO) established an international quality standard for management systems—ISO 9000. The quality system is a commonsense, well-documented business management system that is applicable to all business sectors and that helps to ensure consistency and improvement of working practices, including the products and services produced. The ISO standards have been fully revised to match current philosophies of quality management and to provide the structures needed to ensure continuous improvement is maintained.

12.86 The revision of these standards (as of the year 2000) gives users the opportunity to add value to their activities and to improve their performance continually by focusing on the major processes within the organization. ISO standards will result in a closer alignment of the quality management system with the needs of the organization and reflect the way those organizations run their business activities. It will therefore come more into line with TQM and the EFQM Excellence Model. G.2 Need for quality management in statistics

12.87 Both ISO 9000 and the EFQM Excellence Model have received a great deal of international recognition over recent years. Benchmarking networks also have gained prominence. It is therefore pertinent to ask whether more coordinated use should be made of these and other quality management techniques at a strategic level in fields of statistics where the focus is on international comparability. This is particularly so with statistics such as those in the Harmonized Index of Consumer Prices, which are compiled for treaty purposes by member states of the European Union following detailed methodological guidelines laid down in European law.

12.88 The arguments are fivefold:

  • (i) Such important nonoptional statistics with production and uses enshrined in legislation must have the full trust of users across the European Union;

  • (ii) The quality of international comparisons is dependent on the weakest link; thus, goodquality statistics from one country may be of little value if not matched by statistics of equally good quality from other countries;

  • (iii) The potential for misleading analysis and conclusions arising from differences in the application of standard methodology;

  • (iv) The reduction of empowerment in ensuring the establishment of adequate control processes when production is delegated to member states; and

  • (v) The limited scope for centralized validation and quality management of decentralized production.

G.3 Specific quality management models in a PPI

12.89 PPI operations may differ from those of a CPI in several areas. The distinction between fieldwork procedures and complexity of continuity in the samples of goods and services can lead to implementation of different models in a PPI and CPI. For example, the PPI involves telephone interviewing and self-administered postal questionnaires, while the CPI largely uses personal interviewing. Also, the PPI basket has goods and services produced by specific firms according to economic branch, while the CPI basket contains many goods and services that are standard and can be purchased at many outlets. Therefore, several countries have adopted quality management models that may enhance representativity of the index. For example, the ABS undertakes an ongoing Sample Review and Maintenance Program process that is concerned with

  • Adequacy of the sample of respondents,

  • Adequacy of the specifications priced,

  • Appropriateness of the pricing basis underlying the reported prices, and

  • Accuracy of reported prices.

12.90 The U.S. Bureau of Labor Statistics conducts a Structured Schedule Review (SSR) that serves as a cornerstone of the quality control program in its PPI. The assumptions that underlie the development of this system are

  • (i) Survey quality is largely determined at the data initiation stage; and

  • (ii) Quality-related problems are associated with various causes, such as faulty procedures, inadequate training, and imprecise collection forms or uncontrolled operator errors, and they require an SSR system to assist in diagnosing the source of error.

    Other Resources Citing This Publication