XVIII. Survey (Collection) Design
- International Monetary Fund
- Published Date:
- March 1995
837. This chapter and the one that follows are primarily concerned with the development of an ITRS or of ES to collect BOP data. This chapter focuses on principles of survey (or collection) design, and chapter 19 deals with form design. However, collections for several other types of data sources—official sources, households, and ITS—are not discussed. While surveys to obtain data from official sources could be designed on the basis of principles described in these two chapters, the development of collection methodology for the official sector is not normally complex. Collections of household data and of ITS, both of which are used by BOP compilers, are typically undertaken by other compilers.
838. Effective survey design for an ITRS and for ES requires: well-defined objectives, sufficient legislative authority, thorough coverage of the population or the activity being measured, appropriate methodologies, well-designed survey forms, responsive reporters, good statistical processing procedures, appropriate levels of resources, and responsiveness to the needs of users. All of these requirements are addressed in chapters 18-21.
839. Every survey should have a clearly defined conceptual framework that is harmonized with the conceptual frameworks of related statistical collections. A conceptual framework should be developed in accordance with international standards, such as those contained in the BPM, and tailored to meet national requirements.
840. In addition, users (whose requirements may vary with time) of the data that are collected should be actively involved in the development process to ensure that their concerns are being addressed. Compilers should meet with primary and secondary users in order for the users to articulate their criteria in terms of frequency, timeliness, accuracy, and level of detail. Priorities and various options should be discussed—particularly when user expectations exceed the scope of resources available to compilers—and users should be asked to support necessary increases in such resources. Also, as users often have different perceptions and concerns, compilers may have to make judgments among conflicting requirements. Alternatively, compilers could encourage the formation of ad hoc or permanent user committees to develop more uniform views. Formal surveys may also be employed to reach a broad range of users.
841. Legislation giving compilers the authority to collect required data is necessary for both an ITRS and for ES. Such legislation creates a legal obligation (and an appropriate penalty for noncompliance) for reporters to provide the information, allows compilers to examine (should the need arise) the accounts or records of reporters, stipulates confidentiality of information reported (so that commercial and personal data are available only to compilers of BOP and related statistics), and facilitates publication of data.
842. National statistical offices, which are usually responsible for ES, are often endowed by national legislation with appropriate authority to implement such surveys. In a number of countries, exchange controls provide sufficient authority to undertake an ITRS. Many countries that have abolished exchange controls have also introduced special legislation in order to continue with the ITRS.
843. Sufficient legislative authority is imperative for effective BOP collection. If appropriate legislation does not exist, it must be created. Once a legislative basis for compulsory reporting and confidentiality of data has been established, reporters will cooperate more readily. When relevant legislation is insufficient or nonexistent, reporters may assume that compliance is voluntary, that requests for certain data exceed the mandate of the compiling agency, or that responding to the survey is not important.
844. Compilers should not refrain from using legislative authority to enforce compliance. Awareness of compiler assertiveness and willingness to use prosecution and fines, if necessary, may encourage recalcitrant reporters to comply. However, legislative enforcement authority should be used only when all other avenues for obtaining data have been exhausted. Appropriate legislation also places obligations on the compiler, especially with regard to protection of confidential data, and compilers should be scrupulous in ensuring that these obligations are met. When BOP statistics are published, it is sometimes desirable to reveal or comment on a particular large transaction. Therefore, the legislation should permit compilers to comment if the information is already publicly available or the agreement of reporters can be obtained. In any event, relevant reporters should be consulted.
Selection of an Appropriate Data Source
845. Selection of the most appropriate data source is by no means straightforward. The choice is affected by many factors, including compatibility of the source with BOP concepts, the timeliness with which data are provided, the frequency with which data are available, the quality of data coverage, usability of the data in BOP compilation, resource costs to the compiler, the respondent burden to the data provider, and the legal authority of the compiler.
846. Generally, no data source will rate best according to these criteria. The BOP compiler should determine, in consultation with users, which factors are the most important and make a decision accordingly. Also, the choice of an appropriate data source will often require trade-offs.
847. Collection of data from enterprises is predicated on determination of the statistical unit with responsibility for supplying data. For this purpose, it is necessary to ensure that all entities involved in an activity are uniquely identifiable; otherwise, there is scope for omission and duplication.
848. International guidelines, such as the SNA, distinguish unit levels in relation to business activity—namely, the establishment, the enterprise (the legal entity), and the enterprise group. The term establishment typically refers to an entity involved in activity at a particular location, such as a shop, a factory, or a mining site. The concept of location must be relaxed when establishments with some activities, for example—transportation, are defined. An enterprise consists of the operations of the legal entity resident in one country—for example, a corporation or a branch of a nonresident company. The enterprise group or, as the SNA refers to it, the family of legal entities is the operations of a set of legal entities that are resident in one country and connected in a parent-subsidiary relationship.182 An enterprise group includes entities that are subsidiaries of a nonresident parent company.
849. The enterprise group is the statistical unit preferred by many BOP compilers conducting ES of BOP activity because many international transactions are organized on this basis. For example, an enterprise group may have a central organization that handles the external financial transactions of the group. The use of the enterprise group, rather than the enterprise, as the statistical unit often reduces the workload for the compiler and for the reporting community. However, if BOP activities are handled in several centers within a group, arrangements can be made to collect data from each center, and another statistical unit (such as the enterprise) may be more appropriate. An important qualification is that, if an enterprise group covers more than one institutional sector (for example, a group may consist of a bank and nonbank financial enterprises), arrangements should be made to collect separate data in respect of each sector. Therefore, the preferred statistical unit is the enterprise group at sector level. For a compiler using an ITRS, the preferred unit may be the enterprise. However, it may be desirable to link enterprise records to an enterprise group record.
The BOP Register
850. The BOP register is a set of records containing information on economic units that are included, or have the potential to be included, in BOP surveys. The term population frame is often used synonymously with the BOP register.
851. If units in the register are stored at the enterprise group level, the register should include some information on company group structures. This information is important if an enterprise is identified as potentially suitable for inclusion in the BOP register. In these cases, the compiler must know whether or not the enterprise is part of a group that has already been identified. An enterprise register for an ITRS could also record information on a group basis even though the collection (survey) may use the enterprise as the statistical unit. The register should include unit name, address, contact officer, telephone and facsimile numbers, and area and size of BOP activity.
852. While the BOP register is a logical database, it need not be physically separated from other statistical registers. For example, some national statistical offices maintain a single register for all surveys, including BOP collections.
Sources for a BOP Register
853. There are many sources that a compiler could use to prepare a BOP enterprise register. The following examples are not exhaustive.
854. A list of enterprises may be compiled from an ITS list of exporters and importers of goods. Apart from exports and imports, these enterprises may have external claims and liabilities—particularly in respect of trade credit and, in the case of importers, loans for imports. The enterprises may also be users and suppliers of international services—transportation, in particular.
855. An ITRS may be used to compile a list of enterprises. Enterprises that should be included in a BOP register may be identified by domestic banks in an ITRS. However, enterprises that settle transactions through accounts with nonresident banks may have to be identified from other sources.
856. Government licensing and regulatory agencies may also be sources of enterprise units for a BOP register. These sources may include official foreign investment agencies that accept applications from nonresidents to establish or expand direct or portfolio investment in the host country or applications from resident enterprises to invest abroad. A similar source is an agency, usually the central bank, that provides approval to buy or sell foreign exchange. This source may be useful for identifying new loans from nonresidents when amounts borrowed are used to make payments directly abroad. The central bank should also be a good source for identifying financial institutions involved in international transactions, and insurance regulatory bodies may be able to identify insurance enterprises, agents, and brokers undertaking international insurance activity. Agencies issuing import and export licenses or trade commissions that maintain lists of exporters could be useful sources if the information is not available from ITS. Agencies that regulate access of nonresident transport operators to the domestic economy may have lists of such operators and their domestic agents. Regulatory bodies that supervise the travel industry may provide lists of enterprises involved in that industry.
857. The tax department may provide information on resident entities paying withholding taxes on behalf of nonresidents, foreign-owned enterprises (direct investment enterprises) granted taxation holidays or requiring permission to expatriate profits, and enterprises with income from foreign sources.
858. Corporate affairs and stock exchange authorities generally require certain information from companies being registered or listed. Such information often includes the names of larger (including nonresident) shareholders. These authorities may be good sources for identifying enterprises that are direct investment enterprises or that otherwise have nonresident shareholders with large portfolio holdings. These institutions may also be approached to identify issuers of bonds and other forms of securities.
859. Industry associations may provide list of members involved in certain activities, such as trade in services and various international financial transactions.
860. The financial press often provides a wealth of information that can trigger further inquiries by the compiler. Financial news is a particularly important source for obtaining up-to-date information on major BOP transactions that should be identified and followed up quickly.
861. The compiler may add to BOP collection forms one or two simple exploratory questions on BOP activity not covered by survey forms.
862. Information may be obtained from nonresident sources. For example, through the use of lists published by foreign authorities, the compiler may be able to monitor securities issued by residents in foreign markets.
Developing and Maintaining the BOP Register
863. The development of the BOP register, which is shown in illustration 18.1 on page 190, may be regarded as a two-part activity. In the first phase, enterprises with potential for BOP transactions are identified via sources previously described. These enterprises are then compared with units already listed on the register.
Illustration 18.1Register Maintenance for an Enterprise Survey
864. The compiler should make every effort to identify all units with potential for BOP transactions and be satisfied that nothing significant is missed. It is also important for the compiler to provide feedback to suppliers of source information. He or she should not hesitate to query a supplier of source data and to suggest better ways of collecting or presenting information.
865. In the second phase, more information is obtained on enterprises that were identified from initial sources and are not already on the register. It is unlikely that all enterprises identified will be entered in the BOP enterprise register because some units will not be engaged in activities of interest. An exploratory survey may be used to discover what, if any, BOP activities the enterprises are involved in and the size of those activities. Model form 1 in appendix 2 is an example of a form that could be used for an exploratory survey. In such surveys, the form should be kept simple so that collection and processing costs are minimized.
866. Enterprises identified (from the exploratory survey) as being engaged in BOP activity are entered on the register. (In illustration 18.1, a line extends from the exploratory survey to the register.) The register then becomes the source list for enterprises to be included in BOP collections. (Examples of types of collections that may be made from enterprises on the register are shown in illustration 18.1.) The register also serves as the source list for future exploratory surveys that determine whether characteristics of enterprises on the register have changed over time.
867. Many sources will provide information on the size of enterprise activity. For example, a list of importers from ITS may classify importers by size of importing activity in a specific time period. A threshold can be established for including enterprises in the exploratory survey—particularly if source lists are large. (However, the compiler may wish to add units of particular significance directly to the register, rather than including them in the exploratory survey, so that these units participate in BOP collections as soon as possible.) Units below the threshold may be recorded on supplementary lists. Ideally, sample surveys of these enterprises would be undertaken to ensure that the units are not engaged in activities of a size to be measured by collections conductedfrom the register.
868. Implementation of the exploratory survey may reveal problems with source data; for example, a number of duplicates may be found, or it may be difficult, in practice, to identify all units listed by a source.
869. Enterprises determined (from the exploratory survey) not to be engaged in BOP activity should be recorded on a list of residual enterprises and monitored. These enterprises should be approached via future exploratory surveys approximately once every five years.
870. Once the BOP register is established, it should be updated and extended as necessary. The compiler should keep abreast of developments taking place in the economy—for example, by reading the financial news. He or she should also be alert to any major changes among sources used for initial identification of enterprises with potential BOP transactions. If the compiler becomes aware of major transactions being undertaken by newly identified enterprises or by enterprises previously identified but not on the register, he or she should immediately record the enterprises on the register and include these in relevant collections.
871. Also, the compiler should undertake periodic exploratory surveys of all entities on the BOP register. Units that are on the register but not actually included in BOP collections—for example, those below thresholds—could be approached every two years. All such entities could be approached at the same time or, for example, one quarter of the register could be approached every six months. This periodic exploratory survey would reveal any changes that should be made to information on the register and, in particular, identify changes in activities and sizes of enterprises recorded there.
872. Register sources should be reviewed progressively, and lists of units newly identified from these sources should be checked against the existing register. Unless more immediate action is required, information on newly identified units could be obtained by including them in the next exploratory survey.
873. An ITRS should contain a number of basic features if it is to produce statistics of good quality. The ITRS should:
meet BOP conceptual requirements;
cover all BOP cash transactions using foreign or domestic currency and be supplemented to measure noncash transactions, including reinvested earnings attributable to direct investors;
include enterprise transactions settled through accounts with nonresident banks;
encompass procedures to measure stocks of external financial assets and liabilities;
provide for reconciliation of stocks and flows at the enterprise (or enterprise group) level. For this purpose, recording the identity of larger transactors is important.
874. A closed ITRS, prima facie, has an advantage over an open or partial system because, in the former, it is possible to reconcile bank positions with the measurement of BOP transactions. Nevertheless, many built-in checks may be provided in an open or a partial system, and these can provide assurance of data quality.
875. Collection forms should be well designed so that data are correctly recorded. An important and often difficult part of data collection is the classification of transactions. The reporter should provide information that is sufficient to ensure correct coding. A system in which the reporter describes and codes the transaction produces the best results—if the coding is checked by the compiler. whatever approach is adopted, it is essential that the compiler be closely involved with coding transactions. It is important to ensure that settlements covering a number of BOP categories or settlements representing both payments and receipts are recorded on a gross basis and that all underlying BOP transactions are identified.
876. Within the framework of the ITRS, the compiler may collect data from alternative sources. The most obvious example is the choice between recording the purchase and encashment of travelers checks’ by individual travelers or recording traveler check settlements between banks and nonresident correspondent banks. Therefore, it is necessary to have clear rules as to who should report what and which transactions should not be reported.
877. An ITRS is likely to produce a large number of records on transactions—a result that may make this type of collection somewhat costly. Costs may be reduced without affecting quality if thresholds are introduced and used sensibly. If thresholds are used, two conditions are desirable. First, some information on transactions falling below the threshold should be collected and aggregated. Second, transactions falling below the threshold should be examined from time to time—for example, through sample surveys. The examination should be made to determine appropriate classifications for transactions so that data on transactions occurring above the threshold may be supplemented with data on small transactions that are appropriately classified. If these two conditions apply, relatively high thresholds can be used without jeopardizing quality.
878. An important component of a good ITRS is contact between the compiler and the data providers—particularly those enterprises engaging in international transactions of large value. Such interaction facilitates correct classification of transactions and monitoring of individual enterprises so that data can be checked and verified and the compiler kept abreast of developments affecting the BOP.
879. Because of the complexity of an ITRS and the volume of transactions covered, a large computer processing system is usually required. To calculate resource requirements in this area, it is important to quantify: (a) the volume of records to be processed; (b) the average number of characters per record to be entered and stored; (c) the numbers of interrogations and tabulations to be submitted and the frequency of submission; and (d) the number of staff necessary to organize the system efficiently. Paragraphs 1085-1097 of chapter 20 provide information on the development of computer processing systems.
880. Many international transactions reporting systems require large numbers of processing staff to check, code, and enter data. Staff numbers can be reduced significantly if sophisticated computer processes are used—in particular, electronic transmission of data from provider to compiler. The work of processing staff should be monitored to identify and correct any errors. In some systems, every coder’s work is checked by another. This procedure may be expensive, and the mere checking of one’s work by another may not identify all errors.
881. More effective are quality control procedures that tolerate a minor level of error while identifying significant errors and causes thereof. Procedures for checking all large transactions and a sample of smaller transactions should be developed. The checkers should be highly skilled staff. If the error rate on a sample of checked records exceeds the acceptable level for an individual coder, an additional, larger sample of the coder’s work should be checked. If the error rate on that sample is also found to be beyond tolerance, remedial action should be taken—including, in the extreme case, the recoding of entire batches. This type of quality control procedure is more likely to detect individual weaknesses, improve coder skills, and enhance data quality than complete checking procedures. Such an approach is also applicable to ES.
882. Compiler requirements for detailed, timely, and accurate statistics should be emphasized when an ITRS is being developed. The compiler should establish priorities in these areas and the collection strategy’ should be chosen in accordance with these priorities. For example, the requirement for timely statistics may be best satisfied through judicious use of estimation techniques, which will obviously have an impact on collection strategy.
883. Selection of collection strategies for ES depends on a number of factors.
884. One of these factors is segmentation of population. The population may be segmented so that certain homogeneous groups can be easily identified. For example, a few foreign-owned petroleum companies may be a dominant group in an economy, and these may be isolated as one group for collection purposes. Banks and certain other financial institutions could be similarly isolated.
885. Another factor is distribution of population. Enterprises involved in international transactions often show a skewed distribution—that is, relatively few enterprises in an economy are involved in international transactions and even fewer are involved in international transactions in a significant way. Usually, an enterprise must be fairly large before it develops a foreign market for its output or before it decides to deal directly with nonresident principals. This factor has led to the use of thresholds by many compilers.
886. In practice, compilers in many countries use a combination of two or three approaches—census, partial coverage, or sample—when conducting ES. A census is a collection that includes all members of the population. A partial coverage collection includes enterprises with BOP activity or external assets and liabilities above certain thresholds. (Those below the thresholds are excluded.) In a sample survey, enterprises are selected according to rigorous sampling procedures, and the results of the sample are expanded by using simple number raising or ratio estimation techniques.
887. Large transactors must be approached each time a partial coverage collection is conducted. It is therefore important that the list of large transactors be kept up-to-date. Use of partial coverage collections can provide cost savings without much loss of quality.
888. If the partial coverage approach is used for BOP surveys, compilers should develop methods to measure, at frequent intervals, the contribution to BOP activity of all members of the population. These measurements could be made by using an exploratory survey, a sample survey of smaller units, or a benchmark census. An exploratory survey can be used to collect broad information on the size of BOP transactions of individual transactors in the population frame. This information could be used directly to estimate the impact of units not surveyed in the partial coverage collection, or it could be used indirectly in the creation of a framework for a sample survey. Because sample surveys are relatively inexpensive, they can be conducted at frequent intervals—for example, annually—and can provide high quality factors for expanding results from partial coverage surveys. Another advantage with sample surveys is that the impact of non-response is reduced.183 The disadvantage with sample surveys is the presence of sample (or standard) error.
889. Efficient sampling procedures seek to keep both the number of units selected and the sample error to a minimum. These objectives are usually achieved by stratifying the population—that is, grouping population components according to size of selected activity so that units in different strata have different probabilities of selection. Two factors predominate in the determination of sample error. One is sample size; the larger the sample, the smaller the sample error. The other is the variability of the activity being measured; the wider the dispersion of the activity, the greater the sample error. Population size is not an important factor unless the population is very small or the sample size approaches the size of the population. By using stratification techniques, the compiler essentially increases sample size for units likely to have large absolute variability in size of activity. That is, relative sample size increases as variability increases. In fact, the most variable units, which are typically the largest, are often fully enumerated—a circumstance that means there is no sample error for this stratum.
890. In addition to supplementing partial coverage surveys, sample surveys can also be used as the primary survey method. For example, compilers in some countries use sample surveys as the principal source of information on international trade in selected services.
891. Compilers in many countries choose not to use a sample survey approach to supplement partial coverage surveys. Instead, they use benchmark censuses to establish the contribution of smaller units. These censuses are usually costly and undertaken infrequently. Therefore, revisions to results may be made at greater intervals. However, benchmark censuses typically provide more detailed information than other approaches and also establish whether or not some entities, which should now be included in the completely enumerated partial coverage sector, have changed size during inter-census periods.
892. Enterprise surveys are usually conducted by mail, telephone, or personal interview. The choice of collection method in BOP surveys often combines elements of a mail survey and personal contact (telephone and face-to-face interviews). BOP collections usually require complex accounting information from large enterprises—a requirement that, in turn, calls for survey reporters to be well educated and trained in accounting. Fortunately, ES are usually directed to central accounting units or international departments of enterprises, which usually contain highly qualified staff. Because of survey complexity, collection by mail is often appropriate; comprehensive instructions can be provided and the data provider given time to assemble the information. For smaller enterprises that have less of an impact on overall results, a simpler form could be used without a resulting loss of quality. BOP collections are not necessarily comparable to surveys that approach a greater number of small businesses, less sophisticated accounting units or other departments within enterprises, or households in which the general level of education and form completion skills may not be as high.
893. Many ES contain an extensive number of form types. There may be exploratory and regular survey forms. Different forms may be used to collect different data; for example, separate forms may be used for goods, transport, travel, other services, compensation of employees, external assets and liabilities and associated financial and income flows, and certain security transactions conducted by intermediaries. Different forms may be used for different types of units; for example, enterprises in the banking and transport industries may receive forms that differ from those sent to other enterprises. Also, different forms may be sent to direct investment enterprises, direct investors, enterprises that are both direct investors and direct investment enterprises, and other enterprises. Different form types may be used for smaller enterprises or for those that report less frequently.
894. While there are many advantages to tailoring BOP forms to particular enterprises, proliferation of form types should be avoided. Too many BOP form types can confuse respondents and processing staff alike, lead to complicated procedures that are difficult to manage, and cause inconsistencies in collection of data.
895. A common issue is whether to send one form covering all aspects of the BOP activity of an enterprise or a number of forms, each of which covers a different aspect. The former approach simplifies reporting and collection procedures. However, if the information required is likely to come from different parts of the organization or be available at different times, the second approach may be preferred. In any case, the compiler should involve data providers in the decision-making process.
896. All forms should conform to a set of standards. That is, forms should use a common set of model questions and instructions, although there may be departures from these to meet individual circumstances. If the model questions and instructions are modified, these modifications should be implemented in all collection forms. All forms should have a proper numbering system, and the place of each within the collection scheme should be clear.
897. Form design is discussed in detail in chapter 19.
898. Volatility of the activity being measured may affect the frequency with which data should be collected. Assigning percentages to the proportion of an activity that should be collected quarterly rather than annually (or less frequently) is difficult unless one knows the national situation. In the case of financial transactions, the compiler might aim to collect, for example, 95 percent of gross transaction flows on a quarterly basis and to measure the remainder with annual or less frequent surveys. Because financial flows are volatile, it is difficult to make adjustments to account for partial coverage, particularly when net flows are being measured. Hence, nearly complete coverage of the activity is required on a frequent basis. Less frequent collection may be sufficient for surveys measuring less volatile activity. For example, for certain service transactions that are relatively constant over time, the compiler may aim to collect only 50 percent of the activity on a quarterly basis and cover the rest at annual or even less frequent intervals.
899. The compiler should establish historical coverage ratios and use the ratios as a basis for estimating the non-surveyed component. A coverage ratio is the ratio of the value contributed by enterprises that report quarterly (or whenever) to the value for the total population. If there is a stable relationship from period to period, the percentage of activity surveyed frequently could be kept relatively low. The objectives of using the coverage ratio are to operate cost effectively and to prevent the necessity for large revisions at a later date. In all cases, the compiler should ensure that new units or units changing levels of activity are monitored and added to surveys as appropriate.
900. If thresholds are used for an enterprise survey, it will be necessary for the output to incorporate partial coverage factors based on coverage ratios.
901. Choice of processing method typically depends on computing facilities available. For small and simple collections, a personal computer with off-the-shelf spreadsheet and/or database software could be used. For bigger applications, a tailor-made database computer processing system may be required. The system should be designed in accordance with principles outlined in paragraphs 1085-1097 of chapter 20. To determine the type of processing system required, it is necessary to examine the elements of the system in some detail. These elements are summarized in illustration 18.3 and discussed in paragraphs 910-918 of this chapter.
Illustration 18.3Enterprise Survey Processing System
902. To satisfy the criterion for timeliness, the compiler may have to release preliminary statistics that suffer from less than complete response. In this situation, the compiler must determine which reporters should be targeted for quick and complete response and which reporters may be followed up after preliminary results are released. The survey compiler should also adjust data received from reporters by making estimates for non-response.
903. Use of estimates, rather than raw data, may be preferable when some segments of data cannot be checked prior to preliminary release and there is no guarantee that raw data are correct. If the survey and BOP compilers are different persons, the survey compiler—who is familiar with collection procedures and their impact on results—may be in a better position to develop these estimates than the BOP compiler.
904. Different levels and frequencies of output may be required by different users. Consequently, more detailed tables may be produced and published at less frequent intervals. Output requirements should be determined in consultation with users.
International Transactions Reporting Systems
905. Illustration 18.2 on page 196 shows the primary processing activities in a typical ITRS.
Illustration 18.2ITRS Processing Activities
906. Illustration 18.2 depicts a representative system; actual systems may use somewhat different approaches. The diagram shows three types of basic input: bank client forms (completed by bank clients), bank reports (completed by banks), and enterprise reports (completed by enterprises in respect of foreign currency accounts with resident banks, accounts with nonresident banks, noncash transactions, and stock positions of external assets and liabilities). Bank client forms are checked by banks that receive the forms. These forms, as well as reports completed by banks and enterprises, are submitted to the BOP compiler. Forms not coded by data providers are coded at this stage. Initial validation, an important step is performed to identify any obvious errors, such as non-completed fields or inaccurate coding. Data are then entered in the database and subjected to various quality control procedures.
907. Quality control procedures may include: (1) checking the conversion rate between the foreign currency value and the domestic currency value if both amounts are reported; (2) checking the comparability of patterns of transactions reported by enterprises from period to period; and (3) listing large transactions likely to affect overall results. Reporting banks or enterprises may be queried about large transactions; responses may result in amendments to the database. Another quality control procedure consists of reconciling reported positions and flows for individual reporting banks and for individual enterprises. This procedure involves collating data from all sources and examining residuals—activities that may, in turn, lead to data queries and amendments. Transaction values may then be converted to the domestic currency if the value in domestic currency was not collected.
908. The process of summarizing records and analyzing aggregates should include estimation for non-response and any ratio expansion used to take thresholds into account. The analysis may reactivate some quality control procedures, which may—in turn—generate new queries and amendments. New aggregates may have to be generated; this is a reiterative process. (Further information on verifying BOP data can be found in chapter 20, paragraphs 1067-1083.) Results are released after the compiler is satisfied with the quality of the data.
909. The diagram in illustration 18.2 shows a link between the BOP enterprise register and the unit record database. Data from the register may be used to classify transactions by sector and industry. Enterprise reports may provide additional information, such as name changes, for the BOP enterprise register. Also shown in the diagram is the important link between bank client forms and the BOP register. This link demonstrates the matching of transactions data to enterprises and the identification of new enterprises for inclusion on the register.
910. Illustration 18.3 on page 196 shows how an enterprise survey processing system may function. The diagram shows that a collection register for any survey period is derived from the BOP enterprise register. This enterprise register, in turn, is the outcome of register maintenance. All enterprises included in a collection should be on the collection register.
911. The collection register is an essential tool for monitoring and controlling the dispatch and collection of data. The collection register should pass relevant information, such as response history, from one collection period to the next. The register can be a paper document or, in more sophisticated systems, a computer database. For each unit in the collection, the register records information such as the name of the enterprise in the collection, the contact name and telephone number, the form type (if more than one type of form is used), the dispatch date, the due date (and any extensions that may have been granted), the timing and nature of follow-up action, any special information that the compiler should know about the enterprise—for example, response history and special reporting arrangements, records of any telephone conversations, the date the form is received, and the initials of the compiling staff member responsible for accepting the form. The collection register provides important input for the dispatch and collection control process, which is shown, along with a pro forma for a collection register, in illustration 18.4 on page 197.
Illustration 18.4Example of a Collection Register
912. As illustration 18.4 indicates, Ms. Loi, the reporter for ABC Incorporated, apparently required some encouragement to return the completed form but otherwise is a good reporter. The enterprise received form A1 and, on this occasion, was given an extension to April 21. Unfortunately, Ms. Loi did not meet that deadline and was reminded by facsimile on April 22 and by telephone on April 24. The form was received the next day. A staff member with the initials GH undertook the follow-up action and signed off on accepting the form. More details on the follow-up action were recorded on the enterprise’s file; presumably, the space on the register was insufficient to record the follow-up action. The sign-off function is important, and the compiler may wish to introduce more columns to ensure that this function is undertaken correctly. Before signing off, staff should ensure that the form has been correctly completed and that any changes to the name and address of the enterprise, the contact name, the form type, and any special information are recorded in the BOP register.
913. In illustration 18.3, a feedback line extends from the dispatch and collection control process to the collection register and from the collection register to the BOP register. The feedback line reflects the updating of the registers with new or amended information obtained from the dispatch and collection control process.
914. Once collection forms are received, they should be checked for obvious omissions and errors and coded before the data are entered into the database (data entry). Initial validation would include confirming that all requested items are completed and checking for any simple errors or invalid codes. Initial validation may result in office corrections or querying of the reporter. However, complicated checks that are best performed by computer should be left to the quality control stage.
915. Data entered in the database are usually subject to a series of quality control measures. The checks that could be undertaken are discussed in paragraphs 926-927 of this chapter and involve more intense examination of data reported by individual reporters.
916. When the compiler is satisfied with the quality of data reported for individual enterprises, aggregation can be undertaken. Aggregation should, if appropriate, include estimates for partial coverage and non-response. The information necessary to make such estimates may come from the BOP register, which may contain information on numbers and sizes of enterprises, or from a statistical analysis of results from exploratory or benchmark surveys. If a sample survey is used, sample estimation would have to be undertaken. Quality control checks should also be performed at the aggregation stage. These checks are discussed in paragraphs 1067-1083 of chapter 20.
917. Aggregate results deemed acceptable may be entered in a publication database containing data for all periods (historical time series). Thereafter, data may be disseminated. Chapter 21 provides information on publishing and disseminating BOP statistics.
918. As the collection process may also involve amending data for previous periods, unit record databases for previous periods may also be affected. These amendments may, in turn, affect the publication database.
Developing Relationships with Data Providers
919. Personal contact between compilers and reporters is the key to collection of useful and accurate data. When establishing collections, compilers should discuss the collection parameters and the form design with representatives from a large group of enterprises. These initial discussions often lead to mutual understanding, between compiler and reporter, of respective requirements. Such understanding is essential for good collections. Reporters should be encouraged to get in touch with the compiler if there are any issues about which they are uncertain. Contact can be facilitated by having the compiling organization’s telephone numbers highlighted on collection forms and by having a responsive staff member accept calls and answer queries promptly and correctly. The compiler should initiate contact—by conducting personal interviews in more complex cases and by telephone in more straightforward cases—to follow up with reporters who have not responded to a questionnaire. Similarly, the compiler should initiate queries when a reporter has misreported or reported an unusual and/or a large transaction about which the compiler requires further information. The compiler should conduct (preferably in person) regular evaluation interviews to discover how well reporters understand the collection form, the extent to which they comply with instructions, and whether or not reporters have any recommendations for modifying the collection form or procedures.
920. The compiler should respect reporters but not be intimidated by them. The compiler should feel comfortable about taking the reporter into his or her confidence and giving—whenever possible—correct, complete, and honest answers to questions. In fact, the better the reporter’s understanding of what is required, the more helpful he or she is likely to be. If a reporter asks a difficult question and the answer is not readily apparent, the compiler should not be afraid to admit that he or she does not know the answer. Effort should be made, however, to find the correct answer (if it is available) and provide a prompt reply. BOP methodology is evolving, and new information supplied by collection reporters may help further the evolution. Reporters may offer worthwhile insights into the BOP, and the compiler should not have reservations about discussing fundamental BOP issues with reporters.
Dealing with Non-response
921. Although previous collections may have been successful and follow-up action intensive, less than 50 percent of the forms currently due may be returned by the due date. If follow-up procedures have not been sufficient, the initial response rate for ongoing surveys is likely to be even lower. New collections can also be expected to have low response rates. Many enterprises may request additional time in which to complete the form. Extensions to the due date should be granted sparingly; they should be actively negotiated and not passively given. Extensions should be given for only one or two periods; the expectation that the reporter will not need them in the future should be stated; and several days, rather than several weeks, should be granted.
922. The follow-up of non-responding units should be timely, persistent, and persuasive—but never abusive. With limited resources, follow-up action should be targeted. For example, as soon as possible after the due date, facsimiles may be sent to large enterprises with forms outstanding to remind the reporters that responses are overdue and to ask them to indicate when the forms will be submitted. If this approach is unsuccessful, the staff of the enterprises should be contacted by telephone. The compiler should be prepared to negotiate. If some data are of high priority, it may be acceptable for the reporter to provide key aggregates by telephone or facsimile and for the complete form to be returned by a specified future date. For smaller enterprises, a more low-key approach may be appropriate. For example, the compiler may send reminder letters and only follow up by telephone at a later date. Nevertheless, non-response of these enterprises should be followed up in all cases.
923. Experience has shown that most lack of cooperation comes from concern over confidentiality, embarrassment stemming from failure to understand a form, presumption that completion of the form would be overly time-consuming, or poor or negative experiences with other government units. If a survey reporter is proving difficult, some of the following questions may help:
“Some reporters are reluctant to give information because they are concerned about confidentiality. Is this a concern of yours?”
“While we do our best to make the form clear, many reporters find it difficult to follow. Is there some part of it that you find difficult?”
“Some reporters do not have all the information to complete the form. Is this a concern of yours?”
These questions, coupled with someone prepared to listen, often melt icebergs or quench fiery furnaces.
924. If appropriate legal authority exists, the compiler should be prepared to resort to legal action—that is, to take an enterprise to court for failure to supply data. Occasional use of legal authority can supply the ultimate encouragement to obtain data, although the compiler should first try to identify the reason for non-response and take other suitable action. Suggestions that legal penalties may be invoked should be made only if the compiler actually intends to carry out legal proceedings.
925. Follow-up procedures should be designed to obtain 100 percent response, even if the last part of the response is tardy.
Verifying Reported Data
926. After data is received from data providers, it is important for checks to be undertaken to verify the quality of the data. Such checks would identify:
(1) any logical discrepancies in data
(For example, checks could be undertaken to ensure that components add to totals, that comparable figures reported in different sections of the form are consistent and, in the case of external assets and liabilities, that the reported reconciliation of stock positions and flows is correct.)
(2) empirical discrepancies in data
(For example, checks could be undertaken to ensure that relationships between financial fees and use of financing, investment income and the stock position of external assets and liabilities, and freight and imports are within historically derived ratio tolerances.)
(3) consistency in reporting from period to period
(For example, checks could be made to verify that closing external asset and liability positions from the previous period agree with the opening position for the current period or that activities reported in the previous period continue into the current period.)
(4) large transactions that are likely to have a major impact on the overall results
(5) agreement of data reported in the BOP collection with data reported in other related collections or other information that the compiler may have on the unit.184
927. These examinations may result in corrections being made by the compiler if errors in data can be corrected without contacting reporters—or in the compiler contacting, by telephone or in person, the data provider. Identification of the cause of the anomalies is important to ensure data quality.