Questionnaire

Low	Few major projects are tendered in a competitive process, and the public has limited access to procurement information.
Medium	Many major projects are tendered in a competitive process, but the public has only limited access to procurement information.
High	Most major projects are tendered in a competitive process, and the public has access to complete, reliable, and timely procurement information.

View Table

Low	Few major projects are tendered in a competitive process, and the public has limited access to procurement information.
Medium	Many major projects are tendered in a competitive process, but the public has only limited access to procurement information.
High	Most major projects are tendered in a competitive process, and the public has access to complete, reliable, and timely procurement information.

Definitions of Key Terms

Term	Definition
Tender	The process of inviting bids, evaluating them, and awarding contracts
Competitive	Competitive procurement involves opening the process to multiple bids to generate competition among bidders and obtain the best value. In contrast, noncompetitive procurement happens when the buyer either selects the company to buy from or restricts the bidding process to certain suppliers. Procurement from a list of prequalified bidders is considered competitive if the prequalification process has been open to all potential bidders.
Access to information	See “published” in the Glossary.
Procurement information	Includes The notice of procurement opportunities Procurement documents (including technical standards and evaluation criteria) Number of presented bids, number of accepted bids, and identification of bidders Notice of award of contract

View Table

Term	Definition
Tender	The process of inviting bids, evaluating them, and awarding contracts
Competitive	Competitive procurement involves opening the process to multiple bids to generate competition among bidders and obtain the best value. In contrast, noncompetitive procurement happens when the buyer either selects the company to buy from or restricts the bidding process to certain suppliers. Procurement from a list of prequalified bidders is considered competitive if the prequalification process has been open to all potential bidders.
Access to information	See “published” in the Glossary.
Procurement information	Includes The notice of procurement opportunities Procurement documents (including technical standards and evaluation criteria) Number of presented bids, number of accepted bids, and identification of bidders Notice of award of contract

It may also include procurement plans, summary of contract, standard procurement monitoring reports, and decisions on bidder complaints.

Institutional Design

The aim of this dimension is to determine whether the system of procuring capital projects is structured to maximize value for money. The focus of the design assessment is on the legal and regulatory framework for procurement. A general requirement for competitive procurement does not mean that all procurement will have to be competitive. Procurement legislation will often specify circumstances in which other procurement methods may be accepted. However, the provisions for exceptions should not be so broad that they undermine the general requirement for competitive procurement. For that purpose, in addition to monitoring and legal action by procurement-supervision departments and agencies, competition authorities will often have a mandate to oversee relevant markets and impose sanctions in case of violations.

• A low score on institutional design implies that the legal and regulatory framework for procurement is weak. Competitive procurement is not required and there are no strong provisions for public disclosure of procurement information. Limited access to information means that procurement information is not easily available, for instance, on an open website. If information must be requested directly from the procuring agency, or if access to information requires website-user registration, then access should be considered limited.

• A medium score indicates that the legal and regulatory framework requires competitive procurement of major projects but that it does not establish requirements for access to timely and complete procurement information. If major projects are only competitively procured when externally financed, then the procurement system does not qualify for medium scoring in this dimension.

• A high score indicates that there are legal and regulatory requirements for competitive procurement of major projects, and publication of timely and complete procurement information is required. Open, transparent, and competitive procurement is required for major projects. There are institutions mandated to monitor and propose correction of noncompetitive behavior of procuring agencies and bidders. Cartelization and collusion among bidders are legally punishable. Complete, reliable, and timely information should be proactively disclosed. Procurement laws or regulations specify the type of information and timetable for publishing procurement information. Box 7.1 describes procurement legislation in Bulgaria, which was assessed high on institutional design but lower on effectiveness in 2018.

^{Box 7.1.}

Procurement in Bulgaria: Legal Framework and Institutions

Bulgaria’s public procurement law was substantially overhauled in 2006 as part of the country’s accession to the EU and has been amended frequently, including in 2016 and 2018 (Table 7.1.1).

^{Table 7.1.1.}

Public Works Contracts in Bulgaria (2016 to Mid-2017)

Sources: Public Procurement Law 2018; PPA.

^{Table 7.1.1.}

Public Works Contracts in Bulgaria (2016 to Mid-2017)

	Procurement with a value equal to or above 5 million Bulgarian lev				Procurement with a value greater than 2 but less than 5 million Bulgarian lev
Type of Procurement	Number	Percent	Amount (millions of leva)	Percent	Number	Percent	Amount (millions of leva)	Percent
Open tender	238	94	6,790	94	325	95.3	1,022	95
Negotiated with a prior call for particiaption	5	0.20	40	0.50	1	0.30	5	0.50
Negotiation without prior notice	2	0.08	15	0.20	6	1.70	22	2
Limited	3	0.12	177	2.30	5	1.50	13	1.25
Undefined	6	0.25	211	3	4	1.20	13	1.25
Total	254	100	7,233	100	341	100	1,075	100

Sources: Public Procurement Law 2018; PPA.

View Table

^{Table 7.1.1.}

Public Works Contracts in Bulgaria (2016 to Mid-2017)

	Procurement with a value equal to or above 5 million Bulgarian lev				Procurement with a value greater than 2 but less than 5 million Bulgarian lev
Type of Procurement	Number	Percent	Amount (millions of leva)	Percent	Number	Percent	Amount (millions of leva)	Percent
Open tender	238	94	6,790	94	325	95.3	1,022	95
Negotiated with a prior call for particiaption	5	0.20	40	0.50	1	0.30	5	0.50
Negotiation without prior notice	2	0.08	15	0.20	6	1.70	22	2
Limited	3	0.12	177	2.30	5	1.50	13	1.25
Undefined	6	0.25	211	3	4	1.20	13	1.25
Total	254	100	7,233	100	341	100	1,075	100

Sources: Public Procurement Law 2018; PPA.

The Public Procurement Agency (PPA) is an independent body under the Ministry of Economy. Its mandate includes drafting law on public procurement, giving methodological and other forms of guidance, performing mandatory ex ante controls, monitoring and analyzing procurement markets, alerting supervision authorities to possible irregularities, and maintaining an electronic database with information on all procurement procedures that contracting authorities are required to submit the Public Procurement Register. In cases of irregularities, the PPA would inform the National Audit Office, the State Financial Inspection Agency, and the respective managing authorities.

The Commission for the Protection of Competition (CPC) is charged with implementing the Law on Protection of Competition, as well as with control of procedures under the procurement and concessions laws. As the first-instance review body, the CPC examines and decides on claims of irregularities in public procurement and may interrupt public procurement procedures and impose sanctions for noncompliance.

The National Audit Office performs independent audits of national public finance for legality, efficiency, and effectiveness in the use of public funds. It audits ministries, departments, and municipalities. However, it has limited ability to sanction and can only forward its findings to the Financial Inspection Agency.

Under the Ministry of Finance, the Financial Inspection Agency is an entity set up in 2006 to ensure the protection of public financial interests. It carries out inspections of the budget and the financial-economic and accounting activities of public bodies, and it has the authority to impose sanctions.

In addition, the managing authorities of each individual operational program and the Audit of EU Funds Executive Agency carry out audits and controls on the distribution and use of EU funds in Bulgaria.

If a claimant is not satisfied with the decision of the CPC, it may appeal against the decision before the Supreme Administrative Court. The decision of its three-member chamber will be final and binding for all parties in the case.

Sources: Public Procurement Law 2018; PPA.

If procurement is conducted by multiple entities, under several parallel procurement systems, then scoring should be based on the two most widely used systems. Some countries, instead of an integrated procurement system, allow some ministries, agencies, or class of projects to follow specific procurement legislation—in such cases, the assessment should address at least the two major procurement systems involving major projects.

If procurement of PPPs (including concessions) does not follow the general procurement legislation, then their procurement systems should be assessed. The military, intelligence agencies, and police, being sectors in which security reasons sometimes constrain competition, should not be considered in the assessment—extension of this exemption to other sectors should be avoided.

Important Documents

Documents	Uses
General procurement legislation, plus legislation including alternative provisions for specific sectors, ministries, PCs, and PPPs, including concessions	Assess provisions for competitive procurement
Legislation establishing procurement monitoring departments or agencies	Assess provisions for monitoring procurement and correcting deviations from standard practices Assess provisions against cartelization and bidder collusion
Procurement regulations, including regulations for specific sectors	Assess if regulations also support competitive procurement
Periodic or sporadic reports of procurement monitoring departments and agencies, including statistics, procurement analysis, and notice of corrective action and punitive legal processes	Assess the existence and functioning of monitoring provisions and corrections mechanisms (also relevant for the assessment of effectiveness regarding effective competition)
Call-for-tender documentation of several major projects	Assess qualification requirements and selection criteria (also relevant for the assessment of effectiveness)
Procurement audits	Assess whether audits indicate systemic weaknesses in procurement practices

View Table

Documents	Uses
General procurement legislation, plus legislation including alternative provisions for specific sectors, ministries, PCs, and PPPs, including concessions	Assess provisions for competitive procurement
Legislation establishing procurement monitoring departments or agencies	Assess provisions for monitoring procurement and correcting deviations from standard practices Assess provisions against cartelization and bidder collusion
Procurement regulations, including regulations for specific sectors	Assess if regulations also support competitive procurement
Periodic or sporadic reports of procurement monitoring departments and agencies, including statistics, procurement analysis, and notice of corrective action and punitive legal processes	Assess the existence and functioning of monitoring provisions and corrections mechanisms (also relevant for the assessment of effectiveness regarding effective competition)
Call-for-tender documentation of several major projects	Assess qualification requirements and selection criteria (also relevant for the assessment of effectiveness)
Procurement audits	Assess whether audits indicate systemic weaknesses in procurement practices

Effectiveness

The effectiveness rating should apply the three reference answers to PIMA question 11.a (Is the procurement process for major capital projects open and transparent?), looking at actual procurement results. Procuring entities may engage in tailoring of qualification requirements and selection criteria in ways that repel competition. And, in the absence of anticollusion legislation and corresponding enforcement tools, bidders may neutralize formally competitive public procurement. The effectiveness assessment should cover whether poor practices result from deficiencies in the institutional design (for example, restricted mandate for competitive procurement; too-broad exceptions to competitive procurement; absence of institutions aiming at monitoring procurement and fostering competition; absence of legal provisions against cartelization and collusion) or from implementation issues (for example, low capacity, corruption).

The assessment should clarify the share of procurement that is subject to national procurement rules. Many international financial institutions (IFIs) and development partners (DPs) have their own procurement rules that have to be followed unless there is an explicit decision to rely on national rules or the rules of another agency (see Box 7.2 on World Bank practices in this regard). If a significant share of procurement is done according to IFI or DP rules, then this may affect the share of competitive procurements and the effectiveness on this dimension.

The effectiveness assessment should be based on procurement statistics and verify whether the system is working as intended. It is not uncommon that formal procurement regulations require that most procurement be competitive, but that the actual share of competitive procurements is considerably lower. Effective competition means not only open, transparent, and competitive tenders, but also more than one bidder presenting accepted bids, as well as absence of evidence of cartelization or collusive behavior. Also, formal regulations often mandate a high degree of transparency in procurement activities, but actual transparency is much lower, for instance, because procurement websites have limited coverage or are not updated in a timely manner.

• Low effectiveness implies that few major projects are subject to clearly perceived effective competition.

• Medium effectiveness implies that many major projects are subject to clearly perceived effective competition.

• High effectiveness indicates jurisdictions where the vast majority of major projects are effectively competitively procured and where there is proactive, open disclosure of complete, reliable, and timely procurement information.

The ministry in charge of public procurement policy and control, or the main contracting agencies, should be able to provide evidence that procurement rules are complied with. If this cannot be documented, then there may be discrepancies between formal requirements and actual practices. The scores should be downgraded if the assessment team perceives evidence of significant challenges to an effective competitive procurement, such as pervasive corruption or mismanagement of the procurement processes, unreliable data, or lack of critical relevant data.

If there are many failed tenders or the average number of bidders for competitive and open tenders is consistently low, then the competition may be impaired. Absolute absence of failed tenders for lesser projects may also signal competition issues, particularly when most tenders have a single bidder—hinting at market partition among bidders. In those cases, monitoring reports and action plans addressing poor competition should be requested from authorities for analysis.

These issues should be explored and commented on in the assessment, in particular whether they are a result of an ineffective procurement system design or whether there are other causes.

Useful Data Series

Data	Questions to Address
Number and value of tenders conducted by the category of procurement process ((for example, international open tender, open tender, restricted tender, direct award) Number and value of major projects conducted by the category of procurement process	What is the share of different procurement methods?
Average number of received bids and accepted bids for each tender, using competitive and open tender classes Average number of received bids and accepted bids for each major project	What is the actual degree of competition in competitive tender processes? What is the actual degree of competition in major projects?
Number of failed tenders (tenders announced but for which no contract was issued) Average number of failed bids for each major project	What share of tenders is unsuccessful? What share of major projects is unsuccessful?
Shares of competitive procurement following national rules and following IFI/DP rules, by category of tender process and by major project	What is the importance of IFI and DP projects in the tender process? What is the importance of IFI and DP projects in major projects?

View Table

Data	Questions to Address
Number and value of tenders conducted by the category of procurement process ((for example, international open tender, open tender, restricted tender, direct award) Number and value of major projects conducted by the category of procurement process	What is the share of different procurement methods?
Average number of received bids and accepted bids for each tender, using competitive and open tender classes Average number of received bids and accepted bids for each major project	What is the actual degree of competition in competitive tender processes? What is the actual degree of competition in major projects?
Number of failed tenders (tenders announced but for which no contract was issued) Average number of failed bids for each major project	What share of tenders is unsuccessful? What share of major projects is unsuccessful?
Shares of competitive procurement following national rules and following IFI/DP rules, by category of tender process and by major project	What is the importance of IFI and DP projects in the tender process? What is the importance of IFI and DP projects in major projects?

Questionnaire

Low	There is no procurement database, or the information is incomplete or not timely for most phases of the procurement process.
Medium	There is a procurement database with reasonably complete information, but no standard analytical reports are produced from the database.
High	There is a procurement database with reasonably complete information, and standard analytical reports are produced to support a formal monitoring system.

View Table

Low	There is no procurement database, or the information is incomplete or not timely for most phases of the procurement process.
Medium	There is a procurement database with reasonably complete information, but no standard analytical reports are produced from the database.
High	There is a procurement database with reasonably complete information, and standard analytical reports are produced to support a formal monitoring system.

Definitions of Key Terms

Term	Definition
Procurement database	Relational database software that includes a dataset in which all major steps of each tender are recorded.
Phase of the procurement process	A step defined in procurement regulations, which may vary based on the tender type. Examples include public announcement of tender, clarifications, receipt of bids, evaluation of bids, notice of tender award, and contract signing.
Reasonably complete information	Information should be available according to the steps described above under “Phases of the procurement process.”
Standard analytical report	A standard report drawing information from the procurement database, centrally designed and readily available. It may be periodically published or ready to be run at any time with no additional programming required. It should not only present statistical information, but also present some analysis of the degree of effective competition in public procurement. A user-defined report does not qualify as a standard report.
Formal monitoring system	A set of activities designed to evaluate the ongoing performance of the procurement system and propose improvement or corrective action.

View Table

Term	Definition
Procurement database	Relational database software that includes a dataset in which all major steps of each tender are recorded.
Phase of the procurement process	A step defined in procurement regulations, which may vary based on the tender type. Examples include public announcement of tender, clarifications, receipt of bids, evaluation of bids, notice of tender award, and contract signing.
Reasonably complete information	Information should be available according to the steps described above under “Phases of the procurement process.”
Standard analytical report	A standard report drawing information from the procurement database, centrally designed and readily available. It may be periodically published or ready to be run at any time with no additional programming required. It should not only present statistical information, but also present some analysis of the degree of effective competition in public procurement. A user-defined report does not qualify as a standard report.
Formal monitoring system	A set of activities designed to evaluate the ongoing performance of the procurement system and propose improvement or corrective action.

Institutional Design

This dimension aims to determine whether there is a procurement database to monitor whether the procurement system is operating as intended. Legal systems that aim at competitive procurement must require mechanisms for monitoring procurement and acting when actual procurement practice deviates from the standard and when bidding behavior signals cartelization or collusion among bidders. As recommended by the OECD, the system should allow free access through an online portal for all stakeholders, including civil society and the general public, to public procurement information (OECD 2018). The database and the monitoring mechanism should help identify problems in the procurement system and bring these problems to the attention of responsible officials.

• A low score implies that, there is no procurement database or the information in the database is incomplete or not timely. Significant parts of government may be excluded from the requirement to feed the procurement database or be obliged to provide only limited information. There may also be inadequate rules to ensure that information is recorded in a timely manner.

• A medium score means that the database has reasonably complete information. It should be used by all procuring agencies, with possible exceptions for remoteness and lack of communication capabilities. It should record information on at least 75 percent by value of all nonmilitary public procurement. The information should have a level of detail that is consistent with what is required under the different steps of the government procurement process. Procurement not subject to tender should be recorded in the database.

• A high score implies that the database is used to produce standard analytical reports and support a formal procurement monitoring system. The existence of a standard report conveys that the report represents the views of management and is expected to be run frequently. It is generally not possible to produce reasonably complete standard reports without a procurement database—standard reports must be timely and reasonably complete, which is difficult to achieve if data are collected and reports are prepared manually. A formal monitoring system constitutes a set of activities designed to evaluate the ongoing performance of the procurement system. It must be required by law or regulation and define a responsible entity and requirements for data collection, data analysis, and reporting.

Important Documents

Documents	Uses
Procurement database documentation	Assess scope and functionality
Regulations on the collection and recording of information in the procurement database	Assess whether regulations ensure comprehensive coverage and timeliness
Analytical reports produced from the procurement database.	Assess quality, comprehensiveness, and timeliness of reports

View Table

Documents	Uses
Procurement database documentation	Assess scope and functionality
Regulations on the collection and recording of information in the procurement database	Assess whether regulations ensure comprehensive coverage and timeliness
Analytical reports produced from the procurement database.	Assess quality, comprehensiveness, and timeliness of reports

Effectiveness

A formal monitoring system must consist of more than distribution of standard reports. Standard reports, by definition, facilitate some type of analysis, if nothing more than by the way data is presented. A monitoring system needs to (1) analyze data in standard reports; (2) draw conclusions and possibly make recommendations; (3) communicate those conclusions and recommendations to senior officials for the purpose of improving the procurement system. A formal monitoring system is not effective if it does not perform these activities based on standard reports.

An effective monitoring system requires dedicated and independent staff. Just publishing standard reports on the procurement system website does not constitute an effective monitoring system. At least one person must be dedicated to monitor the procurement system, and that person must have freedom to identify and report problems with the procurement system. Ideally, the monitoring system should be operated by a unit separate from the primary procuring agency. The monitoring system should be considered less effective if it does not appear to operate independently, even if not located organizationally within the primary procuring agency. The rating can be downgraded if the assessment team perceives evidence of significant misrepresentation of procurement data or recommendations deviating from open, transparent, and competitive practices.

Timeliness of analytical reports is a key indicator of effectiveness. If analytical reports are not released in a timely manner, then problems cannot be identified and addressed in a timely manner. Ideally, analytical reports should be available within one month of the end of the reporting period, if not on a real-time basis.

If there is no procurement database or the information in the database is incomplete or not timely and analytical reports are not available at all, or are published more than six months after the analyzed period, effectiveness should be low.

Medium effectiveness implies that the procurement database is reasonably comprehensive, but analytical reports are missing or produced more than six months after the analyzed period or do not cover annual procurement.

The high effectiveness rating is reserved for jurisdictions where the procurement database is used by a monitoring system that produces monthly or quarterly analytical reports drawing conclusions and making recommendations for improvement. Box 7.3 describes the e-procurement system in Bangladesh.

Useful Data Series

Data	Questions to Address
Frequency of analytical reports (for example, monthly, quarterly, annual)	What is the frequency of analytical reports? Are monthly, quarterly, or annual procurement performance results analyzed? Are results compared over time? Are actual results compared with performance goals? Are procurement practices analyzed and recommendations produced?
Publication dates for analytical reports	What is the timeliness of analytical reports over time?

View Table

Data	Questions to Address
Frequency of analytical reports (for example, monthly, quarterly, annual)	What is the frequency of analytical reports? Are monthly, quarterly, or annual procurement performance results analyzed? Are results compared over time? Are actual results compared with performance goals? Are procurement practices analyzed and recommendations produced?
Publication dates for analytical reports	What is the timeliness of analytical reports over time?

^{Box 7.3.}

Electronic Procurement System in Bangladesh

With assistance from the World Bank in 2011, the Bangladeshi government introduced a web-based electronic government procurement system, e-GP. The system is managed by the Central Procurement Technical Unit, part of the Implementation Monitoring and Evaluation Division, Ministry of Planning. The system is currently used by 1,252, or 95 percent, of 1,324 procuring entities, with the 71 entities not participating being small rural units of government. The system covers procurement of works, goods, and services, with the exception of consulting services. Therefore, virtually the entire development budget that requires procurement is acquired through e-GP, which captures data on each step in the procurement process. The public has access, through the e-GP website, to tender documents, bid statistics and summary contract data relating to each tender, and key performance information covering all tenders announced. Bangladesh plans to extend use of the e-GP to all procuring entities by 2022. The procurement methods used in e-GP have been overwhelmingly competitive, as shown in Table 7.3.1.

^{Table 7.3.1.}

Methods Used for Tendering in Bangladesh’s e-GP Procurement System, July 2011 to September 2018

^{Table 7.3.1.}

Methods Used for Tendering in Bangladesh’s e-GP Procurement System, July 2011 to September 2018

Procurement Method	Tenders Initiated	Percent	Value (billions of taka)	Percent
Open-tendering method	141,529	68.0	1,628.6	86.8
Limited-tendering method	63,318	30.4	183.3	9.8
Request for quotation	1,777	0.9	0.2	0.0
One-stage, two envelopes tendering	1,452	0.7	64.9	3.5
Direct procurement	23	0.0	0.0	0.0
Selection under a fixed budget	8	0.0	0.0	0.0
Selection-based consultant qualifications	5	0.0	0.0	0.0
Quality cost-based selection	4	0.0	0.0	0.0
Total	208,116		1,877.2

View Table

^{Table 7.3.1.}

Methods Used for Tendering in Bangladesh’s e-GP Procurement System, July 2011 to September 2018

Procurement Method	Tenders Initiated	Percent	Value (billions of taka)	Percent
Open-tendering method	141,529	68.0	1,628.6	86.8
Limited-tendering method	63,318	30.4	183.3	9.8
Request for quotation	1,777	0.9	0.2	0.0
One-stage, two envelopes tendering	1,452	0.7	64.9	3.5
Direct procurement	23	0.0	0.0	0.0
Selection under a fixed budget	8	0.0	0.0	0.0
Selection-based consultant qualifications	5	0.0	0.0	0.0
Quality cost-based selection	4	0.0	0.0	0.0
Total	208,116		1,877.2

The e-GP system has the capability to produce analytical reports. The system has a reporting module containing a variety of standard reports to monitor the procurement system. Additional standard reports can be added as desired. Data on manual procurements by the 5 percent of procuring entities that do not use e-GP are not entered into the system and thus are not reflected in system reports. This problem will diminish over time as the remaining procuring entities use e-GP. The e-GP system currently publishes quarterly on its website a performance indicators report covering 42 indicators. Data contained within the report are cumulative, beginning in fiscal year 2015/16. Two enhancements are currently being developed that will promote transparency: a data dashboard available to the public and a civil engagement feature that will allow the public to provide feedback on contract implementation.

The e-GP system represents a significant achievement. The system is a modern, fully functional information system, containing all the features expected of such systems. The main challenges going forward are to expand the coverage of the system, disclose more information on tenders processed, and ensure that discretionary data not captured through mandatory system controls, for example, complaint information, are entered accurately and in a timely manner. The last of these challenges may be the most difficult to achieve, as it relates to human behavior. Detailed monitoring of the system can minimize shortcomings in this regard.

Source: e-GP, MOP.

Questionnaire

Low	Procurement complaints are not reviewed by an independent body.
Medium	Procurement complaints are reviewed by an independent body, but the recommendations of this body are not produced on a timely basis, nor published, nor rigorously enforced.
High	Procurement complaints are reviewed by an independent body whose recommendations are timely, published, and rigorously enforced.

View Table

Low	Procurement complaints are not reviewed by an independent body.
Medium	Procurement complaints are reviewed by an independent body, but the recommendations of this body are not produced on a timely basis, nor published, nor rigorously enforced.
High	Procurement complaints are reviewed by an independent body whose recommendations are timely, published, and rigorously enforced.

Definitions of Key Terms

Term	Definition
Tender	must be registered according to procurement law, regulation, or written procedure.
Independent	See glossary
Timely	A recommendation is timely if it is made before the contract is signed.
Publish	See glossary
Rigorously enforce	Implement as intended.

View Table

Term	Definition
Tender	must be registered according to procurement law, regulation, or written procedure.
Independent	See glossary
Timely	A recommendation is timely if it is made before the contract is signed.
Publish	See glossary
Rigorously enforce	Implement as intended.

Institutional Design

This dimension aims to assess if there is a complaints system to ensure that the procurement system is properly designed and is operating as designed. Complaints may suggest procurement procedures were not properly followed for a single tender. They can also mean that the procedures are not well designed. The complaints system should identify these types of issues and make recommendations to resolve them or prevent them in the future. A complaints review system does not exist if bidders are not given a reasonable time window, compatible with the complexity of the tender, for presenting complaints. For example, in Estonia complaints must be presented within 10 days (Box 7.4.)

The independence of the complaints system regarding the procuring agency will be assessed. There is no independence if the review entity is the procuring agency, an entity dependent from it, an entity dependent on the same parent line ministry, or the parent ministry itself. Independence is sometimes difficult to assess, given that in some countries, membership in a complaint review body may be short term—even appointed for review of a single tender. At a minimum, independence requires that the majority of the reviewers are not employees of the procuring agency.

• A low score indicates that the legal and regulatory framework does not require an independent complaints system. In some cases, there is no formal complaints mechanism at all. In other cases, complaints may be fled, but they are handled by an entity not independent from the procuring agency.

• For a medium score, the legal and regulatory framework does require an independent complaints body, but the arrangements for enforcement of the decisions of this body are inadequate. Recommendations of the review body are not required to be delivered on a timely basis, to be published, or to be rigorously enforced.

• For a high score, the legal and regulatory framework ensures that the independent complaints body has the necessary legal standing and capacity to produce recommendations that are timely, published and rigorously enforced. The recommendations of the review body should have legal force and cannot be institutionally disregarded.

Important Documents

Documents	Uses
Legal framework for procurement complaints review body	Assess institutional design
Representative sample of decisions issued by the complaints review body	Analyze whether decisions are published in a timely manner
Legal framework for decision review after issuance of complaints review body recommendations Representative sample of procurement decision revision after issuance of complaints review recommendations	Analyze whether complaints review recommendations are rigorously enforced and whether procurement legislation allows for nonenforcement

View Table

Documents	Uses
Legal framework for procurement complaints review body	Assess institutional design
Representative sample of decisions issued by the complaints review body	Analyze whether decisions are published in a timely manner
Legal framework for decision review after issuance of complaints review body recommendations Representative sample of procurement decision revision after issuance of complaints review recommendations	Analyze whether complaints review recommendations are rigorously enforced and whether procurement legislation allows for nonenforcement

Effectiveness

The effectiveness assessment should include analysis of the performance of the complaint mechanism. This should include the number of complaints that have been lodged the past three years, the average time to consider a complaint, and the share of complaints for which procurement decisions are over turned or penalized. A breakdown by type of procurement and by government sector could be useful. The rating can be lower if there are indications of corruption, significant misrepresentation of complaints, or unfair recommendations. The assessment should give adequate weight to the complaints related to major projects.

High fees to file a complaint may undermine the effectiveness of the complaint system. Small fees may be appropriate to discourage frivolous complaints. However, large fees might discourage genuine complaints, and thus reduce openness and transparency. See Box 7.4 for examples of fees in the Estonian procurement complaints mechanism.

Even if there is no system for compiling and analyzing complaints, the complaint system may be partly effective. The complaint system is aimed at addressing irregularities on an individual tender and bid basis, and this function may be filled even in the absence of a system for compiling and analyzing complaints. However, complaints also have a bearing on system design and overall operations of the procurement system.

The timeliness of the complaint mechanism, as well as its independence and the disclosure and enforcement of review decisions, are core indicators for its effectiveness.

Low effectiveness implies that the average time to resolve complaints is more than six months. A low number of complaints can also be an indication that it is difficult to lodge complaints and that the mechanism is not effective.

If independent reviews are not published or enforced, the complaints review system could receive a medium effectiveness rating in terms of efficiency if the average time to resolve complaints is two to six months.

Independent reviews that are published and rigorously enforced could receive a high effectiveness rating if the average time to resolve complaints is less than two months. Box 7.4 describes the procurement complaint system in Estonia, which was deemed to be highly effective in the 2019 PIMA.

Useful Data Series

Data	Questions to Address
Number of complaints as a percentage of total tenders	Do the complaint shares indicate that the complaint mechanism is well established?
Number of complaints related to major projects as a percentage of tenders of major projects	Do the complaint shares indicate that the complaint mechanism is well established for major projects?
Average time for a decision on a complaint to be made, by stage of review	What is the effectiveness of the complaint mechanism?
Actual time for the decisions on complaints related to all major projects in the last 3 years	What is the effectiveness of the complaint mechanism regarding major projects?
Number of tenders audited and percentage in which violations of procedure are noted	Does the complains mechanism indicate systemic weaknesses in the procurement system?
Number of complaints that end up in court. Number of court ruling confirming the Review Committee’s rulings	How timely and fair are decisions about complaints?

View Table

Data	Questions to Address
Number of complaints as a percentage of total tenders	Do the complaint shares indicate that the complaint mechanism is well established?
Number of complaints related to major projects as a percentage of tenders of major projects	Do the complaint shares indicate that the complaint mechanism is well established for major projects?
Average time for a decision on a complaint to be made, by stage of review	What is the effectiveness of the complaint mechanism?
Actual time for the decisions on complaints related to all major projects in the last 3 years	What is the effectiveness of the complaint mechanism regarding major projects?
Number of tenders audited and percentage in which violations of procedure are noted	Does the complains mechanism indicate systemic weaknesses in the procurement system?
Number of complaints that end up in court. Number of court ruling confirming the Review Committee’s rulings	How timely and fair are decisions about complaints?

Questionnaire

Low	Cash flow forecasts are not prepared or updated regularly, and ministries/ agencies are not provided with commitment ceilings in a timely manner.
Medium	Cash flow forecasts are prepared or updated quarterly, and ministries/ agencies are provided with commitment ceilings at least a quarter in advance.
High	Cash flow forecasts are prepared or updated monthly, and ministries/ agencies are provided with commitment ceilings for the full fiscal year.

View Table

Low	Cash flow forecasts are not prepared or updated regularly, and ministries/ agencies are not provided with commitment ceilings in a timely manner.
Medium	Cash flow forecasts are prepared or updated quarterly, and ministries/ agencies are provided with commitment ceilings at least a quarter in advance.
High	Cash flow forecasts are prepared or updated monthly, and ministries/ agencies are provided with commitment ceilings for the full fiscal year.

Definitions of Key Terms

Term	Definition
Cash	Funds readily available in government bank accounts to make payments through a treasury system.
Cash flow forecast	A forecast of cash inflows and outflows that shows gross flows and cash balances (net) on daily, weekly, monthly, or quarterly intervals.
Prepare or update	The word “prepared” suggests that there is a clear starting point. In some countries, cash flow forecasts start anew each fiscal year and are updated during the year. More commonly, forecasts are always rolling, looking forward for a fixed period. In these cases, cash flow forecasts are being updated on a rolling basis.
Commitment Ceiling	Limit on a ministry’s or agency’s authority to commit to future spending, that is, incur a potential future obligation to pay, for instance, through signing a contract. Ceilings may relate to either (1) the maximum sum of new commitments that can be entered into in a period, or (2) the cumulative total of outstanding commitments.

View Table

Term	Definition
Cash	Funds readily available in government bank accounts to make payments through a treasury system.
Cash flow forecast	A forecast of cash inflows and outflows that shows gross flows and cash balances (net) on daily, weekly, monthly, or quarterly intervals.
Prepare or update	The word “prepared” suggests that there is a clear starting point. In some countries, cash flow forecasts start anew each fiscal year and are updated during the year. More commonly, forecasts are always rolling, looking forward for a fixed period. In these cases, cash flow forecasts are being updated on a rolling basis.
Commitment Ceiling	Limit on a ministry’s or agency’s authority to commit to future spending, that is, incur a potential future obligation to pay, for instance, through signing a contract. Ceilings may relate to either (1) the maximum sum of new commitments that can be entered into in a period, or (2) the cumulative total of outstanding commitments.

Institutional Design

The aim of this dimension is to determine whether there are mechanisms to ensure that cash to make payments is available when needed. The detailed focus is on the formal regulations regarding commitments, and on cash flow forecasting and management.

In-year budget or payment ceilings contribute to cash management. Budget appropriations are recorded in the treasury expenditure control system as authorizations to spend (make payments). Many countries sub-divide the annual appropriation into monthly or quarterly allocations. These have a similar, but less direct, effect to commitment controls and could be seen as an alternative to commitment controls when assessing the design of the system.

• A low score indicates that there is no legal or regulatory requirement for systematic cash flow forecasting. In this environment, cash management will often be ad hoc and there may be uncertainty regarding whether an investment project will be able to make payments in a timely manner to ensure efficient project implementation.

• For a medium score, there is a legal or regulatory requirement for cash flow forecasts to be prepared at least quarterly and ministries are provided commitment ceilings at least a quarter in advance. The commitment ceilings may cover all spending, or be limited to certain types of spending, for instance, capital investments.

• A high score implies that there is legal or regulatory requirement for an advanced cash management system. This includes monthly cash forecasts, and commitment ceilings for the whole fiscal year are provided at the beginning of the year.

Important Documents

Documents	Uses
Legal framework for cash planning and cash management	Assess formal requirements for cash planning
Regulations for expenditure/ commitment control and/or budget execution	Assess design of commitment control mechanism

View Table

Documents	Uses
Legal framework for cash planning and cash management	Assess formal requirements for cash planning
Regulations for expenditure/ commitment control and/or budget execution	Assess design of commitment control mechanism

Effectiveness

The effectiveness assessment should focus on how consistently the mechanisms defined in laws and regulations are applied in practice. This assessment should be based on specific data for cash flow forecasts and commitment ceilings, compared to actual cash flows, commitments, and payments, for the past three years. Does government make cash with fund capital expenditure available as and when needed?

• Low effectiveness indicates that cash forecasts are not reliable. Cash forecasts may be missing altogether, or not documented. Alternatively, actual cash payments tend to be lower than forecast and cash payments during the year are lower than commitment ceilings.

• Medium effectiveness indicates that the reliability of cash forecasting is mixed and ministries are provided commitment ceilings at least a quarter in advance. There may be examples of commitment ceilings not being funded in terms of ensuring cash availability.

• High effectiveness means that cash forecasts are reliable and commitment ceilings for the whole fiscal year are provided at the beginning of the year. There are no examples of commitment ceilings not being funded. Box 7.5 describes cash forecasting arrangements in Armenia.

Useful Data Series

Data	Questions to Address
Cash flow forecasts, broken down by quarter or months	Is there a bias toward restricting funding early in the year?
Actual cash flow, broken down by quarter or months	Are there systematic differences between forecasts and actual cash flows?
Commitment ceilings compared with commitments and actual spending for the same set of projects	How effective is the commitment control system?

View Table

Data	Questions to Address
Cash flow forecasts, broken down by quarter or months	Is there a bias toward restricting funding early in the year?
Actual cash flow, broken down by quarter or months	Are there systematic differences between forecasts and actual cash flows?
Commitment ceilings compared with commitments and actual spending for the same set of projects	How effective is the commitment control system?

Questionnaire

Low	The financing of project outlays and payments are frequently subject to cash rationing.
Medium	Cash for project outlays is sometimes released/paid with delays.
High	Cash for project outlays is normally released/paid in a timely manner, based on the appropriation.

View Table

Low	The financing of project outlays and payments are frequently subject to cash rationing.
Medium	Cash for project outlays is sometimes released/paid with delays.
High	Cash for project outlays is normally released/paid in a timely manner, based on the appropriation.

Definitions of Key Terms

Term	Definition
Outlays	See glossary
Cash rationing	Cash is rationed when there are more invoices to be paid than can be paid, and invoices are paid selectively. This leads to cash not being released within due time (usually 30 days from the date of an invoice, but it can vary among countries, sometimes being 45 or even 60 days).
Cash release	Cash is released when a payment request is approved. It is assumed that if the payment request is approved, the payment is made—in other words, that cash rationing decisions are made before a payment request is approved. The mode of payment can be electronic, check, or replenishment of petty cash.
Timely manner	Payment is made in due time, based on the date of the invoice.
Cash release based on the appropriation	Expenditures are made according to the intended purpose and maximum amount authorized in an appropriation. This assumes that expenditures can be made by the end of the fiscal year up to the amount of the appropriation.

View Table

Term	Definition
Outlays	See glossary
Cash rationing	Cash is rationed when there are more invoices to be paid than can be paid, and invoices are paid selectively. This leads to cash not being released within due time (usually 30 days from the date of an invoice, but it can vary among countries, sometimes being 45 or even 60 days).
Cash release	Cash is released when a payment request is approved. It is assumed that if the payment request is approved, the payment is made—in other words, that cash rationing decisions are made before a payment request is approved. The mode of payment can be electronic, check, or replenishment of petty cash.
Timely manner	Payment is made in due time, based on the date of the invoice.
Cash release based on the appropriation	Expenditures are made according to the intended purpose and maximum amount authorized in an appropriation. This assumes that expenditures can be made by the end of the fiscal year up to the amount of the appropriation.

Institutional Design

The aim of this dimension is to determine whether there are formal mechanisms to ensure that contractual obligations for payment can be met. Such mechanisms can be based on laws and regulations or on lower-level instructions, for instance, treasury guidelines. The language in the questionnaire refers to the actual effects. The effects should be covered in the effectiveness assessment, while the assessment of institutional design should focus on the formal mechanisms established to handle this issue.

• A low score on institutional design means that there are no formal mechanisms to ensure timely release of project funds when payments become due. In this situation, the Treasury may have full flexibility in delaying payments in times of cash shortages, and to reduce cash releases or payments below approved appropriations for the year as a whole.

• A medium score implies that there are formal mechanisms to ensure timely release of project funds, but they are not sufficiently strong to ensure that funds always are released for payment in line with appropriations. The treasury may be authorized to delay release of funds within certain confines, for instance, for a limited period of time, or curtail releases to a certain level of appropriations if this is required. Some countries have legal provisions that budget releases can be limited to 90 or 95 percent of appropriations if there are cash shortages.

• A high score indicates that there are strong mechanisms to ensure timely release of funds for payment, in line with the annual appropriations. This may include a legal provision that all appropriated funds shall be released during the year, unless there is a formal budget amendment.

Important Documents

Documents	Uses
Legal and regulatory framework for budget execution	Assess institutional design
Detailed guidelines for treasury operations	Assess detailed provisions for cash releases

View Table

Documents	Uses
Legal and regulatory framework for budget execution	Assess institutional design
Detailed guidelines for treasury operations	Assess detailed provisions for cash releases

Effectiveness

The effectiveness assessment should use the criteria in the questionnaire to assess the actual performance of the cash management system:

• Low effectiveness means that “frequently” cash is rationed—several invoices for major projects are not paid in a timely manner.

• Medium effectiveness indicates that “sometimes” cash is rationed—most invoices for major projects are paid in a timely manner.

• High effectiveness implies that invoices are always paid in a timely manner, except for incidental human errors. In addition, cash releases should be in line with the appropriations.

The assessment should look at cash releases compared with budget appropriations, including any budget amendments during the year. Payment delays may be caused by a fiscal shock, poor fiscal forecasts, and poor budget estimates, as well as poor cash management. If these lead to formal budget amendments, the assessment of cash management practices should be based on the revised appropriations. Box 7.6 illustrates that even a robust institutional design may be insufficient to ensure effectiveness in a difficult situation.

• If there is limited information on whether invoices are paid in a timely manner, an alternative could be to look at whether major projects have accumulated payment arrears due to cash shortages:

• If cash shortages frequently lead to arrears, effectiveness would be low.

• If cash shortages have caused arrears in a few major projects, effectiveness is medium.

• If there are no accumulated arrears, effectiveness is high.

Useful Data Series

Data	Questions to Address
Size of new, or the cumulative total of, arrears for capital projects	Does cash rationing lead to systemic arrears?
Average time between the date of invoices and the authorization of payment	What is the average maturity of outstanding invoices?

View Table

Data	Questions to Address
Size of new, or the cumulative total of, arrears for capital projects	Does cash rationing lead to systemic arrears?
Average time between the date of invoices and the authorization of payment	What is the average maturity of outstanding invoices?

Questionnaire

Low	External financing is largely held in commercial bank accounts outside the central bank.
Medium	External financing is held at the central bank but is not part of the main government bank account structure.
High	External financing is fully integrated into the main government bank account structure.

View Table

Low	External financing is largely held in commercial bank accounts outside the central bank.
Medium	External financing is held at the central bank but is not part of the main government bank account structure.
High	External financing is fully integrated into the main government bank account structure.

Definitions of Key Terms

Term	Definition
External financing	See the Glossary.
Commercial bank	Provides retail banking services to private individuals and organizations, including government agencies. It may be privately owned or owned by the government.
Main government bank account structure	If a treasury single account (TSA) system exists, the structure includes all bank accounts and their subaccounts that make up the TSA system, with a top account that consolidates cash resources by netting of balances in other accounts. Some accounts included in the TSA system may be in commercial banks. If a TSA does not exist, the main account structure includes only accounts and their subaccounts held in the name of the Treasury or MoF in the central bank.

View Table

Term	Definition
External financing	See the Glossary.
Commercial bank	Provides retail banking services to private individuals and organizations, including government agencies. It may be privately owned or owned by the government.
Main government bank account structure	If a treasury single account (TSA) system exists, the structure includes all bank accounts and their subaccounts that make up the TSA system, with a top account that consolidates cash resources by netting of balances in other accounts. Some accounts included in the TSA system may be in commercial banks. If a TSA does not exist, the main account structure includes only accounts and their subaccounts held in the name of the Treasury or MoF in the central bank.

Institutional Design

The purpose of this dimension is to assess how well external financing of capital projects is integrated with government funds. For some countries, external financing can be an important component of the capital budget. Integration of donor bank accounts into the government’s bank account structure gives the government an opportunity to see donor bank balances and flows, and thus to better plan and manage cash flows related to project expenditures. The assessment of institutional design focuses on the formal rules, policies, and agreements governing the external financing, whereas the influence of these are covered in the effectiveness assessment below.

• A low score indicates that there is no legal or regulatory requirement that external financing flows are at the central bank. External funds are largely outside the scope of government cash forecasting and cash flow management. The government will usually have limited consolidated information about the balances in the commercial bank accounts containing external funds, and the inflows and outflows through these accounts. Some partial information may be available from different sources, but this can generally not be used for cash management.

• A medium score indicates that external financing is required to be held at the central bank, but not as part of the main government bank account structure. Information about the balances and the transactions in these accounts will often be available to the government and can be used for cash forecasting purposes. However, the funds held in these accounts are not fungible with other government funds and thus cannot be incorporated in the government cash management function.

• A high score indicates that there is a legal or regulatory requirement for full integration of external funds in the main government bank account structure. The external funds can be considered part of the government’s cash planning and management framework, although there will often be limitations on how these funds can be utilized in the agreements with the financing sources.

Important Documents

Documents	Uses
Funding agreements with major donors	Clarify requirements for banking arrangements and access to external funds
Overview of government banking arrangements	Assess consistency between banking arrangements for internal and external funds

View Table

Documents	Uses
Funding agreements with major donors	Clarify requirements for banking arrangements and access to external funds
Overview of government banking arrangements	Assess consistency between banking arrangements for internal and external funds

Effectiveness

The effectiveness assessment should include an analysis of the volumes of external funds available in commercial bank account s, in the central bank outside the TSA, and in the TSA. If there are inconsistencies between the formal arrangements for external funds and the actual practices, then effectiveness could be lower or higher than what the design score implies.

The assessment of effectiveness should focus on whether there is adequate treasury or MoF oversight of external financing flows and whether flows are available when needed. If an IFI or a DP exhibits a pattern of not providing financing when needed, then the bank account structure is not effective in achieving the aims of this dimension. On the other hand, if the external source keeps money in commercial banks but the Treasury or MoF is promptly informed about the external financing flows and they are available when needed, the system may be more effective than indicated by the design score.

The assessment should be based on how promptly the information on external financing flows is provided to the the Treasury or MoF so that it can be incorporated in the government’s cash management framework. The information should include the date and amount of cash releases or payments for the related projects. Some financing releases, for instance, the first release under a new financing arrangement, may be subject to special safeguards and not be representative. Delays related to conflicts between the country and the donor, for instance, because reporting requirements are not being met, should not be included in this assessment.

• Low effectiveness means that there are significant delays in providing information to the MoF or treasury on accessing and releasing external funds.

• Medium effectiveness means that there are some delays in providing information to the MoF or treasury on accessing and releasing external funds.

• High effectiveness means that information on external funding is up to date and the funding is always available when required. Box 7.7 describes banking arrangements for external financing in Mauritius.

Useful Data Series

Data	Questions to Address
External funds held under different banking arrangements	What is the volume of financial resources under the different banking arrangements?
Processing time for releases of external financing	Are there delays in the releases of external financing?
Time taken/delay in informing the MoF/treasury about the release of external funding	When and how do the donors inform the MoF/treasury about the cash releases/ payments made by them on projects funded by them?

View Table

Data	Questions to Address
External funds held under different banking arrangements	What is the volume of financial resources under the different banking arrangements?
Processing time for releases of external financing	Are there delays in the releases of external financing?
Time taken/delay in informing the MoF/treasury about the release of external funding	When and how do the donors inform the MoF/treasury about the cash releases/ payments made by them on projects funded by them?

Questionnaire

Low	Most major capital projects are not monitored during project implementation.
Medium	For most major projects, annual project costs, as well as physical progress, are monitored during project implementation.
High	For all major projects, total project costs, as well as physical progress, are centrally monitored during project implementation.

View Table

Low	Most major capital projects are not monitored during project implementation.
Medium	For most major projects, annual project costs, as well as physical progress, are monitored during project implementation.
High	For all major projects, total project costs, as well as physical progress, are centrally monitored during project implementation.

Definitions of Key Terms

Term	Definition
Major capital projects	See the Glossary.
Monitor	Monitoring should capture physical and financial progress for major projects under a ministry and is separate from supervision of individual projects. A supervising engineer employed by the government who supervises and certifies work done in compliance with the contract does not constitute monitoring in this context.
Project implementation	Implementation in this context means the work performed from budget approval to when the physical structure is ready to become operational.
Physical progress	Construction contracts typically have well-defined milestones of physical work to be done. The milestones are usually the basis for making progress payments to the contractor, allowing for measuring physical execution against financial execution.

View Table

Term	Definition
Major capital projects	See the Glossary.
Monitor	Monitoring should capture physical and financial progress for major projects under a ministry and is separate from supervision of individual projects. A supervising engineer employed by the government who supervises and certifies work done in compliance with the contract does not constitute monitoring in this context.
Project implementation	Implementation in this context means the work performed from budget approval to when the physical structure is ready to become operational.
Physical progress	Construction contracts typically have well-defined milestones of physical work to be done. The milestones are usually the basis for making progress payments to the contractor, allowing for measuring physical execution against financial execution.

Institutional Design

The aim of this dimension is to ascertain if there are legal or regulatory requirements that the government have a monitoring system for major capital projects. This system should assess, on a regular and relatively frequent basis, if project implementation is going according to the plan, or if the project faces cost overruns, delays, or other problems.

• A low score implies that there is no legal or regulatory framework for systematic monitoring of major capital projects. Projects are managed individually, and there is no consolidated information about project progress across government departments.

• A medium score implies that there is a legal or regulatory framework for monitoring annual project costs, as well as physical progress for most major projects. Monitoring of costs typically is made against appropriations and against the project implementation plan or contracts. Physical monitoring should indicate the project status compared with the broad stages in the project cycle, the project implementation plan, or physical milestones in the contract. The monitoring information may be reported and consolidated for groups of ministries or the whole government.

• A high score means that there is a legal or regulatory framework for central monitoring of all major projects, including for in-year reports. Central monitoring means that information on the implementation status of all major projects is brought together in one place. This allows identification of overall trends, such as a pattern of delays or cost overruns, and creates the potential to reallocate funds between projects. The monitoring should cover developments in expected total project as well as annual costs. Central monitoring can be conducted by a wide range of organizations, including a specialized monitoring unit. Box 7.8 describes the S-curve approach that is commonly used for project monitoring.

^{Box 7.8.}

S-Curve Project Monitoring

The S-curve provides a simple early-warning tool to monitor whether projects are on track. Based on cash flow forecasts contained in the implementation plan, an S-curve chart can set out a lower and higher bound for expected project expenditure during the implementation timeframe. If observed project expenditure and revised forecasts stay between the two boundaries, the project is on track. However, if expenditures proceed too slowly, the project is delayed and likely facing challenges, which will result in cost overruns. Intervention by the supervisor can be initiated as soon as warning signs emerge.

View Full Size

S-Curve for Project Management

Source: IMF staff, based on Pinto and Venkataraman (2008).Note: The S-curve is used to monitor actual cost/time against planned cost/time.

• Download Figure
• Download figure as PowerPoint slide

Important Documents

Documents	Use
Regulations for project and portfolio monitoring	Assess legal requirements for monitoring
Guidelines for project and portfolio monitoring	Analyze methodologies and templates for monitoring
Representative sample of monitoring reports	Assess how methodologies are applied in practice

View Table

Documents	Use
Regulations for project and portfolio monitoring	Assess legal requirements for monitoring
Guidelines for project and portfolio monitoring	Analyze methodologies and templates for monitoring
Representative sample of monitoring reports	Assess how methodologies are applied in practice

Effectiveness

The assessment of effectiveness should be based on how monitoring information is used to improve portfolio performance. The greatest benefit of monitoring implementation of a group of projects is to identify trends that might warrant actions. If information is collected on implementation status, but no analysis is made or conclusions drawn, the monitoring system is not effective. Likewise, if the monitoring system is focused on resolving problems for individual projects in isolation, the monitoring is not effective for the purpose of this dimension.

The assessment of the effectiveness of monitoring should cover all projects in the public investment portfolio, regardless of funding sources. It should include projects procured as PPPs as well as externally financed projects.

• Low effectiveness implies that systematic data on portfolio delays and cost overruns are missing for the majority of projects in the portfolio or that many projects in the portfolio are behind schedule or over budget (average past three years).

• Medium effectiveness implies that there are systematic data on portfolio delays and cost overruns for the majority of projects in the portfolio and that some monitored projects are behind schedule or over budget.

• High effectiveness implies that there are systematic data on portfolio delays and cost overruns for the majority of projects in the portfolio and that few projects are behind schedule or over budget. Box 7.9 describes portfolio monitoring arrangements in Honduras.

Useful Data Series

Data	Questions to Address
Number and cost of major projects that are subject to monitoring	What is the overall scope of portfolio monitoring?
Number of major projects that are behind schedule, and by how much	What are average project delays?
Number of major projects experiencing cost overruns, and by how much	What are average cost overruns?
Examples of corrective actions taken based on monitoring reports	How does monitoring affect issue resolution?

View Table

Data	Questions to Address
Number and cost of major projects that are subject to monitoring	What is the overall scope of portfolio monitoring?
Number of major projects that are behind schedule, and by how much	What are average project delays?
Number of major projects experiencing cost overruns, and by how much	What are average cost overruns?
Examples of corrective actions taken based on monitoring reports	How does monitoring affect issue resolution?

^{Box 7.9.}

Portfolio Monitoring in Honduras

Important investment projects in Honduras are monitored during their execution, from the financial and physical progress point of view. The executors are responsible for monitoring the projects. They report via the integrated financial information system, with additional reporting on presidential priority projects. Monthly, the Budget Directorate produces budget execution reports by institution and for the central administration. Reports of physical and financial execution of the programs and projects incorporated in the public investment program (including companies and decentralized entities) are published quarterly and forwarded to the National Congress. Likewise, enhanced monitoring is carried out on strategic and high-risk projects, and portfolio reviews are carried out periodically with the main international financing agencies. The Public Investment Directorate (DGIP) prepares reports of projects and programs at risk, calling attention to situations that warrant corrections or more detailed investigation.

DGIP data for 186 projects started and concluded between 2003 and 2015 show that US$1,765 million was executed, of a total of US$2,446 million budgeted (72 percent, Figure 7.9.1). The average duration was percent higher than initially planned, and the variations in the execution period were significant (Figure 7.9.2).

View Full Size

Final Project Cost Compared With Plan, 2003–15

(Percent)

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Final Project Duration Compared With Plan, 2003–15

(Percent)

Sources: IMF staff estimates, Honduras Ministry of Finance.

• Download Figure
• Download figure as PowerPoint slide

Questionnaire

Low	Funds cannot be reallocated between projects during implementation.
Medium	Funds can be reallocated between projects during implementation, but not using systematic monitoring and transparent procedures.
High	Funds can be reallocated between projects during implementation, using systematic monitoring and transparent procedures.

View Table

Low	Funds cannot be reallocated between projects during implementation.
Medium	Funds can be reallocated between projects during implementation, but not using systematic monitoring and transparent procedures.
High	Funds can be reallocated between projects during implementation, using systematic monitoring and transparent procedures.

Definitions of Key Terms

Term	Definition
Reallocation	Revision to appropriations specific to capital projects by moving funds from one appropriation to the other. Reallocations are made by the executive with authority delegated to it by the legislature. This might show up in budget execution reports as “revised budget.”
During implementation	In this context, means any time after a project appropriation has been approved.
Systematic	See the Glossary.
Monitor	See Dimension 13.a.
Transparent	See the Glossary.
Procedure	Includes both process and criteria

View Table

Term	Definition
Reallocation	Revision to appropriations specific to capital projects by moving funds from one appropriation to the other. Reallocations are made by the executive with authority delegated to it by the legislature. This might show up in budget execution reports as “revised budget.”
During implementation	In this context, means any time after a project appropriation has been approved.
Systematic	See the Glossary.
Monitor	See Dimension 13.a.
Transparent	See the Glossary.
Procedure	Includes both process and criteria

Institutional Design

The aim of this dimension is to see whether fund reallocations can be used to expedite implementation of the portfolio of projects. This is done by shifting funds to projects that are progressing more quickly than planned from those that are progressing slower than planned. Shifts between projects for the purpose of this dimension require that projects fall under separate appropriations. When multiple projects are included under a single appropriation, the legislature is giving explicit authority to the executive to shift funds between projects within that appropriation. This dimension is oriented to a portfolio-wide perspective, which assumes multiple appropriations.

• A low score indicates that the legal or regulatory framework does not allow reallocation of funds between projects under different appropriations during project implementation. It is not possible to transfer unspent funds under one appropriation to another project requiring additional funds, without going through a budget amendment process.

• A medium score indicates that the legal or regulatory framework does allow reallocation of funds between projects under different appropriations, but the mechanism does not ensure that reallocation decisions are based on systematic monitoring and transparent procedures. There is therefore a risk that funds are reallocated from a project that might be able to utilize the funds or to another project that will not be able to use all the additional funding. There may also be ambiguities regarding the rules for when reallocations are allowed and any restrictions on the reallocations, for instance, as a share of the initial budget.

• A high score indicates the legal or regulatory framework allows reallocation of funds between projects under different appropriations based on systematic monitoring and transparent procedures. This framework provides clear rules for when reallocations are allowed and how they are carried out, often with limits on the amount or share that can be reallocated, and depends on a monitoring system of the type described under dimension 13.a. Systematic monitoring, including of total project costs, would imply that if the score on 13.b is high the score on 13.a should be medium or high.

Important Documents

Documents	Use
Legal framework for reallocation of funds between investment projects	Assess institutional design
Documents for projects that have had reallocation of funds and the effects on project implementation.	Analyze how reallocation effects implementation

View Table

Documents	Use
Legal framework for reallocation of funds between investment projects	Assess institutional design
Documents for projects that have had reallocation of funds and the effects on project implementation.	Analyze how reallocation effects implementation

Effectiveness

The analysis of effectiveness should be based on whether reallocations have allowed for accelerated implementation of some projects and whether there have been delays in the projects the funds were reallocated from. The analysis should use data for actual reallocations in the past three years and show how much has been reallocated as share of the total capital budget.

The effectiveness assessment should include whether rules regarding reallocations have contributed to high execution rates for the capital budget. There may also be other reasons for high capital budget execution, but reallocations have a major effect on budget execution rates. Low execution rates indicate that the mechanism is not effective, whereas high execution rates imply the opposite:

• Low effectiveness: There is no evidence that reallocation has promoted accelerated implementation of projects or capital budget execution is low.

• Medium effectiveness: There is some evidence that reallocation has promoted accelerated implementation of projects and capital budget execution is medium.

• High effectiveness: There is significant evidence that reallocation has promoted accelerated implementation of projects and capital budget execution is high.

Useful Data Series

Data	Questions to Address
Total amount of money that has been reallocated between projects, and as a percentage of the total capital budget	What is the importance of the budget reallocation mechanism?
Rate of overall capital budget execution—under-or overspending compared with initial approved budget	How effective is capital budget execution?
For a sample of projects subject to reallocation, financial and physical progress reports	Has the reallocation allowed for accelerated implementation of projects? Has the reallocation caused delays in the projects the funds were reallocated from?

View Table

Data	Questions to Address
Total amount of money that has been reallocated between projects, and as a percentage of the total capital budget	What is the importance of the budget reallocation mechanism?
Rate of overall capital budget execution—under-or overspending compared with initial approved budget	How effective is capital budget execution?
For a sample of projects subject to reallocation, financial and physical progress reports	Has the reallocation allowed for accelerated implementation of projects? Has the reallocation caused delays in the projects the funds were reallocated from?

Questionnaire

Low	Ex post reviews of major projects are neither systematically required nor frequently conducted.
Medium	Ex post reviews of major projects, focusing on project costs, deliverables, and outputs are sometimes conducted.
High	Ex post reviews of major projects focusing on project costs, deliverables, and outputs are conducted regularly by an independent entity or experts and are used to adjust project implementation policies and procedures.

View Table

Low	Ex post reviews of major projects are neither systematically required nor frequently conducted.
Medium	Ex post reviews of major projects, focusing on project costs, deliverables, and outputs are sometimes conducted.
High	Ex post reviews of major projects focusing on project costs, deliverables, and outputs are conducted regularly by an independent entity or experts and are used to adjust project implementation policies and procedures.

Definitions of Key Terms

Term	Definition
Ex post review	See glossary for a definition of “review.” Ex post means that reviews are conducted after construction has been certified as completed or if the project has been officially terminated before completion.
Deliverable and output	In this context, the deliverable is the physical structure built. This is synonymous with output as defined in Dimension 2.c.
Independent	See glossary
Project implementation policy and procedure	A policy or procedure included in regulations or manuals and other written documents intended to guide project implementation.
Systematic	See glossary

View Table

Term	Definition
Ex post review	See glossary for a definition of “review.” Ex post means that reviews are conducted after construction has been certified as completed or if the project has been officially terminated before completion.
Deliverable and output	In this context, the deliverable is the physical structure built. This is synonymous with output as defined in Dimension 2.c.
Independent	See glossary
Project implementation policy and procedure	A policy or procedure included in regulations or manuals and other written documents intended to guide project implementation.
Systematic	See glossary

Institutional Design

The aim of this dimension is to determine whether the executive has procedures in place to learn from implementation experience and whether this learning is used to improve implementation policies and procedures. The assessment of the institutional design should focus on the formal mechanisms in place to initiate such review, whether they are established in law, by regulation, or through practice. It should focus on ex post reviews carried out by government agencies. Ex post reviews carried out by IFIs and DPs will inform the collaboration bet ween the government and these institutions but will usually have limited effect on general government practices. If there is uncertainty about whether an ex post review constitutes a government review, the assessment should look at who defines the terms of reference and manages the review process.

• A low score implies that there is no formal requirement for ex post review of major projects. Projects will usually be subject to financial control and audit, but this does not qualify as an ex post review. In most cases, project completion reports will not qualify as ex-post reviews either. Ex post reviews should as a minimum assess final project costs and outputs, compared with initial plans.

• A medium score indicates that there is a formal requirement for ex post review of major projects focusing on project costs, deliverables, and outputs. The review should also cover the timeliness of project implementation. This dimension is focused on procedures of the executive designed to improve implementation policies and procedures for all projects. Review conducted by an internal audit entity would qualify as ex post review for this purpose, provided it includes assessment of final project costs and outputs. External audits do not qualify as ex post reviews from the perspective of the executive, but are covered separately under Dimension 14.c.

• A high score indicates that there is a formal requirement for ex post review of major projects focusing on project costs, deliverables, and outputs conducted by independent parties and systematically used to adjust policies and procedures. There should be a clear policy for when ex post reviews are carried out.

Important Documents

Documents	Use
Legal framework for investment projects, including for ex post reviews	Verify requirements for ex post review
Guidelines and templates for ex post reviews	Assess methodologies and specific content of reviews
Representative sample of ex post review documents	Assess how guidelines are applied and the quality of the reviews

View Table

Documents	Use
Legal framework for investment projects, including for ex post reviews	Verify requirements for ex post review
Guidelines and templates for ex post reviews	Assess methodologies and specific content of reviews
Representative sample of ex post review documents	Assess how guidelines are applied and the quality of the reviews

Effectiveness

The effectiveness assessment should be based on how ex-post reviews are conducted in practice, including how the results are presented and used. This should be compared to the formal requirements, to see whether the number of reviews and the contents of these are in line with the intentions. Systematic presentation of review findings to senior officials will strengthen the review mechanism. Specific effects of ex post reviews on policies and practices should be identified and documented.

• Low effectiveness: Government ex post reviews cover few major projects

• Medium effectiveness: Government ex post reviews cover some major projects

• High effectiveness: Government ex post reviews cover many major projects and the information has been systematically used to adjust policies and procedures There must be a demonstrable link between the ex post reviews and changes to implementation policies and procedures. Policies that are affected might relate narrowly to the requirements for a proper project implementation plan or address broader issues such as the use of eminent domain for land acquisition or bulk purchasing of construction materials. Box 7.10 describes arrangements for project monitoring (13.a) and ex post reviews (13.c) in Malaysia.

Useful Data Series

Data	Questions to Address
Number of ex post reviews conducted annually	What is the scope of ex post reviews?
Number of ex post reviews that led to revisions to project preparation, appraisal or implementation manuals, or their equivalent.	What are the effects of ex post reviews?

View Table

Data	Questions to Address
Number of ex post reviews conducted annually	What is the scope of ex post reviews?
Number of ex post reviews that led to revisions to project preparation, appraisal or implementation manuals, or their equivalent.	What are the effects of ex post reviews?

^{Box 7.10.}

Project Monitoring and Ex Post Project Reviews in Malaysia

The Malaysia Project Monitoring System II, called SPPII, assists the Malaysian government in the effective monitoring of all projects. The system generates monitoring reports that can be used for weekly monitoring by ministries and agencies, as well as producing monthly monitoring reports that are helpful for senior management (Figure 7.10.1). Yearly reports are generated to enable politicians to have a condensed view regarding progress and status of all projects.

View Full Size

Status of Physical Projects behind Schedule in Malaysia, by Activity, March 2017

Source: Malaysia PIMA 2017.

• Download Figure
• Download figure as PowerPoint slide

The objective of the system is to monitor the financial and physical progress and status of all projects, both at the federal- and at the state-level execution of government funding. It also identifies project risks monthly. This empowers project managers and central agencies to act quickly when problems are identified in projects. The system measures the projects through the California Expenditure Curve (S-curve) principle, which indicates percentage of time lapsed versus percentage of money spent and generates an early warning on project issues and risks. The system contains vital project information such as the Geographic Information System Module that enables the users of the system to have a clear view of the physical location of the project. This information is also of vital importance to monitor progress by region.

The SPPII system generates a problem identification report that identifies all categories of issues that were not managed well during the year and that resulted in cost overruns as well as time overruns. The purpose of the problem identification report is to compile a lessons-learned matrix. Reports are simple to interpret and understand and can be interpreted by technical and financial personnel and by politicians. There is no wasted information to clutter the system.

Ex post reviews are conducted routinely by the Implementation Coordination Unit (ICU) through the SPPII Project Management System on projects after their completion, and results are presented to the National Action Working Committee. ICU has also issued a guideline for line ministries on evaluation of development programs. Aside from coordinating evaluation on programs and projects by all line ministries, ICU has since the 10th Malaysia plan undertaken annual assessment to determine lessons learned and best practices in project management in all phases of the project cycle. This initiative covers 10 percent of completed projects. The ex post reviews led by ICU provide the opportunity to identify deficiencies that occurred during the life cycle of the project and to learn from them.

Source: Malaysia PIMA 2017.

Questionnaire

Low	Ministries/agencies do not systematically identify senior responsible officers for major investment projects, and implementation plans are not prepared before budget approval.
Medium	Ministries/agencies systematically identify senior responsible officers for major investment projects, but implementation plans are not prepared before budget approval.
High	Ministries/agencies systematically identify senior responsible officers for major investment projects, and implementation plans are prepared before budget approval.

View Table

Low	Ministries/agencies do not systematically identify senior responsible officers for major investment projects, and implementation plans are not prepared before budget approval.
Medium	Ministries/agencies systematically identify senior responsible officers for major investment projects, but implementation plans are not prepared before budget approval.
High	Ministries/agencies systematically identify senior responsible officers for major investment projects, and implementation plans are prepared before budget approval.

Definitions of Key Terms

Term	Definition
Senior responsible officer	Has authority to make project management decisions and is accountable for its overall success or failure; will often be designated as project owner or project manager.
Implementation plan	A project implementation plan breaks down a project into the distinct steps required to accomplish a particular goal. (See Box 7.11).
Budget approval	Approval by the legislature.

View Table

Term	Definition
Senior responsible officer	Has authority to make project management decisions and is accountable for its overall success or failure; will often be designated as project owner or project manager.
Implementation plan	A project implementation plan breaks down a project into the distinct steps required to accomplish a particular goal. (See Box 7.11).
Budget approval	Approval by the legislature.

Institutional Design

The aim of this dimension is to ensure that project implementation proceeds according to expected scope, cost, and schedule. The assumption is that this requires at least two key elements and there must be formal requirements for these in the regulatory framework:

• A senior officer who is responsible for the project and accountable for successful implementation. Often, this person oversees all facets of project implementation on a day-by-day basis, but not always. For major projects, it is becoming increasingly common to separate the functions of project owner and project manager. If a staff engineer supervises project implementation day to day but must clear project management decisions with a supervisor (project owner), then the senior responsible officer is the supervisor, not the staff engineer. The purpose of naming a responsible officer is to establish a point of contact, provide consistent guidance over project implementation, and ensure accountability. The person responsible must also have the necessary capacity to carry out project management. If a department director is the formal project manager for several different projects under the department’s responsibility, he or she will have limited capacity to carry out real project management.

• A comprehensive project implementation plan to guide the detailed implementation of the project. This plan should include roles and responsibilities of the project manager and key staff, major tasks, schedule, budget and necessary resources, major risks and key success factors, reporting, and document management. It should be prepared before project approval, to ensure effective project mobilization after approval.

The assessment of institutional design should reflect the legal and regulatory requirements for a senior responsible officer and for a project implementation plan:

• A low score implies that there are legal or regulatory requirements for neither of these two features. Project responsibilities may be diluted, for instance, they are is shared among different departments, or if projects are allocated to an organizational unit that has no accountability for the results. In many countries, implementation plans are rudimentary and prepared late in the project cycle.

• A medium score implies that there are legal or regulatory requirements that senior responsible officers be appointed for all major projects, but not for implementation plans before projects are approved. These plans may be prepared later in the process, or the projects may go ahead without proper implementation plans. The failure to have implementation plans in place before project approval is a major factor in project delays and cost overruns in many countries. The preparation of such plans will bring out key project preconditions, risks, and bottlenecks, all of which should be addressed before project approval. If there are requirements for implementation plans, but not for senior responsible officers, the score will also be medium.

• For a high score, there are legal or regulatory requirements for both senior responsible officers for all major projects and implementation plans prepared before projects are approved. Implementation issues can affect the scope, cost, and schedule in the original project definition. For this reason, it is important that the implementation strategy is discussed as early as possible, preferably in the project appraisal process. The detailed implementation plan should build on this strategy and be included in the project submission to MoF in the budget preparation process. The implementation plan must capture all key project features, including risk identification and risk management. Box 7.11 outlines the key elements of a project implementation plan.

Important Documents

Documents	Uses
Legal and regulatory framework for project management	Assess institutional design
Guidelines for project management and templates for project implementation plans	Assess comprehensiveness of project management guidelines
Representative sample of project implementation plans	Assess quality of project implementation plans
Decision documents for appointment of project managers	Assess when project managers are appointed and what their responsibilities are

View Table

Documents	Uses
Legal and regulatory framework for project management	Assess institutional design
Guidelines for project management and templates for project implementation plans	Assess comprehensiveness of project management guidelines
Representative sample of project implementation plans	Assess quality of project implementation plans
Decision documents for appointment of project managers	Assess when project managers are appointed and what their responsibilities are

Effectiveness

The effectiveness assessment should focus on how well the formal regulations are followed in practice. It is not uncommon that the formally appointed project managers have little direct involvement in project management and limited accountability. In some countries, the appointments follow organizational lines and managers are formally in charge of several hundred projects. Therefore, the effectiveness assessment must look at what are the actual practices when it comes to senior responsible officers. When these are appointed and how active are they in managing the project implementation process? Are they actually held accountable for project progress? How comprehensive are project implementation plans, when are they developed, and how high is the quality of these plans? The assessment should be based on review of a representative sample of project implementation plans.

• Low effectiveness implies that many major projects have neither identified senior responsible officers nor project implementation plans before project approval.

• Medium effectiveness means that most major projects have identified senior responsible officers or project implementation plans before project approval.

• High effectiveness indicates that most major projects have identified senior responsible officers and project implementation plans before project approval. Box 7.11 describes the content of a project implementation plan.

IFIs and DPs often require specific project management arrangements for projects in which they contribute to the financing. This may affect the effectiveness score on this dimension. If there are many externally funded projects and these are based on effective project management practices, this may influence the overall share of major projects that meet the requirements of this dimension. To influence the effectiveness assessment, both the project management practices that are applied and the share of projects covered by these should be documented.

Useful Data Series

Data	Questions to Address
For a sample of major projects recently completed and projects currently ongoing, the stages of project implementation at which the project manager was appointed	What is the effectiveness of requirements for identified project managers?
List of implementing agencies that consistently produce project implementation plans, and at what project stage they are produced	What is the effectiveness of requirements for implementation plans prepared before projects are approved?
Share of major externally financed projects that apply stringent project management practices	Do externally financed projects increase the overall share of projects that are effectively managed?

View Table

Data	Questions to Address
For a sample of major projects recently completed and projects currently ongoing, the stages of project implementation at which the project manager was appointed	What is the effectiveness of requirements for identified project managers?
List of implementing agencies that consistently produce project implementation plans, and at what project stage they are produced	What is the effectiveness of requirements for implementation plans prepared before projects are approved?
Share of major externally financed projects that apply stringent project management practices	Do externally financed projects increase the overall share of projects that are effectively managed?

Questionnaire

Low	There are no standardized rules and procedures for project adjustments.
Medium	For major projects, there are standardized rules and procedures for project adjustments, but do not include, if required, a fundamental review and reappraisal of a project’s rationale, costs, and expected outputs.
High	For all projects, there are standardized rules and procedures for project adjustments and, if required, include a fundamental review of the project’s rationale, costs, and expected outputs.

View Table

Low	There are no standardized rules and procedures for project adjustments.
Medium	For major projects, there are standardized rules and procedures for project adjustments, but do not include, if required, a fundamental review and reappraisal of a project’s rationale, costs, and expected outputs.
High	For all projects, there are standardized rules and procedures for project adjustments and, if required, include a fundamental review of the project’s rationale, costs, and expected outputs.

Definitions of Key Terms

Term	Definition
Standard	See the Glossary.
Project adjustment	Changes to the project scope, location, function, schedule, or cost.
Major projects	See the Glossary.
Fundamental review and reappraisal	Repeated assessment of the project concept, project appraisal, and selection, without regard to sunk costs.

View Table

Term	Definition
Standard	See the Glossary.
Project adjustment	Changes to the project scope, location, function, schedule, or cost.
Major projects	See the Glossary.
Fundamental review and reappraisal	Repeated assessment of the project concept, project appraisal, and selection, without regard to sunk costs.

Institutional Design

The purpose of this dimension is to assess whether project adjustments are based on clear and transparent rules. Standardized rules and procedures for adjustments aim to identify the full effect on the project, as soon as possible, of major changes in assumptions used when the project was first selected.

• A low score implies that there are no standardized rules and procedures for project adjustments. This could mean that project adjustments are not done—once a project is approved the project parameters remain unchanged, at least formally. It could also mean that any project adjustments are done ad hoc, without a clear regulatory basis. In some countries, there are rules for adjustment of contracts related to projects, but these adjustments do not lead to documented changes in the project plans (cost estimates, timetables, and expected results). Such contract adjustments do not constitute project adjustment from the perspective of this dimension.

• For a medium score, there are standardized rules in place. Standardized rules and procedures must address two issues: what triggers an adjustment, and how a project is adjusted. Projects have different risk profiles and vary regarding the ease with which project adjustments can be made. Rules and procedures may be different for school projects than for road projects. However, rules and procedures must be standard for similar types of projects.

• A high score indicates that the rules require a fundamental review of a project under predefined circumstances. For instance, if the cost for milestone X increases by 20 percent, then a fundamental review and reappraisal of the project must be conducted. This may lead to major project changes and even to cancellation. Rules and procedures may differ based on the sector or size of the project.

Important Documents

Documents	Uses
Regulatory framework for project adjustment	Assess institutional design
Guidelines and templates for project adjustment	Assess comprehensiveness of project adjustment guidelines
Representative sample of documents prepared for proposed project adjustments	Assess comprehensiveness and quality of project adjustment proposals

View Table

Documents	Uses
Regulatory framework for project adjustment	Assess institutional design
Guidelines and templates for project adjustment	Assess comprehensiveness of project adjustment guidelines
Representative sample of documents prepared for proposed project adjustments	Assess comprehensiveness and quality of project adjustment proposals

Effectiveness

The effectiveness of this dimension depends on how adjustments are made in practice and whether they follow the issued rules. If the design score is medium or high, but the regulations are not systematically applied and enforced, the effectiveness score should be lower than the design score.

The effectiveness assessment should look at the rationale for project adjustments. If project adjustments are made routinely, without careful consideration of the causes and the justification, the process is not effective. This is particularly true when scope change and cost escalation is accepted without being challenged and discussed. This factor should be assessed by reviewing a representative sample of project adjustment documents.

The assessment should also reflect how often projects are subject to formal adjustment. Given the uncertainty related to major investment projects, it is reasonable to assume that it will be necessary to make adjustments in project plans for at least some of the major projects under implementation. If this never happens, project implementation is unlikely to be effective. In a well-functioning system, it is also reasonable to assume that at least some projects may need to be cancelled or substantially redesigned following fundamental review. When adjustments are done, there should be a consistent update of cost estimates, timetables and implementation plans. Adjustments should not be done implicitly by allowing annual budget allocations to exceed initial project estimates without transparent disclosure.

• Low effectiveness means that project adjustment proposals are not analyzed and documented or no or few major investment projects are subject to formal adjustment over the implementation period.

• Medium effectiveness means that project adjustment proposals are consistently analyzed and documented, and a few major investment projects are subject to formal adjustment over the implementation period.

• High effectiveness indicates that a few projects are subject to formal adjustment over the implementation period and some of these projects are cancelled or substantially redesigned following fundamental review. Box 7.12 describes the arrangements for project adjustments and fundamental review in Korea.

Adjustments can be made through the budget process as well as in-year by the executive. The focus is on ongoing projects. The executive has authority to stop work on a project. However, the revised total project cost must be reflected in the budget process, along with an explanation for any change in the project and project implementation plan. The budget process should include references to projects that have been terminated.

Useful Data Series

Data	Questions to Address
Number and size of major projects, number that were formally adjusted—how and why	What is the share of projects that are adjusted?
Number and size of projects that were terminated or substantially redesigned before completion	What is the frequency of project cancellation or substantial redesign?

View Table

Data	Questions to Address
Number and size of major projects, number that were formally adjusted—how and why	What is the share of projects that are adjusted?
Number and size of projects that were terminated or substantially redesigned before completion	What is the frequency of project cancellation or substantial redesign?

Questionnaire

Low	Major capital projects are usually not subject to ex post external audits.
Medium	Some major capital projects are subject to ex post external audit, information on which is published by the external auditors.
High	Most major capital projects are subject to ex post external audit information that is regularly published and scrutinized by the legislature.

View Table

Low	Major capital projects are usually not subject to ex post external audits.
Medium	Some major capital projects are subject to ex post external audit, information on which is published by the external auditors.
High	Most major capital projects are subject to ex post external audit information that is regularly published and scrutinized by the legislature.

Definitions of Key Terms

Term	Definition
External audit	In-depth review conducted by the supreme audit institution (SAI) according to internationally recognized audit audit principles, or by an independent, external audit company.
Publish	See the Glossary.

View Table

Term	Definition
External audit	In-depth review conducted by the supreme audit institution (SAI) according to internationally recognized audit audit principles, or by an independent, external audit company.
Publish	See the Glossary.

Institutional Design

The aim of this dimension is to assess whether ex post audits of projects contribute to ensure transparency and accountability regarding the expenditure of public funds on capital projects. For the purpose of this dimension, an audit is one that is devoted exclusively to one major capital project or group of projects. An audit of the organization responsible for the project, which would include spending on the project, does not constitute an audit for this purpose. Audit should go beyond financial regularity and look explicitly at project implementation, final project costs compared with initial estimates, and whether planned outputs were achieved.

• A low score indicates that the legal mandate for the external auditor does not cover ex post, external project audits or allow publication of the audit results. The SAI legal mandate may restrict its ability to do this type of audit. Alternatively, ex post project audits may be allowed under the SAI mandate, but it is not allowed to publish the audit results.

• A medium score indicates that the legal mandate for the external auditor covers ex post, external project audits and allows publication of the results of these audits. The institutional design assessment focuses on whether there are any legal impediments to ex post audits and publication of audit results. The questions of whether such audits actually are conducted and if the audit results are published are discussed under “Effectiveness.”

• A high score indicates that the legal mandate for the external auditor covers ex post, external project audits and requires that the results be both regularly published and scrutinized by the legislature. Regular publication requires that the external auditor has a published policy describing what type of audit will be published and when. Audits are scrutinized if they are presented to a public accounts committee of the legislature or its equivalent. The scoring of design should not be based on what the legislature does with the audit information.

Important Documents

Documents	Uses
Legal framework for SAI	Assess institutional design
Annual or medium-term audit plans.	Assess criteria for selection of projects for ex post audit
Representative sample of ex post audit reports	Assess scope and comprehensiveness of ex post audits
Documentation of hearing by the legislature	Assess whether audits are scrutinized by the legislature

View Table

Documents	Uses
Legal framework for SAI	Assess institutional design
Annual or medium-term audit plans.	Assess criteria for selection of projects for ex post audit
Representative sample of ex post audit reports	Assess scope and comprehensiveness of ex post audits
Documentation of hearing by the legislature	Assess whether audits are scrutinized by the legislature

Effectiveness

A key indicator for the effectiveness of this dimension would be the number of audits that have been undertaken over the past three years and that have been published. This would be consistent with the criteria for institutional design:¹

• Low effectiveness means that no or very few major projects completed during the past three years have been subject to ex post audit.

• Medium effectiveness means that a few major projects have been subject to ex post audits that are published.

• High effectiveness means that some major projects have been subject to ex post audits that are published and scrutinized by the legislature. Box 7.13 describes ex post audit of capital projects in Mongolia.

Clear evidence of government decisions based on the audit findings may influence the effectiveness assessment. If audits have critical findings that are published and scrutinized by the legislature, this would be expected to affect how the government implements future projects. If such critical findings never lead to any legislative or executive actions, effectiveness might be lower than what is indicated by the numerically based score. On the other hand, if there are several clear examples that audits influence government decisions, the effectiveness score could be higher than the pure numerically based score indicated previously.

Useful Data Series

Data	Questions to Address
The number and percent of completed major projects that were audited, and the audits published	What is the share of completed and published audits?
Documentation of legislative or executive actions based on audit findings	How many audits have had documented effects on government decisions?

View Table

Data	Questions to Address
The number and percent of completed major projects that were audited, and the audits published	What is the share of completed and published audits?
Documentation of legislative or executive actions based on audit findings	How many audits have had documented effects on government decisions?

^{Box 7.13.}

Ex Post Audit of Capital Projects in Mongolia

In Mongolia, all capital projects are subject to ex post external audit. In addition to auditing annual capital expenditure execution for each budget entity, the National Audit Office (NAO) also undertakes ex post audits for all finished capital projects. NAO produces an annual audit report on capital budget execution, based on completed entity and project audits. NAO may comment on public investment policy, feasibility studies, and procurement policies affecting capital project execution. The NAO submits this report to the Parliament in June every year. This report is also published on the Auditor General Office’s website. Table 7.13.1 provides a summary of capital project execution in 2018, indicating that financial execution is 84.9 percent, while physical execution is 72.6 percent.

^{Table 7.13.1.}

Mongolian National Audit Office’s Report on Public Investment Execution

Source: Government of Mongolia 2019. Note: NDC = National Development Authority.

^{Table 7.13.1.}

Mongolian National Audit Office’s Report on Public Investment Execution

Portfolio Minister	No. of Projects	Total Budget (tugriks)	Annual Budget (tugriks)	Financial Performance		Prepayment, etc.		Performance of Actual Work
				Amount (tugriks)	Percentage	Amount (tugriks)	Percentage	Amount (tugriks)	Percentage
Cabinet of the President	3	770.7	770.7	754.3	97.9	-	-	754.3	97.9
Speaker of the Parliament	1	1,913.3	1,913.3	1,701.4	88.9	-	-	1,701.4	88.9
Prime Minister	8	5,685.3	3,762.8	2,709.9	72.0	-	-	2,709.9	72.0
Deputy Prime Minister	12	12,695.2	6,006.4	5,843.3	97.3	125.0	2.1	5,718.3	95.2
Head of the Cabinet of Ministers	32	23,920.9	11,881.8	10,459.8	88.0	580.0	4.9	9,879.8	83.1
Minister of Environment and Tourism	13	12,789.5	10,389.5	10,178.7	98.0	4,286.4	41.3	5,892.3	56.7
Minister of Foreign Affairs	3	1,800.0	1,800.0	1,800.0	100.0	-	-	1,800.0	100.0
Minister of Finance	14	78,098.1	32,810.0	20,786.3	63.4	-	-	20,786.3	63.4
Minister of Justice	40	59,590.9	37,390.0	37,204.3	99.5	-	-	37,204.3	99.5
Minister of Labor and Social Protection	25	36,723.3	17,340.0	16,852.6	97.2	1,650.9	9.5	15,201.7	87.7
Minister of Defense	4	19,417.4	3,599.5	3,334.8	92.6	-	-	3,334.8	92.6
Minister of Construction and Urban Development	78	430,501.9	80,338.4	67,408.4	83.9	10,608.2	13.2	56,800.2	70.7
Minister of Education	325	739,860.0	262,817.6	222,404.6	84.6	40,658.6	15.5	181,746.0	69.1
Minister of Road and Transportation Development	64	342,951.3	80,796.9	68,408.6	84.7	7,057.0	8.7	61,351.6	76.0
Minister of Mining	5	3,980.0	3,980.0	3,623.1	91.0	526.9	13.2	3,096.2	77.8
Minister of Agriculture and Industry	32	41,521.3	27,818.5	22,154.1	79.6	2,018.8	7.3	20,135.3	72.3
Minister of Energy	20	151,489.7	37,661.9	36,905.0	98.0	1,809.0	4.8	35,096.0	93.2
Minister of Health	63	193,319.5	60,754.8	46,422.7	76.4	14,554.9	24.0	31,867.8	52.4
Head of the Independent Authority Against Corruption	1	195.6	195.6	195.6	100.0			195.6	100.0
Head of NDC	1	340.0	340.0	339.4	99.8	-	-	339.4	99.8
Prosecutor General	4	1,562.0	452.0	450.8	99.7	-	-	450.8	99.7
Auditor General	1	500.0	500.0	498.0	99.6	-	-	498.0	99.6
Total	749	2,159,625.8	683,319.8	580,435.7	84.9	83,875.7	12.3	496,560.0	72.6

Source: Government of Mongolia 2019. Note: NDC = National Development Authority.

View Table

^{Table 7.13.1.}

Mongolian National Audit Office’s Report on Public Investment Execution

Portfolio Minister	No. of Projects	Total Budget (tugriks)	Annual Budget (tugriks)	Financial Performance		Prepayment, etc.		Performance of Actual Work
				Amount (tugriks)	Percentage	Amount (tugriks)	Percentage	Amount (tugriks)	Percentage
Cabinet of the President	3	770.7	770.7	754.3	97.9	-	-	754.3	97.9
Speaker of the Parliament	1	1,913.3	1,913.3	1,701.4	88.9	-	-	1,701.4	88.9
Prime Minister	8	5,685.3	3,762.8	2,709.9	72.0	-	-	2,709.9	72.0
Deputy Prime Minister	12	12,695.2	6,006.4	5,843.3	97.3	125.0	2.1	5,718.3	95.2
Head of the Cabinet of Ministers	32	23,920.9	11,881.8	10,459.8	88.0	580.0	4.9	9,879.8	83.1
Minister of Environment and Tourism	13	12,789.5	10,389.5	10,178.7	98.0	4,286.4	41.3	5,892.3	56.7
Minister of Foreign Affairs	3	1,800.0	1,800.0	1,800.0	100.0	-	-	1,800.0	100.0
Minister of Finance	14	78,098.1	32,810.0	20,786.3	63.4	-	-	20,786.3	63.4
Minister of Justice	40	59,590.9	37,390.0	37,204.3	99.5	-	-	37,204.3	99.5
Minister of Labor and Social Protection	25	36,723.3	17,340.0	16,852.6	97.2	1,650.9	9.5	15,201.7	87.7
Minister of Defense	4	19,417.4	3,599.5	3,334.8	92.6	-	-	3,334.8	92.6
Minister of Construction and Urban Development	78	430,501.9	80,338.4	67,408.4	83.9	10,608.2	13.2	56,800.2	70.7
Minister of Education	325	739,860.0	262,817.6	222,404.6	84.6	40,658.6	15.5	181,746.0	69.1
Minister of Road and Transportation Development	64	342,951.3	80,796.9	68,408.6	84.7	7,057.0	8.7	61,351.6	76.0
Minister of Mining	5	3,980.0	3,980.0	3,623.1	91.0	526.9	13.2	3,096.2	77.8
Minister of Agriculture and Industry	32	41,521.3	27,818.5	22,154.1	79.6	2,018.8	7.3	20,135.3	72.3
Minister of Energy	20	151,489.7	37,661.9	36,905.0	98.0	1,809.0	4.8	35,096.0	93.2
Minister of Health	63	193,319.5	60,754.8	46,422.7	76.4	14,554.9	24.0	31,867.8	52.4
Head of the Independent Authority Against Corruption	1	195.6	195.6	195.6	100.0			195.6	100.0
Head of NDC	1	340.0	340.0	339.4	99.8	-	-	339.4	99.8
Prosecutor General	4	1,562.0	452.0	450.8	99.7	-	-	450.8	99.7
Auditor General	1	500.0	500.0	498.0	99.6	-	-	498.0	99.6
Total	749	2,159,625.8	683,319.8	580,435.7	84.9	83,875.7	12.3	496,560.0	72.6

Source: Government of Mongolia 2019. Note: NDC = National Development Authority.

Questionnaire

Low	Asset registers are neither comprehensive nor updated regularly.
Medium	Asset registers are either comprehensive or updated regularly at reasonable intervals.
High	Asset registers are comprehensive and updated regularly at reasonable intervals.

View Table

Low	Asset registers are neither comprehensive nor updated regularly.
Medium	Asset registers are either comprehensive or updated regularly at reasonable intervals.
High	Asset registers are comprehensive and updated regularly at reasonable intervals.

Definitions of Key Terms

Term	Definition
Asset register	A documented list of fixed assets
Comprehensive	In this context, a comprehensive asset register includes most fixed assets purchased using money appropriated in the budgetary central government capital budget, and for each asset lists at least: Name and description Organization accountable for it Location Date of purchase, delivery, or installation Historic cost Disposal status Asset class Current value
Update	Implies that new assets are added to the register, assets disposed of are deleted, and significant changes in existing assets are recorded.
Updat regularly	asset registers are updated at fixed dates or on rolling dates on fixed intervals, defined in a regulation, manual, or written instruction.
Reasonable interval	Ideally, an asset register should be updated at least every two years.

View Table

Term	Definition
Asset register	A documented list of fixed assets
Comprehensive	In this context, a comprehensive asset register includes most fixed assets purchased using money appropriated in the budgetary central government capital budget, and for each asset lists at least: Name and description Organization accountable for it Location Date of purchase, delivery, or installation Historic cost Disposal status Asset class Current value
Update	Implies that new assets are added to the register, assets disposed of are deleted, and significant changes in existing assets are recorded.
Updat regularly	asset registers are updated at fixed dates or on rolling dates on fixed intervals, defined in a regulation, manual, or written instruction.
Reasonable interval	Ideally, an asset register should be updated at least every two years.

Institutional Design

The aim of this dimension is to determine whether information on existing assets is known. This is an important basis for the production of information that is needed for other institutions. Asset registers are often maintained by line ministries and include specialized information for management purposes. For example, a roads department frequently has a register of road and bridges that includes detailed design and condition information needed for planning maintenance. Individual assets must be identifiable in an asset register, meaning that non-financial assets reported in financial accounts do not constitute an asset register.

• A low score indicates that there is no legal or regulatory requirement for comprehensive, regularly updated fixed asset registers. In some countries, legal or regulatory mandates for asset registers are missing completely. In other countries there may be regulatory requirements for asset registers in some institutions or for some types of assets, but not systematically across government.

• A medium score indicates that there are fixed asset registers and that these are required to be either comprehensive or regularly updated. If most of the government entities have asset registers, then the system is comprehensive. This would qualify for a medium score even if the register only provides historic costs and there is no mechanism for updating. If only a ministry of transport enters information in the asset register with no other ministry, then the asset register is not comprehensive. However, if this ministry of transport each year enters information on new roads and bridges, then it is updated regularly. For the regularity requirement to be meaningful, it must cover a reasonable share of government assets.

• A high score means that the registers are required by law or regulation to be both comprehensive and regularly updated. The legal framework should include provisions to ensure that the information in the asset registers is accessible, and that it is updated at least every second year.

Important Documents

Documents	Uses
Legal and regulatory framework for asset registration	Assess institutional design
Guidelines and templates for asset registration	Assess comprehensiveness and timeliness of asset register updates
Documentation of asset registry	Analyze actual contents

View Table

Documents	Uses
Legal and regulatory framework for asset registration	Assess institutional design
Guidelines and templates for asset registration	Assess comprehensiveness and timeliness of asset register updates
Documentation of asset registry	Analyze actual contents

Effectiveness

The effectiveness assessment should gauge whether the information in fixed asset registers is in line with the formal requirements. Do the actual data match the requirements in terms of comprehensiveness? Are the data updated regularly, within the timeframes defined in regulations? Are there specific challenges that may undermine the comprehensiveness or the quality of the data?

In addition to being comprehensive and regularly updated, information on fixed assets must also be accessible. The asset register is used for the purposes of preparing financial statements, national and sectoral planning, and budgeting for maintenance (Institutions 1, 2, and 9). This dimension is concerned with accessibility of information across organizations and projects, not within a single organization. If information is not easily accessible to meet the needs of these three institutions, it is not effective. It is unlikely that a register that is not centralized is fully effective in this regard.

Unless there is a comprehensive government or public sector balance sheet (see dimension 15.b), the assessment of comprehensiveness could be based on comparing the contents of the fixed asset register with the PIMA estimate of capital stock.

• Low effectiveness: There is no centralized register of fixed assets, or fixed asset registers maintained by respective agencies have only partial coverage.

• Medium effectiveness: Either there is a centralized fixed asset register or fixed asset registers maintained by respective agencies are regularly updated. The registers should cover most government fixed assets and be readily accessible.

• High effectiveness: Fixed asset registers are maintained or consolidated centrally and are verified and updated at least every two years to ensure their comprehensive coverage. The registers should cover all government fixed assets. Box 7.14 describes the asset registry in Indonesia.

Useful Data Series

Data	Questions to Address
Overview of assets in centralized and/or decentralized asset registers	What is the volume of assets recorded in government asset registers? What is the level of detail contained in the asset register, such as asset classes, type of asset, volume physical location, control?
If asset registers are not centralized, a list of ministries/agencies that keep asset registers and with what type of data	Which ministries keep separate asset registers? Are these registers accessible to the MoF?
Dates when asset values and conditions were revised in last 10 years	How often are asset values updated?

View Table

Data	Questions to Address
Overview of assets in centralized and/or decentralized asset registers	What is the volume of assets recorded in government asset registers? What is the level of detail contained in the asset register, such as asset classes, type of asset, volume physical location, control?
If asset registers are not centralized, a list of ministries/agencies that keep asset registers and with what type of data	Which ministries keep separate asset registers? Are these registers accessible to the MoF?
Dates when asset values and conditions were revised in last 10 years	How often are asset values updated?

^{Box 7.14.}

Fixed Asset Recording in Indonesia

Fixed assets are comprehensively recorded in Indonesia, and the data are used for reporting nonfinancial assets and depreciation in financial statements. The MoF Directorate General for State Asset Management (DGSAM) is responsible for tracking the purchase, use, transfer, and disposal of nonfinancial assets. To carry out this responsibility, it has created the State-owned Asset Financial Accounting Management System, SIMAK-BMN, which was introduced in 2006–07. SIMAK data include land, buildings, infrastructure, and equipment. All civilian line ministries enter data into the system continuously and submit formal reports semiannually to one of 71 operational offices of DGSAM located around the country. Data include the nature of the asset, age, historical cost, current value (if revalued), location, responsible agency, condition, asset class, and state of disposal. Assets are revalued when ordered by the president and revaluation was done in 2006 and 2018. DG Treasury includes fixed assets in the financial statements, using data from SIMAK. Depreciation is calculated on a straight line, based on rates for specific asset classes applied to asset-specific data in SIMAK.

SIMAK data are reliable and used for planning capital projects. Asset data from SIMAK are used by line ministry planning and budgeting staff to evaluate proposed capital projects. Existing facilities are identified using SIMAK, and judgments are made as to whether a new or expanded facility is required. The staff is also involved in evaluating asset disposal. Condition and value are verified in SIMAK before approval for disposal is given. The scope and quality of data in SIMAK are reasonably good overall. To verify data accuracy regarding newly purchased assets, spending units and DGSAM reconcile the data with capital spending recorded in SPAN for the comparable period. In addition, the Audit Board of Indonesia (BPK) audits SIMAK annually. For 2019, BPK issued an unqualified opinion at the central government level. Table 7.14.1 shows government fixed assets at the end of 2018 and 2019, illustrating a major revaluation in particular of government land during 2019.

^{Table 7.14.1.}

Fixed Assets of the Indonesian Government, 2018 and 2019

(Billions of rupiah)

Sources: Government Financial statement years 2018 and 2019; Indonesia PI MA 2019; and IMF staff compilations. Note: SPAN = State Treasury and Budget System.

^{Table 7.14.1.}

Fixed Assets of the Indonesian Government, 2018 and 2019

(Billions of rupiah)

Asset	2019	2018
Land	4,565,754	1,018,648
Equipment and machinery	643,684	590,287
Buildings	365,443	287,028
Roads and other networks	852,163	593,241
Other fixed assets	50,631	55,538
Construction in progress	137,289	130,585
Total fixed assets (gross)	6,614,964	2,675,327
Accumulated depreciation	665,369	744,276
Total fixed assets	5,949,595	1,931,051

Sources: Government Financial statement years 2018 and 2019; Indonesia PI MA 2019; and IMF staff compilations. Note: SPAN = State Treasury and Budget System.

View Table

^{Table 7.14.1.}

Fixed Assets of the Indonesian Government, 2018 and 2019

(Billions of rupiah)

Asset	2019	2018
Land	4,565,754	1,018,648
Equipment and machinery	643,684	590,287
Buildings	365,443	287,028
Roads and other networks	852,163	593,241
Other fixed assets	50,631	55,538
Construction in progress	137,289	130,585
Total fixed assets (gross)	6,614,964	2,675,327
Accumulated depreciation	665,369	744,276
Total fixed assets	5,949,595	1,931,051

Sources: Government Financial statement years 2018 and 2019; Indonesia PI MA 2019; and IMF staff compilations. Note: SPAN = State Treasury and Budget System.

Questionnaire

Low	Government financial accounts do not include the value of non- financial assets.
Medium	Government financial accounts include the value of some nonfinancial assets, which are revalued irregularly.
High	Government financial accounts include the value of most nonfinancial assets, which are revalued regularly.

View Table

Low	Government financial accounts do not include the value of non- financial assets.
Medium	Government financial accounts include the value of some nonfinancial assets, which are revalued irregularly.
High	Government financial accounts include the value of most nonfinancial assets, which are revalued regularly.

Definitions of Key Terms

Term	Definition
Government	Financial statement of accounts financial as defined in national accounting account standards.
Revalue	To value an asset after initial delivery to ascribe the current value of the asset.
Irregularly	Not regularly.
Regularly	See definition in Dimension 15.a.

View Table

Term	Definition
Government	Financial statement of accounts financial as defined in national accounting account standards.
Revalue	To value an asset after initial delivery to ascribe the current value of the asset.
Irregularly	Not regularly.
Regularly	See definition in Dimension 15.a.

Institutional Design

The purpose of this dimension is to assess whether government financial statements provide information on nonfinancial fixed assets (for example, see Box 7.15). This information assists in the calculation of capital stock and net worth, and thus contributes to institution 1. There is some synergy between Dimension 15.a (asset register) and this dimension. Dimension 15.a addresses knowledge of the characteristics of nonfinancial assets. This dimension addresses the accounting value of nonfinancial assets.

• A low score implies that there is no legal or regulatory requirement that government financial statements provide systematic information about nonfinancial assets. There may be some ad hoc information available in government unit accounts, but this information is not included in the government financial statements in a systematic and comprehensive manner.

• A medium score implies that there is a legal or regulatory requirement for inclusion of some fixed assets in the government financial statements, but not for revaluation of these assets on a regular basis. A medium score also applies if most assets are required to be included but there is no provision for them to be revalued regularly, or if few assets are required to be included but they should be revalued regularly.

• A high score implies that most fixed assets are required to be included in the government financial statements, and that these should be revalued regularly.

Important Documents

Documents	Uses
Legal basis for government accounts	Assess legal requirements for inclusion of nonfinancial assets in government financial statements
Accounting regulations, standards, and guidelines	Assess accounting standards and guidelines for accounting and revaluation of asset values
Government financial statements for the past 3–5 years.	Assess whether and how nonfinancial assets are disclosed in actual financial statements (effectiveness).

View Table

Documents	Uses
Legal basis for government accounts	Assess legal requirements for inclusion of nonfinancial assets in government financial statements
Accounting regulations, standards, and guidelines	Assess accounting standards and guidelines for accounting and revaluation of asset values
Government financial statements for the past 3–5 years.	Assess whether and how nonfinancial assets are disclosed in actual financial statements (effectiveness).

Effectiveness

The effectiveness of this dimension depends on the share of government assets that are included in the financial statements. The assessment of comprehensiveness could be based on comparing the stock of non-financial assets in the government accounts with the PIMA estimate of capital stock.

• Low effectiveness means that few government fixed assets are included in the accounts.

• Medium effectiveness means that some government fixed assets are included in the accounts.

• High effectiveness means that most are included in the accounts, and that these are revalued regularly. Box 7.15 describes accounting of government assets in Estonia.

Useful Data Series

Data	Questions to Address
Value of nonfinancial assets in government financial statements	How comprehensive is the coverage of nonfinancial assets in the financial statements?
Volume of assets in asset registry	Are the financial statements consistent with the asset registry?
Estimated public capital stock from PIMA database	Is the value of nonfinancial assets in the financial statements comparable to estimated public capital stock?
External audit reports	Does the external audit report have observations regarding recording or valuation of government non-financial assets?

View Table

Data	Questions to Address
Value of nonfinancial assets in government financial statements	How comprehensive is the coverage of nonfinancial assets in the financial statements?
Volume of assets in asset registry	Are the financial statements consistent with the asset registry?
Estimated public capital stock from PIMA database	Is the value of nonfinancial assets in the financial statements comparable to estimated public capital stock?
External audit reports	Does the external audit report have observations regarding recording or valuation of government non-financial assets?

^{Box 7.15.}

Accounting for Fixed Assets in Estonia

Estonia introduced accrual accounting for government in 2000, and all government assets are appropriately recorded and accounted for in the financial statements. National accounting standards are based on International Public Sector Accounting Standards and cover all material parts of these standards. According to law, all government assets are identified by unique serial numbers and included in government asset registries. Asset values are reviewed and updated annually. Depreciation schedules are adjusted when capital maintenance projects are undertaken. All central government entities (approximately 150) do their accounting through a common accounting application, managed by the Shared Service Center under the MoF. For these entities, depreciation of assets is computed monthly, on the basis of depreciation rates determined by each entity but following central guidelines. All other general government and public sector entities (approximately 850) use separate accounting applications but submit monthly accounting reports in a predefined format for consolidation with central government reports.

State Audit Office reports confirm that government accounting and financial statements are comprehensive and of high quality. Financial statements are consolidated and reported at three different levels: Central government, local government, and consolidated public sector, including corporations controlled by central or local government entities. There are detailed statements of public assets, including depreciation, acquisition, revaluations and disposal during the year. The consolidated statements are based on the central government accounting system and the reports provided by the other public sector entities. These reports do not include transaction-level data but provide the necessary detail to allow for reconciliation and elimination of internal transactions, including aggregate transactions with other government entities. For the 2016 accounts, the State Audit Office found that the accounts generally gave a fair and true value of financial transactions and values. However, the audit report observed that the auditor could not confirm the recorded value of assets in the Railway Infrastructure Company, given negative developments in the railway freight market. This was the only main observation regarding public assets.

^{Table 7.15.1.}

Nonfinancial Assets in Estonia, 2017

(Millions of euro)

Source: Estonia PIMA 2019. Note: NA = not applicable.

^{Table 7.15.1.}

Nonfinancial Assets in Estonia, 2017

(Millions of euro)

	Land	Buildings	Defense	Equipment	Other	Work in Progress	Total
Balance	1,277.0	8,430.1	213.3	2,495.3	159.8	1,038.8	13,614.3
Acquisition cost	1,277.0	13,302.4	597.5	4,881.7	297.0	1,038.8	21,394.4
Accumulated depreciation	0.0	4,872.3	384.2	2,386.4	137.2	0.0	7,780.1
Average depreciation rate (%)	NA	3.7	7.5	5.4	5.1	NA	NA

Source: Estonia PIMA 2019. Note: NA = not applicable.

View Table

^{Table 7.15.1.}

Nonfinancial Assets in Estonia, 2017

(Millions of euro)

	Land	Buildings	Defense	Equipment	Other	Work in Progress	Total
Balance	1,277.0	8,430.1	213.3	2,495.3	159.8	1,038.8	13,614.3
Acquisition cost	1,277.0	13,302.4	597.5	4,881.7	297.0	1,038.8	21,394.4
Accumulated depreciation	0.0	4,872.3	384.2	2,386.4	137.2	0.0	7,780.1
Average depreciation rate (%)	NA	3.7	7.5	5.4	5.1	NA	NA

Source: Estonia PIMA 2019. Note: NA = not applicable.