Chapter 1. Introduction
- Moisés Schwartz, and Ray C. Rist
- Published Date:
- January 2017
The underlying premise of this book is that a strong global economy requires a strong International Monetary Fund (IMF), and that a strong IMF requires a strong independent evaluation culture and practice.
For more than 70 years, as the principal multilateral institution responsible for global economic and financial stability, the IMF has faced enormous challenges. The world economy has undergone a multitude of crises, some of which have affected individual economies or regions while others have been global. Over the years, the Fund has continuously revisited and adjusted to frequently changing circumstances of the world economy via its surveillance activities, lending instruments, and technical assistance. As one of the oldest and most established of the international financial institutions (IFIs), the Fund has faced expectations for immediate responses and high performance in times of crisis from all corners of the globe.
Though some observers see the Fund’s responses to crises over the years as reasonably appropriate and believe that the Fund has learned from these instances, the economic turbulence of the last decade has shocked this and other IFIs and added to the apprehension felt by parts of the membership and other stakeholders. The membership of some of these IFIs has increasingly questioned their mandate and scope of activities, and their relevance as appropriate institutional responses to global issues. This questioning—along with some of the IFIs’ governance structures, which lack adequate voice and representation of their full membership (particularly from developing countries)—have put these organizations at a crossroads.
Dissatisfaction by part of the membership regarding the handling of crisis-related episodes, and the lack of proper voice and representation of emerging markets in IFIs, have motivated the creation of alternative arrangements such as the Chiang Mai Initiative and institutions such as the BRICS New Development Bank and the Asian Infrastructure Investment Bank. Some emerging market countries, while not disengaging from the more experienced, established, and global institutions are resorting to regional or more partial alternatives to address some of their specific concerns.
Some observers have suggested that the current incremental and piecemeal approach to governance reform in some IFIs will take a very long time to bear fruit and that a more far-reaching approach needs to be considered, such as a new international Bretton Woods–style conference that would address issues such as the continuing shift in global economic power, the Fund’s mandate, its surveillance, financial resources, and the role of the SDR (Boorman and Icard, 2011; Rajan, 2008; and Woods, 2009).
The IMF’s governance structure has been a source of both internal and external dissatisfaction. Had the IMF been created recently, members’ voting shares and representation would likely differ from what they are today. While some progress has been made in updating the governance structure to reflect current global economic realities, the realignment process has been slow, to the frustration of many fast-growing and increasingly globally integrated emerging market countries.
There is no doubt that the IMF needs ongoing reform. Its mandate, governance, and activities must evolve to make them more compatible with current world economic circumstances, and with what the membership expects from the institution. While other groupings or bodies have emerged whose contribution is indisputable, such as the G20 and the Financial Stability Board, the IMF has a global membership and global reach. Moreover, the Fund’s qualified staff and worldwide experience make it the only institution capable of delivering on global priorities for financial and economic stability. To be fully prepared to meet the great macroeconomic challenges of the 21st century, the world needs a strong and effective IMF that satisfactorily reflects the growing demands and expectations of its membership.
While confidence in the IMF among the membership unquestionably should be reinforced primarily through governance reform that recognizes the shifts in global economic power, the institution must also carry out its functions effectively and be seen to be doing so. Further enhancing the role of independent evaluation in the IMF and its integration in the institution’s culture, in our view, offers a vital tool for achieving this. As elaborated in this book, independent evaluation is essential in providing credibility and legitimacy to the IMF—particularly among those who may perceive that the institution is dominated by, or serves the interests of, a narrow group of members. Independent evaluation can take the IMF to a higher level of effectiveness with a more targeted emphasis on, and attention to, both accountability and learning. While evaluation has long played a function in the IMF, and its role has expanded substantially with the creation of the Independent Evaluation Office (IEO), we contend, however, that independent evaluation has yet to take on a role within the IMF that fully reflects its potential contribution.
The Twin Pivots of Accountability and Learning
This book addresses the two main objectives of independent evaluation: accountability and learning from experience. But more specifically, the book focuses on the challenges that the IMF faces with respect to becoming a true learning organization by integrating independent evaluation into its culture.
For the IMF to become a more effective organization through the use of independent evaluation, three conditions have to be met: being “low on fear” among the staff, such that everyone feels free to challenge conventional wisdom; being “information rich,” such that knowledge is abundant and available within and between organizational divisions and the institutional culture constantly seeks new information; and being open to criticism and learning, such that the institution admits to problems and tries to address them. Achieving these conditions depends on having mechanisms in place for both accountability and learning—the core elements of evaluation.
Evaluation provides information on accountability, by conducting assessments of performance against standards, objectives, previous performance, and financial and institutional resources. Evaluation helps an institution learn, by identifying lessons from experience that can be incorporated in subsequent policies or operational practices, and by examining how the organization may need to change the prevailing behavior in order to improve performance.
The benefits of independent evaluation for accountability and learning in IFIs have long been recognized (see Chapter 3). Since these institutions are not subject to market forces, which tend to drive inefficient entities out of the market, independent evaluation helps to identify ineffective outcomes and potential corrective actions as well as effective processes and outcomes that can be replicated. In doing so, it can help to increase efficiency and confidence in the institution and to avoid protracted periods of suboptimal outcomes that represent a cost to society.
Evaluation, and its two pillars of accountability and learning, acquire increased relevance during turbulent times (Furubo, Rist, and Speer, 2013). Thus, besides the traditional arguments in favor of evaluation in IFIs, independent evaluation for the IMF is perhaps now more important than ever before. This is because in the current turbulent times evaluation can serve as a unifying force that keeps the membership together, a “glue” that helps the Fund to stay focused on its objectives as it provides assurances to the membership that it is a credible, legitimate, and learning institution. Hence the Fund, now more than before, must embrace independent evaluation and use it as a tool to “speak truth to power” and thus provide confidence to its members and other stakeholders.
Independent Evaluation at the IMF
Reform initiatives at the IMF have typically focused on areas such as governance, mandate, surveillance, lending instruments, and technical assistance. At different times, depending on circumstances, stakeholders have held diverse views on what each of the reform components should entail. And what about independent evaluation?
The creation of the IEO in 2001 was an important milestone in the IMF’s history, and itself a crucial IMF reform at that time. But since then the discussion of IMF reform has barely referred to the concept or the relevance of independent evaluation. It is as if the creation of the IEO represented an end in itself, as opposed to a beginning of a new era in which evaluation would be a vibrant element of the organization that requires continuous nurturing.
The IEO is a small but active unit, with a pipeline of reports that have contributed to learning and institutional change within the Fund. These reports have strengthened the Fund’s external credibility, and thus have also served as an accountability device. Since its inception in 2001, the IEO has to date produced 26 evaluation reports covering a wide range of Fund activities and operations.1 While readers may have preferred some of these reports over others, there is a clear recognition that the cumulative weight of the IEO’s analyses has contributed significantly to making the Fund a stronger institution. Evaluation reports are discussed in global capitals and at IMF Executive Board meetings, and many of the recommendations they contain have been endorsed by the Board. Follow-up by IMF management on Board-endorsed evaluation recommendations has in some cases been slow, but many of IEO findings and recommendations have filtered through and affected the institution’s policies and practices.
While the process in place suggests considerable success for independent evaluation at the IMF, our view is that the reform that brought about independent evaluation at the IMF is not finished: the establishment of the IEO was only the start of a process and a series of outcomes that still need to be fostered and cultivated. Successful independent evaluation is important for the IMF to be perceived as legitimate and credible—and to achieve it, the independent evaluation function needs to be further integrated in the learning process and culture of the Fund, so that it can be utilized to its full potential and hence contribute to the IMF in the most effective way.
What Needs to Change?
While there is a clear recognition that the IEO has been successful (Ocampo, Pickford, and Rustomjee, 2013), it is our contention that it has mainly been perceived as an accountability device rather than as a mechanism for the Fund to learn from experience. To fully reap all the benefits that the IEO has to offer, the Fund also needs to perceive independent evaluation as a learning mechanism. For this to take place, an evaluation culture needs to mature and flourish within the IMF. Such a culture legitimizes independent evaluation and uses evaluation findings and insights for policymaking, performance improvements, and organizational renewal. Though some progress has been attained in this regard, the Fund still needs to go some way before a strong evaluation culture is widespread and established within the organization.
Since its inception the IEO has implemented several procedural changes, most of which have come as a result of the two external evaluations of the IEO. As detailed in later chapters, some of these changes have been noteworthy while others have been more subtle; some have been immediate while others have been implemented over time. We maintain that a more fundamental transformation is needed. Further procedural improvements, while helpful, will not capture all the benefits that independent evaluation can bring to the Fund. This book seeks to alter the perception that it is only IEO’s operational procedures that need to change and rather to promote the recognition that independent evaluation can help to transform the Fund into a more effective organization that learns from experience.
What would it take for independent evaluation to play this role? Would the culture of the Fund need to change as well? What type of culture would be needed? Is there a role for the leadership of the organization to attain such an objective? Can independent evaluation provide both accountability and learning elements in the IMF? Can the IMF adequately and appropriately respond to issues of accountability? Can the IMF learn from independent evaluation? This book addresses these questions as a necessary part of any debate about whether and how the IMF can become a more effective organization.
Outline of This Book
Figure 1.1 depicts the plan of the book and describes the flow of events or prerequisites that need to take place in order for independent evaluation to successfully contribute to the IMF’s transformation into what the literature defines as a “learning organization” (see Box 1.1).
Figure 1.1.The Role of Independent Evaluation in Fostering a Learning Organization Box 1.1The Learning Organization
The concept of the learning organization and its relevance to the IMF is discussed in detail in Chapter 7. This box provides an introductory definition.
A learning organization has been described as:
[A] form of organization that enables the learning of its members in such a way that it creates positively valued outcomes, such as innovation, efficiency, better alignment with the environment and competitive advantage. Again, the focus is not so much on the process of learning but more on conditions that may allow successful outcomes to flourish (Huysman, 1999: 61).
Another definition of a learning organization is:
. . . a space for generative conversations and concerted action. In them, language functions as a device for connection, invention, and coordination. . . . When people talk and listen to each other this way, they create a field of alignment that produces tremendous power to invent new realities in conversation, and to bring about these new realities in action (Kofman and Senge, 1995: 33).
Most organizations have some degree of organizational learning, but this does not imply that therefore a learning organization has been created. Generating the latter requires a deliberate initiative from management that is sustained over time. Learning can be taking place throughout an organization, but sporadically, haphazardly, and without direction. To achieve a learning organization requires systematic, sustained efforts, clarity of purpose, and clarity of intentions.
As seen in the different chapters of the book, some of the stages in the flow chart already take place, while others need to be developed or reinforced. Currently, independent evaluation contributes to organizational learning/use of new knowledge by the Fund, but not yet to the IMF reaching the ideal of becoming a learning organization. That is, what the IEO represents and produces has affected the Fund’s learning and provided accountability for the Fund’s actions, and IEO reports have had a positive effect on the processes the Fund uses to absorb and apply the new knowledge generated by independent evaluation. But the crucial elements of culture and management leadership that would be needed for the Fund to transform itself into a learning organization are still largely missing.
The remaining nine chapters of this book address the main phases of Figure 1.1. Following this introduction, Chapter 2 provides a brief history of the IEO and the motivations for its establishment. Chapter 3 relates the relevance of the IEO in changing the institutional framework governing relationships among the stakeholders of the IMF and how the establishment of an independent evaluation function profoundly altered the prevailing contract regarding the conduct of evaluation within the Fund. Chapter 4 addresses the relationship and tensions between accountability and learning in the conduct of evaluation and poses the question of whether these purposes of evaluation need to be framed in terms of one or the other. Chapter 5 further analyzes how accountability and learning have featured in the IEO’s work and presents evidence about the degree to which IEO’s findings incorporate these two elements. The chapter concludes that since its creation the IEO has successfully fostered both accountability and learning. Chapter 6 discusses how structural elements associated with the IEO’s independence have caused senior staff and management to see the IEO as primarily an accountability instrument, and only in a more subordinate role as a learning device for the organization. The chapter asserts that this has affected IEO’s effectiveness and has shaped relations between the IEO and the rest of the organization.
Chapter 7 addresses the tripartite relationship among organizational learning, the learning organization, and independent evaluation. Most organizations have some degree of organizational learning, but only some would be considered learning organizations. The chapter contends that independent evaluation plays a role in promoting a learning organization. Chapter 8 examines how new knowledge produced by the IEO is received, resisted, or ignored by the IMF. Independent evaluation does, indeed, generate new knowledge, and its absorption represents organizational learning. The chapter concludes that the IEO has been an important element in contributing to organizational learning within the Fund, but that the Fund has not yet been able to use independent evaluation optimally to transform itself into a learning organization. Chapter 9 discusses the role of organizational culture and leadership in the promotion of a learning organization; it considers whether the culture of an organization can change and the role that independent evaluation can play in that regard. Chapter 10 concludes, calling on IMF management to take a more active role in instilling the positive value of independent evaluation across the organization and thus enabling independent evaluation to bring the IMF closer to the ideal of a learning organization.
The full list of IEO evaluation reports (2002–16) is presented in Annex 1. The IEO has to date also revisited six of its previous evaluation reports. These evaluation updates aim to determine whether the main findings and conclusions of the original IEO evaluations remain relevant and to identify any outstanding and or new issues related to the evaluation topic that merit continued attention. The list of evaluation updates is also presented in Annex 1. Summaries of IEO evaluations (2002–16) are presented in Annex 2.