News and Highlights

Mr. Hao Lei, Deputy Director General of the Budget Department, Ministry of Finance,

Dr. Wencai Zhang, Director General of the International Financial Cooperation Department, Ministry of Finance and former Vice President of ADB,

Mr. Zhang Yansong, Director General of the Yunnan Provincial Department of Finance,

Mr. Shixen Chen, Vice President of ADB,

Dr. Li Kouqing, President of the Asia-Pacific Finance and Development Institute and Shanghai National Accounting Institute,

Distinguished officials from the Ministry of Finance and Yunnan Provincial Government of the People’s Republic of China,

Friends, colleagues and partners in evaluation and in development,

A pleasant good morning to you! Zǎoshang hǎo! 

Welcome to the 4th Asian Evaluation Week here in the City of Kunming, a place rich in history spanning nearly 2,400 years. Already at his time, the famous explorer Marco Polo described Kunming as a “large and splendid city” upon seeing its great prosperity, the diversity of its people and the abundance of its land. 

As the southwest gateway to the famed Silk Road, Kunming’s thriving trade facilitated the convergence of people, goods and knowledge thereby supporting the link between the East and the West. We could not have chosen a more appropriate place to hold our annual gathering of evaluators from across Asia and from around the world for stimulating meaningful partnerships in advancing our evaluation discipline.

Before I delve into our central theme for this year’s AEW, let me pause for moment to pay special tribute to Farzana Ahmed, our colleague and friend, whose determination, never ending energy and intellectual leadership were critical for the launching of AEW four years ago. Farzana will be retiring from ADB soon and this will be her last meeting as an ADB/IED staff. Thank you Farzana for everything, we will miss you and hope to see you here again as part of your new mission. 

This year’s AEW theme is titled “Quality Evaluation for Better Results: Local, National, Regional Perspectives”. I cannot emphasize enough the demands that the evaluation field is now facing to help achieve the global development results outlined in the 2030 Agenda. Concerted efforts at the local, country and regional levels are essential to realize these goals and evaluation is called to play a central role in this endeavor.

Quality evaluation 

The expectations about evaluation contributing to advancing today’s development agenda hinge crucially on the quality of our assessments. This is one of these cases in which as the Roman philosopher Seneca succinctly puts it, “It is quality rather than quantity that matters.” 

But what do we really mean by quality evaluation? Do we have a shared understanding within the evaluation community on this matter?  In answering these questions, let me indicate in the first place that evaluation in itself is an exercise in the search for quality.  In this search, there are two dimensions of quality that we must be all clear about: 1) quality of the evaluation itself; and 2) quality of the evaluand or the object of evaluation. 

Quality of the evaluation itself

Let me address the first dimension, or the quality of evaluation itself. When speaking of this subject, this dimension is the one that receives most of the attention. Clearly, views on quality of evaluation are diverse, arising from individual work experiences, and the many standards and principles that have been adopted. 

However, distilling through the various evaluation policies, principles and standards across institutions and organizations, we find a common thread on three areas defining quality of the evaluation work: a) expertise and methodology; b) enabling environment; and c) utilization and fit for purpose. These three areas complement each other to determine the quality of an evaluation. 

Concerning expertise and methodology, these refer to the evaluators’ competencies (i.e. proficiency, analytical and technical skills) and the necessary moral compass that should guide them in undertaking evaluation work, as well as to methodologies underpinned by a high degree of rigor, particularly related to validity, reliability and replicability. 

With respect to the second area, an enabling environment works hand-in-hand with strong expertise and rigorous methodology. By enabling environment, I refer to the institutional underpinnings of the evaluation function with a focus on independence, that is, on structural and functional independence. 

Lastly on the area of utilization and fit for purpose, quality is seen from the perspective of stakeholders and end-users of evaluation. What I mean is that evaluation should meet the expectations and knowledge needs of users for them to be able to do the right things and do them right.

Quality of the evaluand or the object of evaluation

Let me now turn to the second dimension of the search for quality in evaluation, that is the quality of the evaluand or the object of evaluation. We should not lose sight of the importance of this dimension, for our work is essentially about finding and judging quality in everything we assess. As Robert Stake has stated, two distinct aspect that are necessary for defining the quality of an evaluand: 1) criteria, which refer to the attributes to judge the evaluand; and 2) standards, which denote the amount of these attributes needed for arriving at a specific judgment.

On criteria, in the case of ADB and several other organizations here present, for instance, the most used criteria are relevance, effectiveness, efficiency, and sustainability. On the other hand, standards are the bars within each criterium to define, for example, if interventions are highly relevant, relevant or less than relevant. 

The distinction between criteria and standards is often confused by professionals in the field of evaluation. This confusion must be avoided to appropriately identify quality, merit, and worth in the object of evaluation.

Excellence and influence of evaluation

Within the context of quality evaluation work I have just described, let me pause for a moment and pay special attention to two important aspects of this concept: excellence and influence. 

These aspects are exemplified by Zhang Qian, China’s famous and great envoy of the Han Dynasty, who despite being held in captivity for 12 years on his mission to explore the West, proficiently delivered on this mission and brought back influential knowledge that paved the way for the development of the Silk Road and opened the communication of China with the rest of the world. 

And so, on excellence in evaluation, good evaluators should be guided by the personal and intrinsic urgency of delivering on their mission, meeting the highest standards, while being aware of the consequences that our assessment can have. We should always keep in mind, as I have said in previous years, that evaluation has real consequences. 

Concerning influence of evaluation, the mere submission of good quality reports cannot be the only standard that defines our work. The final test lies on the influence of our contribution. I refer to the direct or instrumental impact, the conceptual or contextual influence, and the advocacy or persuasion, attributed to evaluation.

As Mark and Henry have pointed out, a comprehensive view of the mechanisms underlying evaluation’s influence can help ensure that our work fulfills its objective and role in the service of social betterment. Certainly, in assessing the global progress on the SDGs, a comprehensive view of influence mechanisms should involve local, country and regional perspectives, which explains the intellectual and strategic focus of this meeting.

Concluding remarks

Colleagues and friends, Kunming is also known as the City of Eternal Spring – favorable climate, passionate people, and plants and flowers that bloom all year long. Let us all be inspired by this wonderful framework to stimulate our exchanges during the course of this week to contribute to quality evaluation with an emphasis on excellence and influence of our practice at all levels – local, national and regional. And let us all be guided throughout this week by the wise precept of Confucius: “Learn avidly, question repeatedly, analyze carefully, and put what you have learned into practice, intelligently.” 

Thank you all for joining us! Xiè xiè!

Marvin Taylor-Dormond

Director General
Independent Evaluation Department Asian Development Bank

What it's like to attend

  • I hope it is not too late to express how glad I am to have participated in the AEW 2018 and how useful I found the sessions. The conversation we had at the Panel also facilitated our conversation with the IMF team of the independent report. We agreed to advance the conversation during the upcoming spring meeting. So the AEW was also good networking opportunity.

    Habib Ur Rehman Mayar

    Deputy General Secretary
    G7+ Secretariat

  • I would like to thank you for a very informative and impactful conference. The networking opportunities, topics and knowledge shared were incredibly valuable. We look forward to next year’s event and would love the opportunity to participate more actively and bring some of the DBSA’s experiences and challenges to the discussion.

    Saphira Surina Patel

    Head of Sustainability, Professional Insights
    Operations Evaluation Unit
    Development Bank of South Africa

  • Those of us who were fortunate to experience the AEW have decided to initiate and encourage programs that will promote and formalize continuing evaluation of all things necessary to assist in our progress going forward in Federated State Micronesia. We would greatly appreciate any suggestions you or those responsible can provide to assist in our progression into this important area of evaluation.

    Ilhen Joseph

    State Auditor
    Office of the National Public Auditor
    Federated States of Micronesia

  • I really appreciated the chance to be involved in the conference as well. I have just begun to become involved in impact evaluation and this conference allowed me to see it in a whole new light.

    Codey Abbey

    Project Manager
    Shaanxi Normal University, People’s Republic of China uses cookies to improve your user experience. To learn more, click here to view our cookie policy. By clicking on OK or continuing to use the site, you agree that we can place these cookies.