CHAPTER 2: BASIC CONCEPTS ON MONITORING AND EVALUATION

Download the full document (pdf format )

M&E is a management tool for those who manage anything from a small project to a country program. Setting a good M&E system requires a careful thinking about overall program management and, particularly on how to manage the linkage between different program elements and partners at all level.

2.1 Definitions on M&E

The terms ‘monitoring’ and ‘evaluation’ are generally used together to refer to the whole process of assessing progress of a project or program towards its results.

Monitoring generally refers to the routine tracking of the project’s ongoing activities, achievements and constraints. It helps to ensure that activities are carried out as planned. It answers the question: What are we doing? E.g. People trained or counseled, condoms distributed.

Evaluation generally refers to the assessment of program/project implementation and its success in obtaining pre-determined project goals/objectives. It answers the questions: What have we achieved and how? What we have not achieved and why?

An indicator is a quantitative or qualitative factor or variable that provides a simple and reliable basis for assessing achievement, change or performance. It is a unit of information measured over time that can help show changes in a specific condition. Indicators of a project/program are identified from input to impact level. (The National level HIV/AIDS indicators are listed in Annex 1).

Characteristics of a good indicator

Specific: An indicator needs to be specific and related to the conditions the project/program seeks to change. It should be clear about what to change and where, when, and how the situation will be changed.

Measurable: Able to quantify the targets and benefits. Quantifiable indicators are preferred because they are precise, can be aggregated and allow further statistical analysis of the data. However, development process indicators may be difficult to quantify, and qualitative indicators should also be used.

Attainable: An indicator (or information) must be attainable at reasonable cost using an appropriate collection method.

Relevant: An indicator must be one that is necessary to measure; it must have relevance to the management information needs of the people who will use the data.

Time bound: The time period in which an indicator will be accomplished should be clearly stated.

2.2 Levels of M&E

There are different levels of M&E (see Figure 2):

Inputs are the financial, human and material resources that are necessary to produce the intended output of a project/program.

Activities/Processes generally refer to the different steps in the implementation of projects/programs. It refers primarily to the fact that the activities are actually happening or not.

Outputs are the immediate results of the activities conducted. Outputs are generally expressed for each activity separately. Examples: the number (or proportion) of people reached through behaviour change activities, the number of condoms distributed, the number (or proportion) of health care providers trained in STI case management, etc.

Outcomes are the medium term results of one or several activities. Outcomes are therefore, mostly expressed for a set of activities. They often require separate surveys to be measured. Examples: the proportion of target population that used a condom at last high risk sex, the proportion of retail outlets with condoms, the proportion of STI clients that received appropriate STI care, etc.

Impact refers to the highest level of results, to the long-term results expected of the project/program. Impact therefore, generally refers to the overall goal or goals of the project/program. Examples: decrease in the incidence or prevalence of HIV; better quality of life for PLWHA.

Figure 2: M&E levels

Note: The National level HIV/AIDS indicators are listed in Annex 1.

2.3 Elements of a Good M&E System

A good monitoring and evaluation system is the only way of establishing what is being done and if the interventions being undertaken are making a difference. HIV/AIDS programs in particular require establishing strong M&E system because the epidemic is relatively new and no one knows where it is headed and as such posing challenges different from many other issues in development. Continuous assessment is necessary given that new interventions are constantly being proposed. Efforts must be made to identify interventions that are more effective and to make them more central in the national response.

In order to establish a functional National M&E system five key elements are important.

i. Presence of an M&E unit: Established M&E unit with qualified staff and enough budget. The Unit also needs to build links with regions, sector ministries, research institutions, NGOs, FBOs, civil associations and donors.

ii. Clear goals and objectives of the program: It needs well-defined national program goals, objectives and targets where regular reviews/evaluations of the progress of the implementation of the National/Regional program is undertaken. Guidelines and guidance need to be put in place on the M&E to regions and sectors.

iii. A core set of indicators and targets: It is important to identify priority/core indicators and additional indicators that cover program inputs, activities/processes, outputs, outcomes and impact. Also, selection of indicators needs to be through full participation of stakeholders and maintaining relevance and comparability. The process needs also to utilize past and existing data collection efforts (e.g. DHS, BSS and Sentinel Surveillance) to assess national trends.

iv. A plan for data collection and analysis: An overall national level data collection and analysis plan is important. The plan also has to address data collection and analysis systems at lower levels.

v. A clear plan for data dissemination: Establishment of an overall national level data dissemination plan is important. This can include a well-disseminated informative annual report of the M&E Unit and annual meetings to discuss and disseminate M&E and research findings with policy-makers and planners. A clearinghouse/resource center at national level and gradually at lower levels (Regions and Woredas) is also necessary.

M&E is often perceived by project/program implementers in the field as an additional burden that has no clear goal or advantage. This requires providing regular feedback on M&E results to the implementers and also developing an M&E system that is simple to use and to understand, that does not introduce a work overload, and that has clear goal/objectives (Figure 5).

Two golden rules of M&E

Do not define indicators that cannot be measured

Do not collect data that are not useful for decision-making or from which no lessons can be learned