Monitoring and Evaluation

Monitoring and evaluation of programmes; This is an essential part of improving work. Related sections include working with others and setting standards. Key points about monitoring and evaluation are:

1. Both monitoring and evaluation are processes used to assess project progress.
2. 'Monitoring' refers to an ongoing system which routinely collects data and is used to keep the project 'on track'.
3. 'Evaluation' refers to a one-off event conducted to account for resources used and/or to document lessons learned.
4. There are two main approaches to evaluations. Scientific approaches emphasise the importance of objective facts/evidence. Interpretive approaches emphasis the views and perspectives of people affected by/involved with the project. These people are termed 'stakeholders'.
5. Many approaches to evaluation now emphasise the active participation of stakeholders, particularly children and young people. Different stakeholders may have very different levels of power within a project.
6. Indicators are things which are used to measure or assess progress made by a project. They may be expressed as numbers (quantitative) or descriptive words (qualitative). They may be internationally or locally-defined and can be used to measure project activities at different levels, for example processes/activities and outcomes/impacts
7. Good monitoring systems and evaluation approaches collect and compare information from a variety of different sources using different methods. This is termed triangulation.

Defining Monitoring and Evaluation

1. Scope : Some authors use the term monitoring for the tracking of information related to the national HIV/AIDS response and the term evaluation for the gathering of information related to a particular programme or project.
2. Nature : The term monitoring is used to describe a routine system of collecting and analyzing information about the work of the project. On the other hand, the term evaluation is usually used to describe a specific event.
3. Timing : Monitoring is a regular and ongoing process which takes place throughout the life of a project. An evaluation usually occurs at a particular time, for example midway through a project or at the end.
4. Purpose: The purpose of monitoring is relatively specific, in that it usually focuses on keeping a project 'on track'. This involves routinely measuring what the project has done and comparing this with its plans. Evaluation usually has broader purposes, for example, assessing what has been learned as a result of project activities.

Project Monitoring and Evaluation Compared to Other Types of Monitoring and Evaluation

Purpose of Evaluation

There is a tension between these two purposes. For example, this may explain the reason why 'negative' findings, that is what didn't work, are rarely recorded in evaluation reports. Although such findings are very useful for the purpose of learning, they are problematic in terms of accountability, because donors are unlikely to be willing to fund activities shown to be ineffective. Consequently, organizations may not wish to publicise findings of this nature.

Ways of Conducting Evaluations

Stakeholders

1. Power : There may be power imbalances between different groups of stakeholders. For example, donor views may be given greater weight within an evaluation because they control project funds.
2. Participation: Interpretive approaches to evaluation place great emphasis on the active participation of project stakeholders, particularly children and young people, in an evaluation. A number of questions have been identified relating to the participation of children and young people in activities and their evaluation. These are:
   • Have children and young people, and their families, been involved in deciding the measure for the evaluation?
   • Can children and young people benefit from the evaluation, and how?
   • What have been the risks and costs for children of their involvement?
   • Have confidentiality and privacy been respected at all times?
   • Do the children involved know that they are free to refuse, or withdraw at any stage, and that this will not be held against them?
   • If certain children have been excluded from participating, can their exclusion be justified?
   • Have the children concerned and/or their carers, helped to plan, implement, analyse and evaluate the activity
   • Are the children concerned aware of the purpose and nature of their participation, methods, timings, benefits, consequences and outcomes?
   • What have the project workers learnt from the participation of children and young people?
   • Will the children, young people and their families be told of the main findings?
   • Apart from the effects of the activity on participants, how might the conclusions affect other children and young people?
   • What are the constraints on the participation of girls, disabled children, orphans or children living with HIV/AIDS? How can these constraints be overcome?

Participation of children and young people in evaluations may lead to issues being identified which might otherwise be overlooked. For example, in the evaluation of the Families Orphans and Children Under Stress (FOCUS) programme in Uganda, children and young people identified stigma and sexual abuse as major issues.

If people are to participate in monitoring and evaluation processes, they may require training. Experience from the Community-Based Options for Protection and Empowerment (COPE) project in Kampala concluded that many community committee members did not understand the data being collected as they had not received adequate training. The same review also concluded that data collection processes were more likely to be successful if government employees were involved. Indicators Indicators are used in both monitoring and evaluation. Essentially, they are things which can be measured or assessed to see the progress being made by a project. They may be expressed in numbers (quantitative) or through descriptive words (qualitative). They may form part of an international set of core indicators or may be developed locally for a specific project. For example, a number of core indicators have been agreed for use at national level. Indicators may measure different 'levels' of a project. These levels include:

• Inputs - these are the things needed for the project to occur. They are like the raw ingredients needed to cook a meal. Examples include people to do the work and money.
• Processes - these are the activities of a project. In general, these are relatively easy to measure and process indicators often form the bulk of monitoring systems. An example of a process would be training sessions held for staff. Various indicators may be used to measure these processes such as number of training sessions held and number of people trained.
• Outputs - these are things produced by the project. Examples include new or improved services, trained staff, information materials etc.
• Outcomes - these are the positive short-term effects of a programme. Examples include improved child nutritional status and improved school enrolment.
• Impact - this is the longer-term change which arises as a result of several positive outcomes. Common measures of impact of the response to HIV are reduction in new HIV infections and increased survival time of people living with HIV. However, different authors use these terms in slightly different ways. For example, some people regard the number of people trained as an input while others see it as an output. This can be confusing. As a result, some people have argued for a simpler system which measures the services a project delivers and the effect that it has. Examples of these indicators are presented in the table below.