10 excuses not to evaluate your program

Reasons for not evaluating the program

Announcing an evaluation requirement generates two kinds of responses:

enthusiastic, embracing the opportunity to recognise achievements, learn and improve, or
resistant, and full of excuses why you cannot and should not

Those in the latter group are challenging to move forward in their evaluation journey. Instead you are more likely to encounter a range of excuses, such as:

what we do is too complicated…
it’s too risky to speak to our stakeholders
eek …we don’t have any data to use!
I don’t know how
we don’t have any money for that
I already have too much on my plate
I am scared about negative findings
I already know there are problems and no one cares
we have been doing this for a long time … trust me
nothing will change as a result

People often resist evaluation for a range of reasons, most of them due to unfounded fear and perceptions, or because they are misinformed. The following unpicks some of the most common excuses and provides counter points you can fall back on to encourage them onwards.

Reason 1. What we do is too complicated…

Many programs function without clarity.

Evaluation can be confronting to those who find security in ambiguity. They might whinge about operating in the fog but if they are frightened about what an evaluation might uncover, lack of visibility seems more of a friend.

Counterpoint

Loose descriptions of activities and processes are an inadequate basis for determining how to define and recognise effective performance. Through the process of describing what is to be evaluated, it provides the opportunity to articulate clear outcomes within the context of complexity. Clear outcomes and explicit articulation of complexity can then provide a foundation for measuring success, improve the understanding of complex matters and enable you to demonstrate accountability to stakeholders.

You can truly understand whether all your hard work is actually making a difference!

Reason 2. It’s too risky to speak to our stakeholders

Many programs have vocal stakeholders with strong points of view. These could include everything from strong opposition to a program or conversely, calls for increases in program funding and everything in between.

A program manager’s worst nightmare is making the front-page news. It’s no wonder some program managers are reluctant to engage with their feisty stakeholders.

Counterpoint

Putting your head in the sand and ignoring difficult stakeholders doesn’t work! Avoiding engagement, at best, puts off the issues to a latter time and, at worst, means you are unprepared to respond to issues or critique of your program.

If you don’t understand your stakeholders’ points of view, it is impossible to resolve the issues or realise any benefits of stakeholder engagement. Failure to engage with stakeholders, rather than protecting your program, may inflame greater conflict and can contribute to a loss of credibility.

Rather than avoidance, welcome and respect their views and provide as much information as possible to let stakeholders know how their contributions will be used. While it may be uncomfortable initially, fostering a better relationship with stakeholders will pay off in the long run through improved engagement and understanding and ultimately better programs.

Reason 3. Eek …we don’t have any data to use!

This is an excuse we hear a lot. There is no data available, so there is no point trying to evaluate – we won’t get anywhere.

Counterpoint

While there are clearly advantages in being able to use existing information rather than undertaking new data collection, an evaluation can initiate and direct what data should be gathered to monitor and assess program performance against outcomes.

Rather than lament the lack of data, recognise the evaluation as an opportunity to recommend performance measures and a data collection and reporting plan. This data can also be used for public relations and promotion.

Reason 4. I don’t know how

Fear of the unknown is a big motivator. Those unfamiliar with program evaluation can be highly resistant to embarking on the journey.

Counterpoint

Fear is a natural human response and protective instinct. This excuse it likely to be unvoiced – so it is important to try and recognise it in yourself and others.

Educating yourself about program evaluation and the benefits is one of the best ways to overcome this excuse. It can be daunting to take this on, so go slowly and in small increments.

For example, rather than launching into a full scale evaluation, just develop a program logic as a first step and go from there.

Reason 5. We don’t have any money for that

This excuse can be a knee jerk reaction, or a reality. Either way it doesn’t mean that evaluation is off the cards.

Counterpoint

There are several avenues that can be explored here:

reassess the program budget to, look for underspends in other areas or for areas that can be reallocated to evaluation
seek out information about the budget in your broader area – there may be project funds available at this level
ask your corporate area about funding available for research and evaluation – can you get your project on the agenda?

Even if the answer to all of the above is ‘no’, there may be capacity internally to undertake an evaluation project. Capacity might exist within your team, broader area or corporate support.

Reason 6. I already have too much on my plate

Lack of capacity is a reality for many of us and can be a real barrier. Even the smallest evaluation takes time, effort and resources. Viewing evaluation as an easy add on, that can be easily accommodated within an already significant workload, is naïve.

Counterpoint

Like any project, an evaluation needs to be properly planned and resourced. This is particularly important if the exercise is to be conducted internally by the program team. If the evaluation is not given adequate resourcing and priority, it will languish when day to day program responsibilities get in the way.

If lack of capacity is a reality for you, the first thing to do is to reassess priorities. If evaluation is one of the top three on your list, look at how you can reorganise the workload to fit it in. This can be further helped by engaging external evaluators to take on the bulk of the workload.

Reason 7. I am scared about negative findings

“I really don’t want to know what’s not working – it will reflect badly on me and my team.” – Scared program manager

Counterpoint

Evaluation is not about catching anyone out or making you and your team look bad. Overcoming this excuse is a matter of reframing the purpose and outcomes of evaluation.

Evaluation is an opportunity to learn and improve – it’s not about placing blame.

Reason 8. I already know there are problems and no one cares

Cynicism and disinterest are tough ones to overcome. Cynicism typically builds over a period of time and can be tough to dislodge.

Counterpoint

Motivation and positivity are infectious and great weapons to combat cynicism. Cynicism can even be an asset if deployed effectively.

If you are working with a cynic, try the following:

Get them to list all of the barriers to action – and then plan on how to overcome each.
Call out the ‘elephant in the room’ and ask them to put it aside for the meeting/workshop to humour you.
Use your positivity and motivation to keep the energy up, rather than mirroring the cynics body language and style.

Reason 9. We have been doing this for a long time … trust me

This program manager thinks they have all the answers and that evaluation won’t offer any new insights or value.

Counterpoint

There is a lot of danger in this way of thinking. Those that think they know it all can be very closed to new ways of thinking or doing.

The best way to combat this attitude is to start by asking open questions to generate conversation.

By getting them to open up, you will be able to identify areas where you could add value through program evaluation.

Reason 10. Nothing will change as a result.

Reason 10 is really the only good reason to choose not to evaluate. If you know that circumstances won’t change post evaluation – don’t proceed. You’ll lose time, effort and most of all credibility.

If there is clearly no appetite for change among the team or leadership, do not proceed.

All the other excuses are smoke and mirrors – forge ahead.