As an evaluator, an absolutely key part of my role is to identify ways to determine and demonstrate the impact of the projects, programs and/or social ventures that I work on.
An impact of some sort, an influence or effect, is the objective of government-funded projects and programs and of social ventures including social enterprises.
But, impact upon what?
Well, that depends on a number of factors including:
a) What it was that prompted the design of the project, program or venture – a needs analysis, a new policy priority area, an idea or a problem observed firsthand, for example.
What are we trying to achieve?
b) The extent to which the desired effect of the project, program or venture was identified, specified and articulated at the outset
Have we identified what we are trying to achieve?
c) The extent to which those responsible for the design of the project, program or venture wanted to involve other stakeholders in the identification of what the desired effects should be
Is there anyone else that should be involved in defining what we are trying to achieve?
And finally …
d) The extent to which the desired effect or impact could be defined in the terminology available to those designing the project, program or venture
Are we equipped to define what we are trying to achieve?
I was fortunate to study research methods as part of my PhD, and so learned that the ‘social world’ is complex and there are multiple (possibly infinite) variables to consider. This is in direct contrast, it is argued, with a laboratory-based test-tube controlled experiment where there are only, say, two chemicals reacting without interference from anything else (as much as is possible).
The lab scientists can say/write: “I want to find out if x chemical will impact on y chemical in z way.” (Or something like that!)
But what is the x, y and z of the social world?
The social world is different
Lived experience has taught me that, yes, the social world absolutely is infinitely complex and made up of unique individuals and ever-changing and transforming communities.
The health, wellbeing and lived experiences of unique individuals and ever-changing communities is the ‘y’ of the social world. It is *usually* what we want to have a positive impact upon.
And yet, it has been challenging for me, an evaluator, to determine and identify the subtle ways (the ‘z’) that a project, program or social enterprise (the ‘x’) impacts upon the health, wellbeing and lived experience of other people’s lives (the ‘y’).
But explore, reflect on and attempt to define impact we must. Because, not only is an outline of our desired impact required by almost all funding providers, it also does actually help us move closer to the goal if we know what the goal is.
Where does this leave us as we continue venturing out into the so-called social world and attempting to have a (positive?) impact on people’s health, wellbeing and lived experience …?
How do we approach this complex and important task?
Rather than outline a list of rigid and tightly specific steps to take, I offer four qualities to bear in mind when approaching this task.
- Humility – An attitude of humility means you show up, take responsibility for your actions and inactions, use what you know and stop at what you don’t
- Realism – I am both a realist and an idealist, and I really believe that a realistic honest approach is the key to the application of ideological concepts often found in the social justice literature. Social factors and power dynamics rarely change over night.
- Reflection – We are all part of this social world and our own lived experiences are an incredibly rich and often overlooked source of insight on what’s working and what’s not, and whether anything positive or negative is changing as a result of our work. I encourage everyone working in the social justice, or related fields, to take the time to muse, reflect and record their reflections on what they’ve seen firsthand.
- Honesty – When we combine humility, realism (with a good dose of inspiration and an ideal to aim for) with honesty, we create and produce enormous amounts of useful new knowledge that can be shared with whomever we are humble enough to share it with.
If we truly mean to improve the health, wellbeing and lived experiences of real living people, there is nothing to fear in sharing our learnings and our failings because none of us will ever find the ‘perfect’ solution or transform the world over night.
You may be thinking, but how do we define, determine and demonstrate our impact? Its great to be asking these questions and in order to answer you would need to tell me about your project/program/venture. However, you can make an attempt at all of the above. You may find you’re not yet equipped to determine whether you made an impact or not. You may find that you’re busy determining whether you made an impact in one way, when someone you’ve reached comes and tells you their life has changed in x way.
There are many many resources available online, for example check out Evaluation Toolbox. Its pretty thorough.
Be ready to listen! Write that down, show you care and be responsive and flexible. This is all part of being humble, realistic, reflective and honest.
Good luck. What you do matters and you don’t have to be perfect.
If you are interested in starting a conversation with me about any aspect of evaluation in the complex social world, be in touch by email. I can offer support in a range of ways, suitable for a range of budgets. Check these options out here. I also offer one to one mentoring for people working in the social justice field.