Resources: What tools will help me?

There are many tools available to help you collect evidence, including those designed specifically for voluntary and community sector groups. You might decide to use a ready-made, standardised tool. Some of these are free, while others are commercially available. This kind of tool can be useful if it is important to you to be consistent in the way you measure things. You also have the benefit of knowing that it has been tested and found to be reliable.

Alternatively, you could develop your own system – which could be one that is used across your organisation as a whole, or something specific to an individual project. This might be appropriate if your project is small, and/or if you want to measure something that is unique to your project.

What you choose should depend on what it is you want to measure, as well as the resources you have available. For example, you might want to do an overall assessment of your organisation’s impact, using a tool that takes many factors into account. Or you might simply want to find out whether people who visit your project have experienced improvements in one area of their health or wellbeing.

We have collected information under the following headings:

A1.1 General tools

Inspiring Impact

Inspiring Impact is a project run for and by the voluntary and community sector, to encourage groups to do more to measure their impact. The website provides information and guidance on how to do this. The Resource Library has a searchable database of hundreds of tools suitable for different types of evaluation, from questions on a single topic (such as self-esteem or anxiety, housing or domestic violence) to handbooks for conducting a community impact assessment. Each tool has a page describing its key features and cost, if any, and giving web links.

Inspiring Impact have produced Measuring Up!, step-by-step self-assessment tool that allows organisations to review and improve their impact practice – that is, the way they plan, evidence, communicate and learn from the difference that their work makes.

The tool has been designed for use by all kinds of charitable organisations, social enterprises and funders. There are now three versions available:

  • Measuring Up! for small organisations (those with an income of £0-£100,000 or newer to impact practice)
  • Measuring Up! for medium to large voluntary organisations (those with an income of £100,000+ or more experienced in impact practice)
  • Measuring Up! for funders (all types of voluntary sector funders)

A preview of the tool and guidance are available free of charge.

Writing case studies

A free guide and template, How to Write a Case Study in Public Health, is available to download from the UK Health Forum.

Tools available to buy

The Outcomes Star and the Rickter Scale are both commonly used within the voluntary sector. These are designed to capture ‘distance travelled’ and outcomes that are traditionally hard to measure, such as increased confidence, which can be important precursors to other outcomes, such as being ready for employment. Clients rate on a scale how they feel they are progressing against certain outcomes. In conversation with a staff member, the clients think about where on the scale they would like to be and how they can get there. Both tools are available in a number of variations, depending on the outcomes being measured and the client groups and services involved. Training for staff is included in the purchase price.

Questions for measuring general health and wellbeing

A recent review has compared tools for measuring general health, wellbeing or quality of life in community-based projects. Each tool is a set of questions to which participants give an answer on a scale, which allows you to calculate an overall score. They are designed to be used before someone’s involvement in the project and again after some time, so that you can see if the score has improved. The review rated each tool for reliability, length, clarity, cost and suitability for use in cross-cultural settings. Three of the tools rated ‘excellent’ are available free online:

Quality of Life Scale (QOLS)

Covers material and physical wellbeing, relationships with other people, social, community and civic activities, personal development and fulfilment, recreation and independence.

Personal Wellbeing Index (PWI)

Covers standard of living, personal health, achieving in life, personal relationships, personal safety, community-connectedness and future security.

WHO Quality of Life – Brief (WHOQOL-BREF)

Covers physical health, psychological health, social relationships and environment. Email to request the questionnaire.

Existing survey questions: UK Data Service Question Bank

This is a resource that allows you to see what questions have previously been asked in social surveys. You can search on keywords: for example, if you search for questions containing the words “happy” and “life” you will see that a question that is frequently asked is:
“If you were to consider your life in general these days, how happy or unhappy would you say you are, on the whole?”

You do not need to register if you want to search only for questions.

A1.2 Measuring wellbeing

Improved wellbeing (how people feel and function) is a key indicator of success for many community-based projects. The New Economics Foundation has produced a free handbook for voluntary organisations and community groups on how to measure wellbeing. It includes sections on what wellbeing is, how to design a questionnaire and how to analyse the data.

The tools that it recommends are a combination of SWEMWBS (see below), the questions on subjective wellbeing used by the Office for National Statistics (also below), and a question that asks whether or not most people can generally be trusted.

The What Works Wellbeing Centre has produced an online guide, Measure your wellbeing impact: A practical guide for charities and social enterprises. It has a clear, step-by-step process to help you think about what you want to measure and how to do it, including example questions you can ask, and guidance on writing your own questions.

Further information: https://measure.whatworkswellbeing.org/

WEMWBS

A popular tool for measuring and monitoring wellbeing is the Warwick-Edinburgh Mental Wellbeing Scale (WEMWBS). This is a 14-question scale which results in a single score. There is also a shorter version with 7 questions (SWEMWBS). The scales have been tested and are considered reliable. Both versions are free, but you need to register and seek permission to be able to use them. They are copyrighted to NHS Health Scotland and the Universities of Warwick and Edinburgh.

Further information and user guides: www.healthscotland.com/scotlands-health/population/Measuring-positive-mental-health.aspx

Registration: https://warwick.ac.uk/fac/sci/med/research/platform/wemwbs/using/register/

Office for National Statistics (ONS) wellbeing questions

The ONS keeps lists of standard questions used in government surveys:
www.ons.gov.uk/ons/guide-method/harmonisation/secondary-set-of-harmonised-concepts-and-questions/index.html

The ONS has four survey questions to measure personal well-being. These questions have been tested with many people and are known to be reliable. People are asked to respond to the questions on a scale from 0 to 10 where 0 is ‘not at all’ and 10 is ‘completely’. The questions are:

  • Overall, how satisfied are you with your life nowadays?
  • Overall, to what extent do you feel the things you do in your life are worthwhile?
  • Overall, how happy did you feel yesterday?
  • Overall, how anxious did you feel yesterday?

The World Health Organisation (Five) Well-Being Index (WHO-5)

A set of five statements measuring subjective wellbeing, which has been shown to be a reliable measure of emotional functioning and a good screener for depression. People are asked to rate how often they have had these feelings over the last two weeks, on a scale from 5, ‘all of the time’ to 0, ‘at no time’:

  • I have felt cheerful and in good spirits
  • I have felt calm and relaxed
  • I have felt active and vigorous
  • I woke up feeling fresh and rested
  • My daily life has been filled with things that interest me

Further information: https://www.psykiatri-regionh.dk/who-5/Pages/default.aspx

The Cantril Self-Anchoring Striving Scale

This is a simple way of measuring wellbeing that has been used in large international polls. Also known as ‘Cantril’s Ladder’, it asks people:

  • Please imagine a ladder with steps numbered from zero at the bottom to 10 at the top.
  • The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you.
  • On which step of the ladder would you say you personally feel you stand at this time? (ladder-present)
  • On which step do you think you will stand about five years from now? (ladder-future)

Further information: www.gallup.com/poll/122453/understanding-gallup-uses-cantril-scale.aspx

The CORE Outcome Measure

A tool for measuring psychological distress, which includes measures of wellbeing. There are different versions of the tool, including versions for young people and people with learning disabilities. The forms are free to download, but there is software and training to buy for use alongside the measures.

Further information: www.coreims.co.uk

A1.3 Measuring social isolation and loneliness

If you want to measure whether your work has an impact on how lonely or isolated people feel, there are a number of simple, free tools that you can use. What is considered ‘normal’ in terms of community and loneliness varies between different societies, so when you are choosing a measurement tool, consider how appropriate it is for the community that you work with.

The Campaign to End Loneliness has an excellent guide on measuring loneliness among people in your community. It describes four different tools for measuring loneliness, and the features of each. The guide is focused on people in later life, but some of the tools are also suitable for general use. The tools discussed are: the Campaign’s own measurement tool; the De Jong Gierveld Loneliness Scale; the UCLA Loneliness Scale; and single-item scales (i.e. consisting of just one question).

Questions for measuring social isolation

Some other tools for measuring social isolation that have been tested and are freely available, are:

Lubben Social Network Scales (LSNS)

https://www.brandeis.edu/roybal/docs/LSNS_website_PDF.pdf

Medical Outcomes Study Social Support Survey (MOS-SSS)
www.rand.org/health/surveys_tools/mos/mos_socialsupport.html

Multidimensional Scale of Perceived Social Support (MSPSS)
gzimet.wix.com/mspss

The Friendship Scale
afterdeployment.dcoe.mil/sites/default/files/pdfs/assessment-tools/friendship-assessment.pdf

A1.4 Measuring self-efficacy: ability to manage health conditions

If your project is about improving people’s ability to manage their health condition, you could use a self-efficacy tool to measure your impact. You could ask the questions before and again after your project. These tools can be particularly helpful where you are aiming to build capability within communities to address – or give advice on – particular health issues.

You can also construct your own set of questions to measure self-efficacy. The following guide shows you how to do this.

Questions for measuring self-efficacy

The General Self-Efficacy Scale (GSE) is available in 31 languages and is free.
userpage.fu-berlin.de/health/selfscal.htm

A1.5 A faith-specific tool

The Christian Faith Practices Scale is a tool that has been developed among Christian churches in the Protestant tradition, although it could be adapted to suit other faiths.

It asks people how often they participate in certain practices as part of their faith, and so could be useful for measuring how many people in your group are involved in social action, and how often. It includes questions about providing hospitality and care, volunteering time to help others and participating in activities that promote social justice, alongside questions about devotional practices.

You can find the questions here (go to the very end of the article):
www.baylor.edu/content/services/document.php/145852.pdf

A1.6 Return on investment

One way of showing your impact is to demonstrate the value that you create in return for the cost of your activities. You can show the value in terms of money, or in terms of the changes in your stakeholders’ lives (this is called social value – see below). Calculating return on investment can be complex and is not for everyone, but it can be a compelling way of persuading funders that it is worth investing in your work.

FaithAction’s Local Impact Assessment tool

FaithAction has developed this tool to help FBOs consider the resources and value they have in their communities, in different spheres (e.g. the buildings they have, volunteering hours worked, money brought into the community through their work). It is a starting point for calculating what an organisation brings to the community.
www.faithaction.net/resources/

Cost-benefit analysis

Cost-benefit analysis compares the benefits that an activity brings with the costs of carrying it out. It allows you to be able to say, for example, that for every £1 invested (the cost), you might produce £2 in return (the benefits). In order to do a cost-benefit analysis, you need to be able to calculate your unit cost (which is the cost to your organisation of working with one beneficiary), and to estimate the monetary value of the benefits of your project.

NPC provides further information, including this short article:
www.thinknpc.org/blog/getting-started-with-cost-benefit-analysis.

The Big Lottery Fund has a step-by-step guide to carrying out cost-benefit analysis:

Calculating costs saved

If you run a service that is normally funded through the health system, you can potentially save the health system a large amount of money. Finding out the usual cost of running the service, and pointing out how much your organisation can save the system by running it instead, is a powerful argument when bidding for funding. It’s also simpler than conducting a cost-benefit analysis – and helps highlight the significance of faith.

The average unit costs to commissioners for providing many different types of health and social care services have already been worked out. This information is available in the following directory – use the contents section to find what is relevant to you.

A1.7 Measuring social value

Social value

Where value is experienced by your stakeholders as changes in their lives – as is the case for much voluntary sector work – it is called social value.

Social value is often understood as the additional social, environmental and economic benefits that communities can gain from the way a service is delivered, over and above the service itself. For example, if an organisation employs young people who were previously unemployed to deliver the service, there will be additional benefits to the community in terms of jobs and skills.

Social return on investment

The best way to understand social return on investment, or SROI, is as an extension of cost-benefit analysis, which includes wider social and economic outcomes. It can be used both for evaluating what has already been done, and for estimating or forecasting potential value.

An ‘SROI ratio’ can be calculated by giving monetary values to as many outcomes as possible (using techniques such as those above, and/or calculating public money saved). These values are then added together and divided by the amount of the resources used or the initial investment.

Social Value UK has more information and an easy-to-understand introduction to social return on investment (SROI). This goes through the five questions that are key to SROI:

  • Who changes?
  • How do they change?
  • How do you know?
  • How much of the change is down to you?
  • How important are the changes?

More information and details of resources can be found in the boxes which follow.

Social Accounting and Audit

The Social Audit Network (SAN) recognises that there has been a dramatic increase in social impact reporting in recent years. It sees this as welcome, but points to some issues such as how much credibility should be attributed to these reports, the expense of employing independent evaluators, and the necessity of routinely collecting impact data in a way that is not too time-consuming. Through its experience of working with grassroots organisations and believing that they can be empowered by keeping track of their own monitoring and evaluation, SAN has developed a process of social audit and accounting.

In the same way that organisations keep financial accounts, it is also possible to keep social accounts, using a social book-keeping system with output and outcome information. Social accounts undergo an independent annual ‘social audit’ to verify them, just like financial accounts. Social accounts are regularly presented alongside the financial accounts, providing a more holistic picture of an organisation’s performance and impact. They enable an organisation to be confident of its claims and accountable to a wide range of its stakeholders, and reassure the wider public of the authenticity of social impact reporting. At the same time, they can also be used to plan and focus future strategic actions. They are especially useful for organisations that run multiple projects at once, as FBOs often do.

SAN believes that the social audit part of social accounting is essential, particularly for organisations with a central purpose around social transformation and change, which want their social impact to be taken seriously. Without social audit, it is easy to be overwhelmed by detailed reports purporting to explain the social, environmental and cultural change that has happened as a result of an organisation’s activities. Recent legislation in the form of the Public Services (Social Value) Act 2012 (see above) means that this verification process is an even more important factor.

SAN has developed a framework, summarised below. It is not limited to one evaluation methodology and can accommodate data gathered from many appropriate social impact tools.

More information and details of the framework and the SAN regional coordinators near you are available at www.socialauditnetwork.org

FAITHACTION