The Old Metrics Don’t Work in The Online World
Metrics play a significant role in my consulting work.
Over 8 years of on-site consulting I’ve uncovered fresh insights into a church’s health through hard data such as metrics.
I know what to measure.
Analysing baptisms, generational ratios of attendance, serving and small group percentages and more gave me an introduction to a church’s wellbeing.
The normative metrics we’ve used to assess church health count for nothing when assessing online services.
We’ve shifted to views, whatever they are, and engagement numbers which demands a whole new way of looking at our church numbers.
I realise that church health is never determined solely by the numbers but how can we begin to gauge our church ‘s health without some measuring tools?
I soon realised we would need to lean on qualitative measures more than quantitative numbers. Soft data if you like, as against the hard data of metrics.
As you move back to on-site services and grow your church during reentry you will want to take some these qualitative elements to help you assess your church’s health.
I asked good friend and experienced researcher, Dr Mandie Shean, to leverage her expertise in qualitative research to help us wrestle with this problem.
Here are her thoughts, first in her art (coloured by her mum, Sue) and then in her words. This will help you grow a healthy church.
Measuring Success in an Online Enviornment
Success on a Sunday morning may be understood by looking at attendance, the ratios of generations in attendance, the offering, or hearing feedback. These are intuitive measures that give us immediate feedback.
In an online environment it is not the same.
When you run your church online, you might get ten comments and 70 people log in (from your 1000-member church), and it can be hard to judge the success. It can also seem like a bit of a fail compared to what you normally “see” on a Sunday morning (full churches, smiling faces, and bustling cafes).
Rethink What You Are Measuring
You need to let go of your normal measures.
You can’t expect what you normally “see” to be evident online. People can be quietly loving your message, but you will not know.
You may have 70 people log on live to the service but there are five people in each family (you won’t know this). You can also have small numbers log on but hundreds watch it later. Maybe 20 people watched it twice and it changed their life. This is the difference – be open to this.
Values and Vision
All measures should come out of your vison and purpose statements.
While an online environment is different, your vision and purpose hasn’t changed.
Let’s say your values are: knowing God, sharing your faith, and doing good for others. Sit down with your team and get your vision out – every measure should relate back to these values.
Ways to Measure Your Success
There are two ways of measuring success – through qualitative or quantitative data.
Quantitative data shows a general picture of how you are going.
For example, a quantitative survey will show me that 20% of people don’t like the worship or 40% like the coffee after the service.
Qualitative data on the other hand explains why.
People like the coffee because it is rich and at the right temperature. It uses words that helps us to understand the phenomena. Both can be used during this online experience.
Quantitative Data
Microsoft has the option of Forms to collect quantitative data, as does Google. It is simple to set up, you can make it anonymous, and you can get a quick picture of how your church is tracking. Some of the questions you may want to include will be around:
- Are you tuning in?
- Are you enjoying it?
- Are you gaining something from the services?
This is useful information to know so you can tweak your delivery, however it doesn’t really explain what we need to change. That is qualitative.
Qualitative Data
Qualitative data gives people a voice.
At times, researchers produce surveys with only their own ideas.
If you only use surveys with your own questions you will never know anything beyond what you already know.
As an example, I can ask you if you like Vegemite or jam and you have to choose. Then I tell everyone that 80% of people like jam. But it’s not true. You just had no other options.
If I asked open questions, I would have found that most people don’t like either. By limiting the options, I am only checking what I believe to be true. You need qualitative open research, so you find out what they believe to be true.
The other problem with quantitative data is it doesn’t explain why, who, when, or how long.
Going back to the previous example of 20% of people not liking the worship – who are these people? (those in the younger or older age group), how long have they felt like this? (recent or years), and why don’t they like the worship experience? (this is something you can actually address).
When you identify these ideas from your congregation (rather than trying to guess them with your team), you can accurately address them. Your people are your most important source of information, use them.
As a side note, during these COVID19 times, a genuine phone call can add a bit of humanity to the process.
Asking the Right Questions
Ensure your questions link to outcomes or vision. For example, you may ask the seemingly harmless question of “did you enjoy the worship?” Enjoyment is nice but is it your vision or purpose? It is not far off asking “do you like the pastor’s shoes?”
A better question that aligns with your purpose is “Do you engage with the worship?” and then “Explain what makes it easier to engage or disengage with worship”. These ideas give you something to work with and explain the numbers.
FREE INFO SHEET
12 METRICS TO measure church HEALTH
Download your Info Sheet and evaluate your church's health
Untangling Qualitative Data
People get stuck pulling qualitative data together.
You just need to read through it and listen to the ideas that ebb through.
When you collate words together you will find higher order concepts: love, belonging, faith building, learning, support.
Here are three examples for you:
1. Disrespectful, rude, didn’t listen, didn’t feel heard
These words suggest your current communication is not working
2. Welcome, at home, belonging, I feel seen
These words suggest that you have a positive sense of community in the online environment.
3. Alone, don’t know names, no one talks to me, hard to connect
When you look at these as a group, your church lacks community.
Being Open to Feedback
In my opinion, I would rather be criticised than continue doing something that is not working for my organisation.
Yesterday, I asked someone a few questions.
I asked: Do you feel supported by me?
How?
What could I do less of and more of?
Anything I could do differently?
It was a little humbling because I could have gotten negative feedback, however these questions actually check if I am doing what I am intending to do.
Sometimes it is nice not to know (I can feel good about myself), but if you really want to make a difference you need to be open to the truth – whether it is qualitative or quantitative.
Try This Simple Survey for Your Online Service
Do you engage with the worship?
What has helped you most in the Sunday Online Services?
In what way has the service helped you stay connected to our church?
In what way does the service build your faith?
How does the online service help you to live out our mission?
FREE INFO SHEET
12 METRICS TO measure church HEALTH
Download your Info Sheet and evaluate your church's health
