What matters, what's researched, what's measured

I've been thrown into the world of the 'validation of research instruments'. Or what you and I might call 'surveys and questionnaires that we are told can be trusted to do the job they say they will do'.

My whole self dislikes and distrusts any tool that says it's been scientifically validated, but is then used by humans on humans to find out about things that can't be observed - like wellbeing or health.

My visionary self has a dream of a different way of living together as humans, and my top priority is thinking how to get from the currently reality to something closer to this dream. Dreams usually take lifetimes or more, however hard and however strategically you fumble and work towards them.

My pragmatic self accepts that within the way things currently happen in the policy, practice and research world of health and social care, it is impossible to escape validated research instruments as they have power and credibility with those in power. Validated research tools are therefore key to improving things within the social and public service systems we currently have. It matters to people's lives now that things improve now within systems. We have to live in our present, not just dream for the future.

For validated research tools, therefore, the questions become ones of harm reduction, and opening up the debate about these tools.

I am very grateful to an academic* for linking me to a great 'primer' on the validation of research tools, as I've never really known what validation involves. For a published paper with so much information about such an obscure topic, I found it immensely readable https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6004510/
*She's nameless only because I'm writing this early on a Saturday morning and want to get this blog up before coproduction week finishes, ie before I can show it to her and check if she'd like to be named

Reading that paper made it clear that developing a validated research instrument is not a task to be undertaken lightly. It is time, brain and cost intensive.

But - some research funders for some types of research require the use of validated research instruments. So if there isn't a suitable validated research instrument, you can't get the funding, so can't do the research. Or to put it another way, you can only research something if there is already a research instrument that has been through the validation process. No instrument, no research.

It took a while for the significance of that to sink in for me. Some things that really matter to people's lives cannot be researched because the research cannot be funded because there is no suitable validated research instrument.

And then, add in another two layers of challenge.

1. I have yet to see a validated research instrument that can be used inclusively, across the diversity of people who experience whatever it is that is being researched. If used in line with the validation, this means research will perpetuate the exclusion of people who are already unheard, or heard but ignored when decisions are made that affect their lives. If used outside of the validation, for example by rephrasing and reframing questions or changing the response options in order to allow a more diverse group of people to take part, the tool is no longer validated.

2. Step 1 of developing a validated research instrument, according to Boateng and the rest of his research team (ie the people who wrote the paper that I found so helpful), is having a clear conceptual grasp of the topic. Say the topic is 'quality of life'. Whose conceptual grasp of the topic should provide the conceptual framework for the tool? I will always prioritise the conceptual framework of the potential participants whose lives will be affected by any decisions made on the basis of the research. And it is possible to do that. But it is also possible to use pre-existing academic conceptual frameworks. Yep, you can design a new tool based on the frameworks that are based on research that is shaped by what could be done using a previously validated research instrument. I suspect that route is more common, as it is much cheaper and easier to do a literature review than brand new qualitative research with people recruited from your pool of potential future participants.

So what?

So those of us who would prefer to avoid validated research instruments need to engage with the creation of validated research instruments if we are to avoid the perpetuation of exclusion and silencing of some groups of people from the kind of research that policy makers and governments listen to.

And it would be doable. But it is a big investment of effort. So it would be important to make sure that the effort is being invested in developing a tool that would allow research to be carried out on a topic of crucial or strategic importance to those who are usually excluded or silenced.

And that means starting with the research on 'what matters', so we know what is worth that investment.

And that is where coproduction comes in.

Doing research to identify 'what matters' cannot be done just by academic researchers, or just by people with lived experience, or just by think tanks, or just by government research teams. Doing this research needs the skills of academics, the skills of people with lived experience who are already involved in policy or practice work, the skills of policy makers, the skills of people responsible for providing public services, the skills of members of the public. What everyone in that list has in common is possessing a set of expertise, experience and perspective that others on that list do not have, but that are needed for top quality health and social care research. In other words, to do top quality health and social care research means developing our skills as research coproducers. And by 'our skills', I mean the skills of everyone in that list.

I would argue that everyone involved in coproducing research needs to learn the skills that are needed to be research coproducers. It does not come naturally to any of us. For those of us with professional training, it won't have been part of that training so we need training. For those of us without professional training, we need training too.

And I would argue that we need the same training, a training that helps us work out for ourselves how to apply the principles of coproduction to the professional training or lived experience we bring. And a training that helps us see other people's skill sets and ways that the skill sets work together to make us and our research stronger.

Where do we go from here?

I don't know!

I'm putting this out in Coproduction Week, because it is probably the best week of the year to wave a flag saying 'over here!' about something that I've not seen discussed. PLEASE come and find me if you are already grappling with this. I won't be doing anything practical about any of this until I've submitted my doctoral thesis. But this is something that would benefit from a long brew on the back burner. Here's to Coproduction Week 2021 when I will hold myself accountable for what I have (or haven't) done to start developing these rough ideas into actual life changing research changing work.

Lessons from campanology for coproduction

 Campanology is the proper name for the kind of bellringing you do in a bell tower, where each ringer is ringing one bell by pulling alternately on the furry bit or the tail end a rope (while remembering to let go of the furry bit but never letting go of the tail end...).

It struck me that guidance for coproducing a beautiful sound in campanology is very similar to guidance for coproducing a society where public services and people using them share and achieve an aim. 

Here are some rules for campanology. See if they ring true [sorry, couldn't resist the pun] for you in your coproduction.

1.   Know the method* so you have an overall picture of how everyone will move.

2.   Know your place and how you will be weaving between the other bells.

3.   Watch the ropes and listen to the spacing between bells.

4.   When things go wrong, hold your own place in the ‘dance’ so others can orientate themselves back into the right place, but also be willing when there are small wobbles to accommodate those before and behind by shifting fractionally from where you should be - it makes the everything sound more harmonious even though you yourself are now not exactly where you 'should' be. 

5.   If all goes horrible wrong, call 'rounds**' and once everyone is back into the basic rhythm and 'holding pattern' of rounds, and has worked out what went wrong, try again.

6.   It's not about blaming the one who got lost. It's about working out how to all get it right together next time.

7.   The neighbours don't listen to individual bells to see which one is getting it right. They listen to the music. 

8.   Towers*** stand or fall by their tower captain. But all towers need a mix of skills and personalities to be strong, fun places to spend time and learn. 

9.   You've never arrived as a bell ringer. There's always more to learn. But from quite early in your ringing career, you can be part of making simple beautiful music.

* method – a pre-agreed pattern for how the ringers will change the order in which the bells are rung

** rounds – when the bells are rung from highest note down to lowest note in order

***tower – the group of bellringers who belong to a particular bell tower

Being and doing

And breathe!

I know I’ve pushed for too long when my skin flares up, my brain starts to fragment and my breathing is consistently shallow.

It feels as if I’m held together by tension, and if I relax, I may just disintegrate.

So it is definitely time to breathe. And relax my jaw. And experience the moment. And drop by shoulders.

Look at me! Even my relaxation turns into a list of and, and, and. Pressure upon pressure upon pressure. Rush, rush, rush.

Recently I have been spending a lot of time ‘in role’ and not much time ‘being me’. Being ‘in role’ is about performing, delivering, doing. ‘Being me’ is about finding that still point where I just am, and the response you get from me is the same whoever you are and whatever the context.

I have an ambivalent relationship with roles and being. I think roles make me more efficient. They help me focus, achieve and deliver. I like being ‘in role’ because I like achieving. Being in role also lets me do things that I struggle to face doing when I am ‘being me’. It’s the same for my daughter. When she has to do something that feels impossible for her, she hunts for a persona she is comfortable to put on that sees tasks like that as pleasurably simple.  I achieve ‘in role’ by using props like how I dress, sensory inputs (a smell, a taste, a texture) and how I hold my body.  She achieves ‘in role’ by imagining herself that person.

Being ‘in role’ has its dangers. Stay in roles too long and I forget who I am. It takes me an age to slow and deepen my breathing and find that spot where I just am. Until I have found that spot, I don’t process emotions.

Weeks like the last few weeks have required me to play several different roles, often within the same day. I’ve had very little time to switch between roles.  This exacerbates the dangers of being ‘in role’. When I am in a role, I don’t make connections between that role and what happens when I am in different roles. So it’s hardly surprising that I start to feel disconnected from myself and overwhelmed. When I do stop there are so many competing demands on my time, emotions and brain. There is just so much stuff to integrate and connect within me.

As I said, I play roles because that is how I am efficient, and how I manage to do things that I don’t feel capable of doing as me. But, perhaps, I am prioritising efficiency at the expense of being effective. I am less likely to contribute anything uniquely me when I am ‘in role’. I am more likely to adopt the processes and thinking that goes with that role, so come up with an efficient solution or output. My most effective work comes from me seeing situations differently, and therefore responding differently. And that only happens when I drop the role-playing and I am ‘being me’.

In some situations, I now ask whether we are going to interact in our roles, or as our whole selves. That’s my equivalent of asking people whether they want to leave their job title at the door. In my mind, this is also equivalent to asking people whether they want to collaborate (efficient work done ‘in role’) or coproduce (effective work that flows from ‘being who we are’).

Of course, for me, there’s the whole added dimension of living with a committee in my head. Being ‘in role’ is different from handing control to different internal people. But when I/we spend too much time in role, we lose our close internal cooperation and start to fragment. But that’s for another blog.

For now - just breathe.