Equity in research and evaluation design and execution

Louisa Thomson, Head of Consultancy shares how the team has worked to make our evaluation and research practices more equitable and inclusive.

Like many in the research and evaluation industry, we know that power and privilege shapes and can dictate our work – from who commissions a piece of research, who is invited to participate and what happens with the findings.

As a social enterprise, we pride ourselves on working sensitively with people facing multiple forms of disadvantage, making sure they feel heard when participating in research. Yet as a team we know we can do more to fully consider the impact of all forms of discrimination in our work.

In the thick of busy periods, balancing deadlines, and struggling to get fieldwork in place, do we occasionally take the easier route to get something done? Have we ever relied on clients to help us recruit research participants? Or reverted to a standard interview format for speed? If we’re honest with ourselves, we know that sometimes our processes have excluded rather than included.

Making our research more inclusive

Earlier this year, as we were considering how to be more intentionally and consistently inclusive, we started work on an ethnography research project for Impact on Urban Health. It gave us the opportunity to think more deeply about what inclusivity means in our research design and execution. The aims of the research were to explore the experiences of families who have children with behavioural difficulties.

It’s a hugely sensitive area, laden with stigma and a mistrust of people and services. We felt strongly that families were placing a large amount of trust in us as researchers by allowing us a window into their lives. We wanted to avoid any scenario where families might feel coerced to take part, that they were not being listened to or were misrepresented, or to cause distress through talking about their children and circumstances.

These are ethical issues at the foundation of any good qualitative research.

So, what was different about this research project?

Investing time in the planning stage

The research started in the middle of the third national lockdown, and while this was initially a setback as it prevented us from being able to visit families and get to know them in their own contexts (which is the absolute privilege that comes with ethnography), it did give us more time to plan.

The research questions were already agreed but we challenged ourselves not to replicate approaches from previous projects and to start with a blank page. We held a series of research team workshops to co-design the approach, conceiving every ethical challenge, safeguarding issue and potential barrier to participating so we could mitigate them as far as possible.

Consent as an ongoing process

We produced information sheets and consent forms in age-appropriate formats, designed so as not to overwhelm. We went through several stages to make sure families were comfortable with participating, understood the aims of the research and what would be involved.

There was an initial screening (by the recruiter), followed by an informal conversation with a researcher to start to get to know the families.  We drew up a research agreement with each family and then sought their consent to proceed.

Before each research engagement, the researcher checked with the family whether they wanted to continue and reminded them of their rights. 

In the majority of families, we also spoke to the children. Parents gave us advice on how best to approach this.  Then we also made sure that each child consented independently regardless of their age.

Accessibility and transparency

The families were struggling on a daily basis with managing challenging situations with their children, so researchers talked through every aspect of participation (preferred times of day, the length of any research activity, etc) to give them choices that worked for their lifestyle.

We offered translated materials and interpreters for speakers of other languages, and to cover costs such as childcare and devices if it would help families participate.

There is a risk that incentive payments may encourage people to participate solely for financial reasons rather than to support the research aims. However, we knew the stakes were high in terms of both time and emotional cost and we wanted to recognise families’ commitment. For full transparency, we costed each research activity, so everyone knew what to expect.

Choice about how to participate

Giving research participants choice is easy for us to write on paper but it can get lost in translation as research projects proceed.

The questions we asked were consistent, but the ways they were introduced to families varied considerably.  The research process involved interviews, diary tasks, and interactive exercises that were staged over three to four weeks (and in some cases longer). Every researcher had access to a variety of tools, templates, visual prompts, and interview guides and chose what to use and how depending on the family’s needs and preferences.

For example, some families chose to take part via Zoom to give us virtual tours of their home while others preferred to participate by phone. When COVID restrictions eased, researchers met with some families in parks and cafes. We posted hard copies of some of the interactive exercises where families wanted these, along with toys and crayons for the children.

Did this approach make the research more inclusive?

In the end, we worked with eighteen families representing Lambeth and Southwark’s diverse communities. There are things we could have done better, like involving participants in shaping the questions and design of the research, and many of the discussions with children would undoubtedly have worked better in person.

We’re in the final stages of writing up the project and have checked back with each family to make sure they are 100% happy with the way we have represented them. This was time-consuming, but we couldn’t have worked so closely with them over so many months and then taken their stories out of their hands.

Some of the families have shared with us how much they had valued being listened to. They knew the limits of our roles – as researchers rather than people who could influence any support they were receiving or might receive in the future – but recognised there was an opportunity to raise awareness of behavioural difficulties and what daily life is like for many families.  

What next?

The Impact on Urban Health project is just one example. Renaisi works on many more research and evaluation commissions that share similar challenges around inclusivity.

We know that this work is a journey and we’re not done yet so over the coming months we want to share our values with prospective and existing clients so they know what to expect and the kinds of discussions they might expect to have with us about inclusivity.

We are developing a practical checklist to use throughout a project that empowers our project teams to reflect on their practices and take concrete steps towards more meaningful change that shifts the imbalances in our sector.

Get in touch if you’d like to discuss inclusivity in relation to your research or evaluation project.