Unobviously #1 – The User comes Second

The hidden trap of field research, and how to see past the performance

🚗 We are driving from village to village in rural Karnataka. It is April and the roads are lined with more Gulmohars than I have ever seen. It is quite literally a feast for my eyes. Soon, our destination, a primary school rolls into our view. My researcher heart wonders – what secrets does this one hold?

Seriously, what’s not to enjoy about doing field research? (Okay, apart from the logistics.) It takes you on journeys you wouldn’t have otherwise embarked on. It feels like peeking inside houses, catching glimpses of the hidden worlds within. It is instant gratification for our story hungry minds.

You get to be in the (powerful) position of asking questions, so many questions. Outside of this role as a researcher, you probably wouldn’t have been able to conduct such scrutiny over someone else’s life – their relationships, work, dreams and failings.

You feel an added sense of pride due to the belief that field research is the kind of research that really counts. That there is no way to create authentic, useful solutions if you have not stepped on the field and spoken to users.

True.

But, many years of being on the field and conducting interviews have uncovered for me some other truths too.

…

🎬 What if the glimpses of that life and world you caught were merely a performance?

You may already know that as researchers, we alter the field simply by being on it. This Involvement Paradox, under different names, is a vastly studied shortcoming of qualitative research. As Langley and Klag point out, “…involvement is inherently paradoxical: On the one hand, ‘being there’ is seen as crucial for deep understanding. On the other, ‘being there’ may potentially reorient, directly or indirectly, what is available to be understood.” [1]

Think of how your way of speaking changes when there is a guest in your house. Or how you behave better, or just look nicer when you know you’re being videotaped. Now think - is it truly possible for someone to be authentic when they know they’re being observed? Oh, also recorded!

👀 Now here is what you might not know –

While many researchers know about their paradoxical position and power (or curse) they hold to alter the field, they often believe that they would be different from the rest. They feel that they will be able to make authentic observations and have ‘real’ conversations, despite the Involvement Paradox. This illusory superiority, a cognitive bias programmed into our minds, keeps each of us hopeful of being a better-than-the-average-researcher, good enough to look through performances and in 20-40mins of conversation decode the participant’s truth.

💩 Well, in Hagrid’s words - that’s a load of codswallop. Here’s why.

‘Looking through performances’ happens through your very subjective interpretation. Such analysis will be slightly different for every researcher, heavily influenced by your personal world view and biases. Thus, your interpretation of the field is likely to be correct only and only if you understand your participant like the back of your hand. But to know them you need to decode their performance, and to decode their performance you need to know them.

Yet another paradox.

Maybe if we were to just quietly observe, become flies on the wall, we would be able to gather more authentic data? Probably. However, for that you need users to get used to your presence. They need to start seeing you as a ‘regular anomaly’ in their group. This needs long term immersion. But we never have that kind of time. We usually are outsiders, incapable of becoming flies on the wall.

…

🚩 So, do we call quits?

Over the past week, we have visited more than 20 government schools, spoken to more than 60 teachers. Everywhere, I have heard the permanent teachers complain about the temporary (contractual) teachers – their lack of ownership, their lax attitudes and their unwillingness to contribute to teacher tasks. The fact that admin tasks were now happening on an app, and only permanent teachers had user ids, meant that the burden of that mountain of work was solely on them. Temporary teachers were able to shirk it off easily. There were many schools with a permanent teacher count as low as 2. Thus there was barely any scope of distributing and reducing work.

We are chit-chatting in the car now – it’s been a long day and we are heading back to our hotel. I am with Aarthi and Ramesh (names changed), field team members from the organisation I am doing this project for. I say - “This temporary teacher situation is so terrible.” Aarthi tells me casually, “It’s all drama. Temp teachers do all the admin work. Permanent teachers login from their mobiles and hand it to them. I have spent years in these schools, I know.”

Well, we don’t quit, but we sure pivot.

…

⚱️ Unobvious Takeaway: Field Teams Are Gold Mines

- Researchers often see the field team as a channel to reach the end users, a way to get the foot in the door. Change this view. Don’t just spend time making small talk or unintentional conversation with them while they take you from one user to another. Instead, treat them as a primary respondent – one as important as the end user.

- Organisations usually realise the value of their field teams and seek their perspectives. However, there is still a bottom of the pyramid feeling to how they opinion influences project decisions. There needs to be a concentrated effort in increasing their confidence, agency and the org’s ability to process the multitude of field realities experienced only by them. Tools like Ooloi Labs’ Open Knowledge Framework can prove to be very beneficial to achieve this.

…

🎯 Here is how our practice at fold labs has evolved with this understanding –

1. Project Kick-off
   We begin every project with knowledge-gathering sessions involving the field team. We create conversation guides to draw out their latent knowledge about the end user. We bear in mind that just like other folks, they come with their own biases. Thus we include mechanisms that help us understand these biases too.

2. Research & Analysis
   We try to have field team members with us when we interview end users, even if it is a virtual conversation. We follow this up with joint debrief sessions. It helps us understand how genuine the respondent was and if a certain observed nuance was specific to that respondent only, or could be generalized to the larger population of users. This helps strengthen the quality of qualitative analysis.

3. Prototype

   We seek prototype reviews by the field team who would be directly implementing the solution. It is usual to receive very pragmatic inputs from them. This helps us tweak our designs and make them implementation-aligned from the get-go.

4. Design & Development

   For every micro-decision during design (like vocabulary choices and character names in videos), we consider the field team’s voice as representative of the user perspective.

…

🍰 Long story short…

Field teams will turn out to be your biggest allies in truly understanding the field and creating user-centred outputs if you partner with them. I am not saying that talking to users is not important. It is and always will be. But, the field team is a powerful performance filter and a repository of priceless knowledge. It is quite literally impossible to design meaningful outcomes without their inputs and implement it without their buy-in.

…

Recently, I met an organisation to discuss possible ways we could work together. I needed to understand their existing training programs properly and had only a day’s time. They offered - “We can arrange on-field conversations with some trainers and students so that you get a sense of their experiences.”

Guess who I requested to meet instead?

…

[1] Langley, A., & Klag, M. (2019). Being Where? Navigating the Involvement Paradox in Qualitative Research Accounts. Organizational Research Methods, 22(2), 515-538. https://doi.org/10.1177/1094428117741967

---