Why (qualitative) analysis is like catching findings with a net, not building a wall of evidence

This is a short post using a the idea of a fishing net to think about qualitative data analysis. It’s not a practical strategy in itself, but it can help cope with some of the common difficulties in analysis.

I say (qualitative) analysis but qualitative analysis is what I have most experience of. But I have a feeling the same might apply to quantitative analysis too, or at least aspects of it.

[And for the environmentally minded among you, consider the fishing referred to here as very sustainable, selective fishing, not mega-trawlers catching everything! I’m thinking of the small nets a person might throw into the ocean by hand seeking a modest catch.]

What are the relevant features of a fishing net?

Here is a picture drawn by Kate Hughes (and used with her permission). Think of a fishing net. It is a series of thin strings tied together.  The net is not a solid sheet of material. Most of it is actually holes.

Despite being way more ‘gaps’ than substance, a net can catch and hold lots of fish.

Indeed, nets can only work because they are mainly holes. You couldn’t drag a solid sheet through the water.

The Net.jpg

What does this have to do with qualitative data analysis?

A common struggle I see qualitative researchers confronting is a feeling that they  don’t have enough ‘proof’ for the claims they want to make.

Now, of course it is good intellectual hygiene to doubt one’s claims, asking: Could it be otherwise? What other interpretations might be possible? How could I challenge this claim? etc.

But is often not healthy to feel that to make any claim you have to build up a solid wall of evidence if this means everything you claim has its direct match in the data.

Yes, I know this sounds odd. Stay with me.

The point of analysis is to do something with the data. To go beyond the data. To find new meanings. To say something that the data themselves don’t say directly. Otherwise you are just (selectively) reporting. In which case, you might as well just publish your transcripts, observation notes (etc.) in raw form.

I am not advocating a free-for-all where you can claim anything. We remain bound to some degree by what we can and can’t say because of what is and isn’t in the data.

But I do think it helpful to imagine analysis being like tying threads together to catch ideas and insights.

Some ideas and insights, like the tiny fish that pass happily through the net, won’t hold. They may be slippery, stable meanings,  confident insights or robust claims might evade us.

Some might just be too much. Like the big, heavy fish that could break the net, some claims might be more than the data and any analytical method can bear.

But the ones that work will be like the fish the net is designed to catch. Big enough to get caught, not too big to break the net.

 

Yes, but what does this mean?

Well, it means in analysis, and its writing up, we are not trying to prove everything with a direct quotation from data. That’s a symptom of quotitis, and not a good thing.

What we are trying to do is to find strong threads that can withstand the forces needed to keep the insights or interpretations (fish) in place. And strong knots to keep the holes in the right shape so it doesn’t all slip about.

I like the net idea because a net is not rigid. It moves, flexes, bulges, sways. Just like our analysis should. But it also holds tight. Firm but flexible. Robust not rigid. The threads might be bits of data, or lines of questioning we apply to the data. The knots might be theories or conceptual frameworks that allow us to tie this data to that data, this meaning to that meaning.

When you think about it, imagine how heavy a net is when you first throw it in. Not heavy at all – light enough to pick up and throw. Now, think about when it is full of fish. Much heavier. But still what a person might (with some effort) pull in and feed off.

The same about analysis. Being able to haul in great claims is not about big heavy machinery. It’s about using agile, efficient tools. The right kinds of questioning, theory and procedures (including play – see Pat Thomson on this!) and be just enough.

The fisherperson doesn’t use the biggest possible net or the thickest, strongest possible line. They do not tie the strongest, most complex possible knots. They choose the optimal balance between size, weight, strength and stretch.

I find this a refreshing way to think about analysis. It’s about weaving and tying threads together, just enough to hold things in place so you can haul out your claims, based on rich insights.

Just like the fisher chooses the net, using materials that are available, adaptable and suitable, you, the analyst choose your threads and knots. There is nothing automatic about this. Nothing given.

If you’re after very detailed, up-close findings, you’ll need an analytical approach to match. The fisher would choose a net with small holes. Conversation analysis strikes me as one of those very fine-grained approaches to analysis, where every um, and ah, and pause etc is transcribed. A fine analytical weave indeed.

If you’re after grander claims, about big social phenomena, then a different approach might serve you better. Other kinds of discourse analysis might allow you to ‘see’ things like racism, sexism, injustice – catching larger fish – but your net will be quite okay with relatively large spaces between the knots.

Of course there are heaps of theories around, and heaps of analytical approaches, and they don’t match simply to scale in the way I’ve suggested above. But hopefully you get the point!

In summary

Evidence or data can only ever be provisional, a pattern of well woven strings that hold up a bulkier mass, rather than a seamless continuous entity. Analysis is more like weaving threads to make a net than building a wall of evidence. Agile, flexible, light and strong. Not immutable, rigid and opaque.

Let me know

Do you find this idea useful? The opposite? How do you think about analysis in ways that allow for agility and escaping the burden of needing to ‘prove’ everything with a quote or extract from data?

 

 

 

7 thoughts on “Why (qualitative) analysis is like catching findings with a net, not building a wall of evidence

  1. Yvonne Young

    I found this way of thinking reassuring and helpful. I come from a traditional history background and have changed disciplnes. I worry about the weight of evidence and have been struggling to cast my own net. I think this makes me high risk for quotitis. But I am beginning to see that raw data is indeed an unfinished product. It is rather like having raw ingredients sitting in the fridge ready to make dinner. I need to turn it into a meal. Somebody else could use the same ingredients and make something quite different. Both meals could be equally tasty !

    Reply
  2. Evie Bamigbade

    This is really helpful! Thank you very much for sharing your insights. I caught quotitis and found it validating to know that I have the freedom to move beyond the words. I think it’s part of the fear of defence. Wanting to back everything up but staying too close to their words make it harder to move beyond being simply descriptive.

    Reply
    1. nickhopwood Post author

      Hi Evie
      Thanks for your comment! You’re far from alone in having had quotitis! I’ve caught it several times myself, often when I least expected it! It sounds like you have found a way through towards something where readers get the best combination of authentic data and your voice 🙂

      Reply
  3. Kate Hughes

    Thanks Nick. I will use that in the synthesis of findings. But Ipleased bye advisedI did not go with the net approach. Like a net, a cable of reasoning has many strands but they come together in a different way. More of this later hen time permits.

    Reply
    1. nickhopwood Post author

      Hi Kate – thanks for your comment! Ooh, yes I would love to hear more about a ‘cable of reasoning’ with many strands coming together, when time permits!

      Reply

Please join in and leave a reply!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s