Why (qualitative) analysis is like catching findings with a net, not building a wall of evidence

This is a short post using a the idea of a fishing net to think about qualitative data analysis. It’s not a practical strategy in itself, but it can help cope with some of the common difficulties in analysis.

I say (qualitative) analysis but qualitative analysis is what I have most experience of. But I have a feeling the same might apply to quantitative analysis too, or at least aspects of it.

[And for the environmentally minded among you, consider the fishing referred to here as very sustainable, selective fishing, not mega-trawlers catching everything! I’m thinking of the small nets a person might throw into the ocean by hand seeking a modest catch.]

What are the relevant features of a fishing net?

Here is a picture drawn by Kate Hughes (and used with her permission). Think of a fishing net. It is a series of thin strings tied together.  The net is not a solid sheet of material. Most of it is actually holes.

Despite being way more ‘gaps’ than substance, a net can catch and hold lots of fish.

Indeed, nets can only work because they are mainly holes. You couldn’t drag a solid sheet through the water.

What does this have to do with qualitative data analysis?

A common struggle I see qualitative researchers confronting is a feeling that they  don’t have enough ‘proof’ for the claims they want to make.

Now, of course it is good intellectual hygiene to doubt one’s claims, asking: Could it be otherwise? What other interpretations might be possible? How could I challenge this claim? etc.

But is often not healthy to feel that to make any claim you have to build up a solid wall of evidence if this means everything you claim has its direct match in the data.

Yes, I know this sounds odd. Stay with me.

The point of analysis is to do something with the data. To go beyond the data. To find new meanings. To say something that the data themselves don’t say directly. Otherwise you are just (selectively) reporting. In which case, you might as well just publish your transcripts, observation notes (etc.) in raw form.

I am not advocating a free-for-all where you can claim anything. We remain bound to some degree by what we can and can’t say because of what is and isn’t in the data.

But I do think it helpful to imagine analysis being like tying threads together to catch ideas and insights.

Some ideas and insights, like the tiny fish that pass happily through the net, won’t hold. They may be slippery, stable meanings,  confident insights or robust claims might evade us.

Some might just be too much. Like the big, heavy fish that could break the net, some claims might be more than the data and any analytical method can bear.

But the ones that work will be like the fish the net is designed to catch. Big enough to get caught, not too big to break the net.

 

Yes, but what does this mean?

Well, it means in analysis, and its writing up, we are not trying to prove everything with a direct quotation from data. That’s a symptom of quotitis, and not a good thing.

What we are trying to do is to find strong threads that can withstand the forces needed to keep the insights or interpretations (fish) in place. And strong knots to keep the holes in the right shape so it doesn’t all slip about.

I like the net idea because a net is not rigid. It moves, flexes, bulges, sways. Just like our analysis should. But it also holds tight. Firm but flexible. Robust not rigid. The threads might be bits of data, or lines of questioning we apply to the data. The knots might be theories or conceptual frameworks that allow us to tie this data to that data, this meaning to that meaning.

When you think about it, imagine how heavy a net is when you first throw it in. Not heavy at all – light enough to pick up and throw. Now, think about when it is full of fish. Much heavier. But still what a person might (with some effort) pull in and feed off.

The same about analysis. Being able to haul in great claims is not about big heavy machinery. It’s about using agile, efficient tools. The right kinds of questioning, theory and procedures (including play – see Pat Thomson on this!) and be just enough.

The fisherperson doesn’t use the biggest possible net or the thickest, strongest possible line. They do not tie the strongest, most complex possible knots. They choose the optimal balance between size, weight, strength and stretch.

I find this a refreshing way to think about analysis. It’s about weaving and tying threads together, just enough to hold things in place so you can haul out your claims, based on rich insights.

Just like the fisher chooses the net, using materials that are available, adaptable and suitable, you, the analyst choose your threads and knots. There is nothing automatic about this. Nothing given.

If you’re after very detailed, up-close findings, you’ll need an analytical approach to match. The fisher would choose a net with small holes. Conversation analysis strikes me as one of those very fine-grained approaches to analysis, where every um, and ah, and pause etc is transcribed. A fine analytical weave indeed.

If you’re after grander claims, about big social phenomena, then a different approach might serve you better. Other kinds of discourse analysis might allow you to ‘see’ things like racism, sexism, injustice – catching larger fish – but your net will be quite okay with relatively large spaces between the knots.

Of course there are heaps of theories around, and heaps of analytical approaches, and they don’t match simply to scale in the way I’ve suggested above. But hopefully you get the point!

In summary

Evidence or data can only ever be provisional, a pattern of well woven strings that hold up a bulkier mass, rather than a seamless continuous entity. Analysis is more like weaving threads to make a net than building a wall of evidence. Agile, flexible, light and strong. Not immutable, rigid and opaque.

Let me know

Do you find this idea useful? The opposite? How do you think about analysis in ways that allow for agility and escaping the burden of needing to ‘prove’ everything with a quote or extract from data?

 

 

 

Advertisement