Let me start this post by positioning myself:

1. I love to play with new technology. I already put chat GPT to good use for some purposes.
2. I have been a qualitative researcher for 30 years and started to use specialised software for data analysis. The only analysis I did manually was for my Master’s thesis, which was back in 1992.

The question I would like to discuss in this post is:

Can AI and machine learning tools help you to analyse qualitative data?

Let’s look at some examples. A topic search for a study about environmental change among residents in a coastal area resulted in the following main topics:

• Fishing
• Water
• Environment
• Change
• Industry
• Community
• commercial fishing, area
• thing
• planning
• gonna
• houses
• shrimp
• place and

Looking at the interview guide for this study, no surprises here. This is what the entire study was about.

I could now use the auto-coding tool and let the software code all of these topics automatically. While doing this, I can decide how much context I want – a sentence or a paragraph.

Then I can quickly generate an overview, where I can see how the codings of these various themes are distributed among the data.

Or I could create a table where I can see which respondents used the word fishing and its sub-topics:

But what does this mean? What kind of result can I extract from this?

The following is an example of a piece of text that coded using the word ‘fishing’:

“He does fishing, commercial fishing, but actually, he’s not commercial, just for his own freezer.”

Analysing data as a qualitative researcher, the question always is: What is this all about?

Regarding the above example, it is not possible to say what this is all about. More context is needed.

So I expanded the context. And then you learn this:

The respondent was not talking about fishing per se but rather about enjoying the outdoors together with her uncle – who no longer seems to engage in commercial fishing. The code, thus, would not be ‘fishing’ but something like leisure activities.

Why the uncle is no longer engaged in commercial fishing, we might find out later. If so, the code for this would not be ‘commercial fishing’ – a word a machine learning tool can find. It would be about the reason why he gave up on it.

I could go through more of the data segments to give you more examples. But then this would become a very long post. The point is – given the current state of technology – AI is not able to go beyond the words it finds. It cannot read between the lines and tell us what the response is all about.

Often, what a response is all about is not in the words that are used. It is about what these words mean in a given context.

I can only shake my head and smile if I see texts like these:

Remove the headache from qualitative data analysis and lets the software do the heavy lifting. Generate deep insights leveraging the latest AI and machine learning algorithms for faster results.

That’s is a load of….. (fill in the blanks). 

Let’s take a look at another popular tool. Most programs will offer sentiment analysis. This works best if you have hundreds or thousands of short answers to open-ended questions from a survey—thus, very structured data. It might also be helpful for analysing customer reviews for products. Based on my experience, you better double-check the results.

It is certainly not suitable to let a sentiment analysis run over qualitative interview data. I have also tested it with social media comments. You cannot rely on the automated process. You must go through all data and adjust the coding where the machine got it wrong.

Therefore it is an empty promise that AI tools will allow you to analyse thousands of posts or customer reviews quickly. The AI tool will indeed be done in a few minutes. It will, however, take you days to double-check and correct all miscoded data.

Does this mean that AI or machine learning tools are not useful at all for qualitative data analysis?

If you understand what they can and cannot do, yes, they can be helpful.

Let’s go back to the fishing example. In qualitative data analysis, you can start with broad-brush coding. This means you go through and do some rough coding by topics. This is something a topic or concept search can do for you.

It's helpful if you need information on just a few topics. It could also be a way to go through long reports and “fish out” segments that are likely to be relevant.

However, this first rough coding is not the final result!

After this step, you need to go into the data and read all of the data segments that the machine found. You probably need to adjust the length of the auto-coded segments and also the coding.

If someone presents results of a qualitative study and proudly announces that AI has done all of the codings, or if someone has results for you a few hours after all data were collected - you should be very suspicious.

As a qualitative researcher, I am very happy that data transcription can now mainly be done by a machine  - still, some checking is required. But this is an outstanding achievement, saving lots of time.

Using machine learning and AI for coding qualitative data, however, still has some way to go.