Assessing the time required for qualitative analysis: A comparative methodological study of coding interview data in health services research

Abstract

Background

A reliable estimation of required resources is essential for sound research. So far, there have only been a few studies on researchers’ time investment in qualitative studies. The aim of this study, therefore, was to provide an empirical account of the estimation of timescales of qualitative analysis.

Methods

In this methodological study, time expenditure was documented and compared for the focused coding of transcripts of semi-structured interviews within five qualitative studies in health services research. Data were analyzed descriptively by means of absolute frequencies.

Results

Across studies, focused coding was assessed in 94 interviews with a total interview duration of 52 hours and 44 minutes. The number of interviews per study ranged from n=11 to n=27, with a mean duration of 36 minutes. Total coding time amounted to 76 hours, with a mean of 32 min per interview. Coding time per interview time ratio ranged from 0.75 to 1.52 minutes. On average, the time spent on focused coding roughly corresponds to the duration of the interviews. Focused coding tended to get quicker over time, though variation among studies was high.

Conclusion

The results of this study provide a reference for estimating timescales of qualitative analysis and highlights the importance of considering factors such as composition of data and researchers’ experience and involvement. In a specific research project, this effort must be balanced against the objective of the analysis, including the desired accuracy, detail and depth. Further research is needed to specify how specific parameters (i. e. nature of the study population, method of data analysis and use of concepts and theories) affect coding in qualitative analysis.

Keywords

Methodological study - qualitative analysis - coding time - health services research - study planning - qualitative interviews

Schlüsselwörter

Methodologische Studie - Qualitative Analyse - Kodierzeit - Versorgungsforschung - Studienplanung - Qualitative Interviews

Background

A reliable estimation of the time and resources needed to conduct a study is essential for sound research. This is especially true for health services research, where timely results are often needed to inform healthcare practice [1]. To meet these needs, different suggestions have been made for the adaption of study designs and methods: In qualitative research, these suggestions include so-called rapid designs [2] [3], omitting or speeding up the process of transcribing data [4], and pragmatic approaches to data analyzing, e. g. by using frameworks [5], tables and spreadsheets [6] or mapping techniques [7]. Data analysis in particular is discussed as a labor-intensive and time-consuming task [8] [9] [10]. Data analysis is described as often underestimated and challenging: “Even seasoned qualitative researchers can find the process of coding their datum corpus to be arduous at times. For novice researchers, the task can quickly become baffling and overwhelming“ [11]. Applied researchers in particular might find data analysis a ‘daunting task’ [6].

Within this context, the common approach of audio-recorded interviews and in-depth analysis of verbatim transcription has been challenged. So far, the focus in this debate has been on transcriptions, highlighting the potential of approaches directly using notes and/or recordings for analysis [4]. At the same time, speech recognition software and the use of artificial intelligence (AI) might significantly speed up transcription time [12]. Time needed for data analysis, on the other hand, is more difficult to evaluate, as a broad range of approaches is used in qualitative research. Estimations of time and effort for data analysis are rare and mainly stem from anecdotical experience. For example, in a recent textbook, data analysis is projected to take up to half of the time within a research project [9]. For analyzing focus groups, it is suggested that inexperienced researchers might need 30% more time than experienced researchers [13].

There is only a limited number of studies investigating data analysis empirically, mostly comparing traditional forms with rapid forms of data analysis: A comparative study by Gale et al. [14] based on 30 semi-structured interviews showed that the in-depth analysis took 10 weeks longer to complete than rapid analysis. Tylor et al. [15] compared rapid and thematic framework analysis based on 21 semi-structured interviews, focus groups and documents and concluded that rapid analysis delivered a modest time saving: Early analysis and review took about a third of the time of thematic analysis, but rapid analysis interpretation and write up took more than six times longer than thematic analysis. Nevedal et al. [16] compared framework-based deductive traditional and rapid approaches in two similar data sets of semi-structured interviews of approximately 50 hours audio total: In sum, data analysis took 5.5 h/interview within the traditional approach applied to interview transcripts (n=57) and 3.9 h/interview within the rapid approach applied to notes and timestamps on audio recordings (n=72). Further data interpretation required the same number of hours in both approaches (100 h). Eaton et al. [17] compared thematic analysis from scribed interviews (documentation of comprehensive notes) and verbatim transcription of the same six interviews. When compared to verbatim transcription, processing data into text form and subsequent analysis was associated with significant time saving. Neal et al. [16] developed a procedure for rapid identification of themes from audio recordings (RITA), stating a coding time 13% longer than the length of an interview.

Though findings are mixed, these studies generally indicate rapid approaches to be faster. However, details on analysis time were unspecific in most studies (e. g. regarding interview duration or number of interviews) and separation of analytic steps (e. g. transcription, analysis and interpretation) somewhat indistinct, making it difficult to draw reliable conclusions for planning and conducting qualitative studies.

Aim

The aim of this study, therefore, was to provide an empirical account on estimating the time required for qualitative analysis. We focused on in-depth analysis of transcript audio recordings from interviews, as this approach is still widespread. Hereby we wanted to provide a reference for a) further methodological studies investigating the balance of rigour and speed and b) the selection of traditional and/or rapid methods to meet the analytical needs in health services research.

Methods

We carried out a methodological study [18] documenting and comparing the time taken for data-coding within five interview-based studies in health services research undertaken at the Department of General Practice and Health Services Research, University Hospital Heidelberg, Germany. This corresponds to all studies analyzing qualitative data at the time of this study at the department.

Project context of included interview studies

All five included interview studies were embedded in larger mixed-method projects investigating health care services (s. [Tab. 1]): CCC was a process evaluation study of a counseling intervention to educate cancer patients on complementary and integrative health care and to promote interprofessional health care (CCC-Integrative study) [19]. ExKoCare was an observational study and explored coordination and uptake of recommended cardiovascular care in ambulatory care [20] [21], PRiVENT was a process evaluation study investigating the implementation of interventions by weaning experts in patients’ risk of long-term invasive ventilation in intensive care units [22] [23], RESILARE developed quality indicators measuring crisis resilience in primary care practices [24] and UCC developed a disease management concept for venous leg ulceration [25]. All studies were publicly funded for three to five years.

Tab. 1 Overview of project contexts of included interview studies

	CCC	ExKoCare	PRiVENT	RESILARE	UCC
Full title	Process evaluation of a counselling intervention designed to educate cancer patients on complementary and integrative health care and promote interprofessional collaboration in this area (CCC-integrative-study)	ExKoCare: Cooperation networks of ambulatory health care providers: exploration of mechanisms that influence coordination and uptake of recommended cardiovascular care	PRiVENT: Prevention of invasive ventilation	RESILARE: building crisis resilience of primary care practices by developing and evaluating quality indicators	UCC: Development and evaluation of an evidence-based and patient-oriented care concept for the primary care of patients with venous leg ulceration
Term	2019–2023	2019–2023	2020–2025	2021–2024	2020–2024
Funding	Innovation Committee of the Federal Joint Committee, Germany (G-BA): 01NVF18004	German Research Foundation: 416396249	Innovation Committee of the Federal Joint Committee, Germany (G-BA): 01NVF19023)	Innovation Committee of the Federal Joint Committee, Germany (G-BA): 01VSF20029	Innovation Committee of the Federal Joint Committee, Germany (G-BA): 01VSF19043
Overall study design	Mixed-method process evaluation	Mixed-method observational study	Prospective interventional multi-center study	Three-part design: 1) Systematic literature research and qualitative study, 2) modified RAND/UCLA approach, 3) piloting and mixed-methods process evaluation	Observational cross-sectional mixed-methods process evaluation

Data collection and data analysis

A spreadsheet was developed for documentation of coding comprising project name, interview participants, interview number, interview length, date of coding, coder names and coding duration (CU, online appendix 1). Documentation was done independently by all researchers (CA, TF, SK, NK, RPD, CU), guided by written instructions.

In order to be able to compare coding times, time (in minutes) required for coding was divided by the length of the interview (in minutes). A coding-time-per-interview-time-ratio (CIR) of 1 means that the coding time corresponds to the interview time. A CIR>1 means coding took longer than the interview duration, while a CIR<1 indicates coding was shorter than the interview duration.

Data were analyzed descriptively by means of absolute frequencies. Besides tables, a boxplot was used to visualize differences between coding researchers, showing minimum value, first quartile (25^th percentile), median, third quartile (75^th percentile), and maximum value. Values were depicted as outliers when the observation was 1.5 times the interquartile range below the first or above the third quartile. To illustrate trends over time, data were compared in chronological order of coding, displaying trend lines to indicate general directions.

We followed the PRISMA-ScR guideline for design and reporting of this study where applicable (s. online appendix 2), as a reporting guide for methodological studies is not yet available [26] [27] .

Ethics approval

Ethics approvals for the original studies were granted by the respective bodies [19] [20] [22] [24] [25]. Participants provided consent for the respective studies and subsequent data use. For the purpose of this methodological study, only documentation of coding time by the respective researchers was used. All researchers provided consent.

Results

Sample description and overview

All included interview-studies examined health care practice in terms of processes, quality and/or outcomes using an explorative approach informed by previous findings (s. [Tab. 2]). All were explicitly informed by conceptual and/or theoretical perspectives, e. g. frameworks. For data collection, all used semi-structured interviews with different stakeholders comprising open-ended questions on a limited number of themes. Interviews were conducted in German via telephone, audio-recorded and transcribed verbatim. The number of included interviews per study varied from 11 to 27 interviews. Except for one study, all interviews in the respective study phase of each project conducted by the time of this study were included. Due to personnel changes in the research project, only the first 11 interviews of 26 in RESILARE were included in this study. Within UCC, CCC, and RESILARE, additional interviews were conducted and/or analyzed after completion of this study. Within most studies, the interviews included were the initial and/or only qualitative data analyzed. Within CCC, recordings of consultation were analyzed beforehand; within RESILARE, one focus group was analyzed before and one during interview analysis. Within all studies, data analysis was applied to complete interviews.

Tab. 2 Overview of included interview studies

	CCC	ExKoCare	PRiVENT	RESILARE	UCC
Aim	Exploration of HCPs’ perspectives on the intervention	Exploration of GPs’ perception of coordination with cardiologists and obtainment of cardiological knowledge	Evaluation of the implementation of weaning boards and weaning councils in intensive care units	Exploration of stakeholder perceptions of potential crises, mitigation strategies and awareness of climate change adaptation	Process evaluation of the pilot implementation of a care concept for venous leg ulcerations.
Conceptual background	Consolidated Framework for Implementation Research (CFIR) [26]: Applied in overall design of process evaluation, including development of interview guides and conduction of data analysis	Prior quantitative findings on cooperation Theories on information seeking behavior [27] [28]	Consolidated Framework for Implementation Research (CFIR) [26]: Guided semi-structured interviews and used to sort codes within data analysis	Theories on behaviour change [29] [30]	Theoretical Domains Framework of Behaviour Change (TDF) [29]
Time of interview conduction	2021/08–2021/12	2021/06–2021/11	2022/04–2022/06	2021/07–2021/10	2021/07–2021/12
Number of interviews	16	27	14	11 (of 26)	26
Participant groups	2 (12 counselors and 4 counseling team leaders)	1 (GPs)	2 (7 clinicians, 7 other HCP)	1 (GPs)	4 (7 GPs, 6 other HCP, 8 patients, 5 stakeholder)
Main themes in focused coding	Enabling (n=4) and hindering (n=4) factors for the implementation, examined on patient, provider, and system level	Motives and orientation of GPs, occasion for coordination, information behaviour, teamwork (n=4)	Determinants of the implementation of weaning boards and weaning councils at the structural and personal level of intensive care staff (n=4)	General, organisational resilience, climate change adaptation and mitigation, potential quality indicators (n=3)	Program acceptance, perceived effects, contextual factors, intervention fidelity and reach (n=5)
Number of analytical codes	45	27	23	20	23
Number of analytical codings	779	656	637	735	1570
Targeted publications	1–2 papers	1–2 papers	1 master’s thesis (subsequent paper)	1–3 papers	1–2 papers

CCC: Process evaluation of a counselling intervention designed to educate cancer patients on complementary and integrative health care and promote interprofessional collaboration in this area (CCC-integrative-study); ExKoCare: Cooperation networks of ambulatory health care providers: exploration of mechanisms that influence coordination and uptake of recommended cardiovascular care; PRiVENT: Prevention of invasive ventilation; RESILARE: building crisis resilience of primary care practices by developing and evaluating quality indicators; UCC: Development and evaluation of an evidence-based and patient-oriented care concept for the primary care of patients with venous leg ulceration

Across included studies, different types of qualitative data analysis were used, e. g. reflexive thematic analysis [28] and framework analysis [29]. All of these share a combined inductive and deductive approach, comprising four broad stages: a) familiarization, b) open coding, c) focused coding and d) development of analytical themes/write up [9]. The first two steps as well as the development of analytical themes and write-up often have substantial overlaps, wherefore, within this study, we concentrated on examining the third stage of focused coding. Focused coding is an umbrella term used to describe targeted analysis based on initial themes identified during open coding. While open coding involves exploring data into various directions to uncover initial insights, focused coding narrows down the analysis to highlight those insights most relevant to the research questions. Therefore, within focused coding, the data is recoded according to specified themes and codes [30]. Based on the results of focused coding, researchers refine their understanding and develop a clearer picture of analytical themes within the data.

In focused coding, the qualitative data analysis software MAXQDA was used in all projects. Number of main themes, analytical codes and codings differed, but were largely on a comparable scale. For one researcher (SK), the analysis was part of a master’s thesis in form of a complete manuscript draft. Within all other cases, the analysis was done with the aim to publish research results within at least one scientific paper.

Researchers’ characteristics and roles within the interview studies

At the time this study was conducted, all six involved researchers were based at the Department of General Practice and Health Services Research, University Hospital Heidelberg, Germany, having collaborated in research and/or teaching in different compositions beforehand. Two researchers had a professional background in nursing (CA, SB). Academic education varied, but all researchers had a background in health services research. The level of experience in (qualitative) research differed: Three were post-doctoral researchers with experience in the conduction and supervision of several qualitative research projects for at least five years (NK, RPD, CU), three were doctoral (CA, TF) or master’s students (SB) with experience in one or two qualitative research projects at the time of the respective study. One researcher was involved in two of the included interview studies (RPD).

Within the studies, researchers either took the role of primary researcher within the overall study, were responsible for the entire project at the time of this study or were mainly involved in the qualitative interview study, often adopting an advisory role within the overall study as well (s. [Tab. 3]). Regarding the qualitative interview studies, primary researchers were always involved in interview conduction. In most studies, transcription was done by supporting staff using transcription software. Only one researcher transcribed the interviews herself (SB). All researchers read and coded all interviews. Data analysis was discussed among respective research teams.

Tab. 3 Researchers roles within the interview study (at the time of this study)

	CCC	ExKoCare		PRiVENT	Resilare	UCC
	NK	CA	CU	SK	RPD	RPD	TF
Role within the primary study
Primary researcher/investigator	x	x	–	–	–	–	x
Focus on qualitative study	–	–	x	x	x	x	–
Advisor of overall study	–	–	x	–	x	x	–
Tasks (involved in) within the qualitative study
Interview guide development	x	x	x	x	–	x	x
Interview conduction	x	x	–	x	–	x	x
Transcription	–	–	–	x	–	–	–
Analysis of the data set	x	x	x	x	x	x	x
Write-up and publication	x	x	x	x	x	x	x

CCC: Process evaluation of a counselling intervention designed to educate cancer patients on complementary and integrative health care and promote interprofessional collaboration in this area (CCC-integrative-study); ExKoCare: Cooperation networks of ambulatory health care providers: exploration of mechanisms that influence coordination and uptake of recommended cardiovascular care PRiVENT: Prevention of invasive ventilation; RESILARE: building crisis resilience of primary care practices by developing and evaluating quality indicators; UCC: Development and evaluation of an evidence-based and patient-oriented care concept for the primary care of patients with venous leg ulceration; CA: Christine Arnold, CU: Charlotte Ullrich, NK: Nadja Klafke, RPD: Regina Poß-Doering, SB: Sabrina Brinkmöller, TF: Thomas Fleischhauer

Interview data and focused coding

Across studies, focused coding was assessed in 94 interviews with a total interview duration of 52 hours and 44 minutes (see [Tab. 4]). Number of interviews per study ranged from 11 to 27, with a mean duration of 36 minutes. Interview length mean ranged from 21 minutes to 50 min. Total coding time amounted to 76 hours, with a mean of 32 min per interview. Coding-time-per-interview-time-ratio (CIR) ranged from 0.75 to 1.52 in mean interview time with a mean of 0.99 across studies.

Tab. 4 Duration of focused coding

	CCC	ExKoCare		PRiVENT	Resilare	UCC		Sum/mean
A. Interviews
Number	16	27		14	11	26		94
Duration: total	11:44:00	09:28:53		07:09:20	09:07:36	15:15:18		52:44:14
Duration: mean	00:44:00	00:21:04		00:30:40	00:49:47	00:35:12		00:36:09
Min	00:20:00	00:11:20		00:20:04	00:15:25	00:18:43
Max	00:59:00	00:36:39		00:48:31	01:16:09	01:18:38
SD	00:12:52	00:06:43		00:05:47	00:15:25	00:14:31
Time interval of coding	4 months	1 month		1 month	1 month	3 month (TF) 6 months (RPD)
Order of coding	Chronological	Chronological		Chronological	Mostly chronological	Chronological
Theme development	Inductively + CFIR	Primarily inductively		Inductively + CFIR	Primarily inductively	Inductively + TDF
B. Focused Coding
Researcher	NK	CU	CA	SB	RPD	RPD	TF
Total Coding Time	08:42:00	09:07:00	07:04:00	10:15:00	07:01:00	13:16:00	20:54:00	76:19:00
Per Interview
Mean	00:32:37	00:20:16:	00:15:42	00:43:56	00:38:16	00:30:37	00:48:14	00:32:48
Min	00:15:00	00:10:00	00:07:00	00:25:00	00:20:00	00:15:00	00:15:00	00:15:20
Max	01:05:00	00:55:00	00:30:00	01:00:00	01:00:00	01:00:00	01:30:00	01:10:00
SD	00:14:09	00:09:23	00:05:47	00:10:25	00:13:47	00:10:05	00:21:45	00:14:14
CIR (coding-time-per interview-time-ratio)
Mean	0.76	0.96	0.75	1.52	0.84	0.93	1.39	0.99
Min	0.38	0.61	0.44	0.82	0.37	0.51	0.70	0.65
Max	1.33	1.65	1.09	2.99	1.33	1.55	2.22	1.34
SD	0.26	0.26	0.17	0.56	0.33	0.28	0.44	0.17
C. Consenting
Total	4h	4h		–	–	5h
Number of meetings	5	12		–	–	3
Duration	30–60 min	10–30 min		–	–	90–120 min

CCC: Process evaluation of a counselling intervention designed to educate cancer patients on complementary and integrative health care and promote interprofessional collaboration in this area (CCC-integrative-study); PRiVENT: Prevention of invasive ventilation; RESILARE: building crisis resilience of primary care practices by developing and evaluating quality indicators; UCC: Development and evaluation of an evidence-based and patient-oriented care concept for the primary care of patients with venous leg ulceration; CA: Christine Arnold, CU: Charlotte Ullrich, NK: Nadja Klafke, RPD: Regina Poß-Doering, SB: Sabrina Brinkmöller, TF: Thomas Fleischhauer

Distribution and skewness of data differed between projects (s. [Fig. 1]). Distribution of data is compact for most researchers, with two exceptions (PRiVENT and UCC: TF). Distribution was rather symmetric in one coding (ExKoCare: CA) and somewhat positively skewed in one project (UCC: RPD). Outliers can be found in CIR of two researchers (ExKoCare: CU: 1.65, PRiVENT: 2.99). The median CIR lay between 0.71 (CCC) and 1.41 (UCC: TF) with a mean median of 0.98. Across projects, the 25^th percentile was 0.82 and the 75^th percentile was 1.02, with an interquartile range of 0.2.

Fig. 1 Coding-time-per-interview-time-ratio (CIR) per project and researcher.

Development of coding duration

Focused coding was performed over a period of one to six months. Within all studies, interviews were coded roughly in chronological order of the conduction of interviews. Codes were refined within focused coding. Within three studies (CCC, ExKoCare and UCC), codes and codings were regularly consented in 3 to 12 meetings respectively, amounting to approximately 4 to 5 hours.

All studies included at least 11 interviews, with a mean CIR of 1.34 (min: 0.43; max: 2.99) of the first interview and a mean CIR of 1.00 (min: 0.71; max: 1.79) of the eleventh interview (s. [Fig. 2]). Mean CIR of the respective last interview was 1.09 (min: 0.38; max: 1.85). Overall, focused coding tended to get quicker over time. However, variation among studies was high. Trendlines indicate the biggest increase in speed within the PRiVENT study (first interview: 2.99; last interview: 1.42) and a narrower but almost parallel increase in speed in CIR within UCC, ExKoCare and CCC. RESILARE was an exception, as the trendline showed an increasing coding duration (first interview: 0.43; last interview 1.03). Within all studies, there was fluctuation in CIR over time.

Fig. 2 Coding-time-per-interview-time-ratio (CIR) per interview.

Discussion

Principal findings

Results of this study provide a reference for estimating the time required for qualitative analysis. Previous studies indicated that rapid approaches, especially omitting transcription, were faster than traditional approaches to qualitative data analysis [4] [10] [16] [17] [31] [32]. Our results show that on average, the time spent on focused coding corresponds to the duration of the interviews (CIR: 0.99). This is comparable to findings on application of rapid identification of themes from audio recording (RITA) with a coding time of 68 min per 60 min interview, translating to a CIR of 1.13 [16]. However, the application of a framework-based deductive analysis approach led to a coding time of 275 h for 50 interview hours, translating to a CIR of 5.5 [33]. This difference might be due to this study addressing the whole process of data analysis, including independent coding by two analysts and regularly adjudicating differences, not only the step of focused coding.

Our results indicate that CIR tended to get quicker over the course of an analysis, with a mean decrease of 0.34 from the first (1.34) to the eleventh (1.00) interview. This decrease was expected by the research team: in qualitative analysis, the coding system becomes more precise over time and researchers become more confident in the application of codes. Starting with a higher factor, the time saved was greatest within a master’s thesis (SB). For doctoral students, factors were lower and more stable in one project (ExKoCare: CA, mean 0.75, SD: 0.17) and higher in another project (UCC: TF, mean 1.39, SD: 0.44). Potential influencing factors may be that in ExKoCare, only one participant group (GPs) was included, the doctoral student (CA) always coded second (after CU), and coding was carried out over a relatively condensed period of one month with regular consent meetings (n=12). In UCC, by contrast, four different participant groups were included (GPs, practice assistants, patients, other stakeholders), and coding was done largely independently with fewer consent meetings (n=3) of the two researchers involved within a coding period of three months.

Limitations and reach

To facilitate comparability, only studies from one research department were included, where all researchers had a background in health services research and had collaborated in research and/or teaching in different compositions beforehand. The Heidelberg master’s degree program in health services research and implementation science connected all researchers participating in this study: Some were teachers, of whom all taught research methods and some were (former) students; some were both. Therefore, a common (although not uniform) understanding and practice of qualitative analysis can be assumed. All interview studies explored health care practice, used semi-structured interviews in German and had a comparable approach to data analysis using the same QDA-Software. All studies included interviews with one participant per interview, mostly health care providers. While homogeneity in these regards improves comparability, applicability of results to studies with other study populations, methods of data analysis, order of interviews analyzed or use of concepts and theories has to be examined.

Researchers differed in professional and disciplinary backgrounds, experience in qualitative research methods and degree of involvement in the studies. The interview studies varied regarding number and length of interviews, timeframe of conducting and analyzing interviews and overall project context. While these differences reflect a common reality in health services research, they limit reach and further data analysis. Therefore, echoing previous studies [15], further investigations are needed, e. g. using comparable data sets and research teams.

CIR over the course of interview analysis was accessed by descriptive analysis and should be interpreted with caution. Larger data sets are needed to test trends statistically.

Estimating timescales in qualitative analysis

Our study addressed focused coding. To estimate the time required for qualitative analysis, further aspects have to be considered: In our experience, preceding steps of familiarization, open coding and first analytical memos take at least two to three times the interview duration. Subsequent consolidation of analytical themes usually continues in write-up and could take a few weeks. In all studies, a combined inductive and deductive approach was used that included focused coding as a definable analytical step. However, repetition and overlapping of analytical steps is common in qualitative research [9] [30] and might be more prominent in other approaches (e. g. Grounded Theory). In addition, within the included studies, research objectives and therefore qualitative analysis was targeted to meet the scope of one to two research papers. More time would be needed to meet more comprehensive research objectives.

Our findings suggest that in estimating the required time for qualitative analysis, researcher and data characteristics have to be considered: Qualitative research experience, degree of involvement in the research project and proficiency in concept and theories as well as richness of data and composition of participant groups might influence time expenditure of coding. Previous research highlighted research experience as a relevant factor [13]. As our findings suggest, project characteristics such as scope of the rtesearch question, timeline and peer feedback might be further influencing factors. While our results indicate that coding becomes faster over time, there are limitations to increasing speed as qualitative analysis is a highly concentrated activity that cannot be carried out indefinitely at one time.

Our study was based on interview transcripts. While omitting transcription has been described as a major factor in quickening data analysis [4], this factor might lose significance with the increasing availability and precision of automatic speech recognition software and the use of AI in transcription [12]. At the same time, some studies found improvement of accuracy and richness of interpretation when coding directly from recordings, especially when used by experienced researchers [10] [31] [32] [33]. Another study pointed to the benefits of integrating traditional and rapid qualitative analysis for intervention development in a transnational study [34].

Artificial intelligence and the limits of calculability

The potential of AI for automatic coding in qualitative analysis is increasingly discussed. Qualitative data analysis programs such as Atlas.ti and MAXQDA have already implemented AI features promising substantial reduction of total data analysis time. First explorations show that AI can make qualitative research more time efficient, especially in deductive approaches [35] [36]. AI can serve as a useful tool for enhancing researchers’ capabilities when applied within a larger analytical process, structured e. g. by explicit prompts [35] [36] [37]. Limitations lay in capturing more subtle and interpretive themes [36], the inherent bias of AI, influenced by hegemonial ideas prevalent in society [38] and questions of data protection. These factors should be considered in future research investigating the efficacy of AI-supported qualitative data analysis.

In order to assess the proclaimed benefits of AI, references for estimating time requirements and resources in qualitative research are required. However, from our study, no firm guidance can be derived at on how long qualitative research should take (at most) as research designs and research contexts differ: Research questions may be more or less exploratory, research interests may be more descriptive or more analytical, researchers may be more experienced or less experienced, project contexts may allow more or less focus on certain work packages. Our results could rather serve as a reference for project planning to avoid underestimation [8] [9] [10] [11] and to consider rigor and speed according to the specific objective and context of a particular study.

Conclusion

Reliable references are required for estimation of time and resources needed to conduct an empirical study. In any specific research project, however, this effort must be balanced against the objective of the analysis, including the desired level of accuracy, detail and depth. Qualitative data analysis, like all research endeavours, demands specific training, skills, endurance and adaptability. Effective allocation of resources is crucial, yet methodological rigour cannot be arbitrarily economized, given the multiple factors influencing empirical research. Our study emphasizes the importance of considering factors such as composition of data, researchers’ experience and degree of involvement when planning research objectives and designs within a specific timeframe. Further research is needed to specify how particular parameters such as nature of the study population, method of data analysis and use of concepts and theories affect coding in qualitative analysis.

Availability of data and materials

Data generated or analyzed during this study are included in this published article and its supplementary information files.

Authors’ contributions

CU, CA, RPD and MW conceived the idea for this study. CU developed the project and was responsible for concept und design. CA, SB, TF, NK, RPD and CU documented coding times providing data for this study, context information and ideas. CU drafted and prepared the manuscript, with critical input on data presentation and methods from CA and MW. CA, NK, RPD and MW provided regular input throughout the project. All authors reviewed and approved the final manuscript.

This article is part of the DNVF Special Issue “Health Care Research and Implementation”.

Conflict of Interest

The authors declare that they have no conflict of interest.

Acknowledgement

We would like to thank Dr. Katja Krug for critical feedback, especially on data presentation.

• References

• 1 Wensing M, Ullrich C. Description of health services research. In: Wensing M, Ullrich C, Hrsg. Foundations of health services research: Principles, Methods, and Topics. Cham: Springer International Publishing; 2023: 3-14
• 2 Allen LN, Azab H, Jonga R. et al. Rapid methods for identifying barriers and solutions to improve access to community health services: a scoping review protocol. BMJ Open 2023; 13: e066804
• 3 Vindrola-Padros C, Vindrola-Padros B. Quick and dirty? A systematic review of the use of rapid ethnographies in healthcare organisation and delivery. BMJ Quality & Safety 2018; 27: 321-330
• 4 Vindrola-Padros C, Johnson GA. Rapid techniques in qualitative research: A critical review of the literature. Qualitative Health Research 2020; 30: 1596-1604
• 5 Ramanadhan S, Revette AC, Lee RM. et al. Pragmatic approaches to analyzing qualitative data for implementation science: an introduction. Implementation Science Communications 2021; 2: 70
• 6 Watkins D. Rapid and Rigorous qualitative data analysis: The “RADaR” Technique for Applied Research. International Journal of Qualitative Methods 2017; 16 160940691771213
• 7 Burgess-Allen J, Owen-Smith V. Using mind mapping techniques for rapid qualitative data analysis in public participation processes. Health expectations : an international journal of public participation in health care and health policy 2010; 13: 406-415
• 8 Pope C, Ziebland S, Mays N. Qualitative research in health care. Analysing qualitative data. Bmj 2000; 320: 114-116
• 9 Ullrich C, Poß-Doering R. Qualitative methods in health services research. In: Wensing M, Ullrich C, Hrsg. Health services research: Principles, methods, and topics. Springer; 2023
• 10 Greenwood M, Kendrick T, Davies H. et al. Hearing voices: Comparing two methods for analysis of focus group data. Applied nursing research : ANR 2017; 35: 90-93
• 11 Rogers M. Coding Qualitative Data. In: Okoko JM, Tunison S, Walker KD, Hrsg. Varieties of qualitative research methods: Selected contextual perspectives. Cham: Springer International Publishing; 2023: 73-78
• 12 Wollin-Giering S, Hoffmann M, Höfting J. et al. Automatic transcription of English and German qualitative Interviews. Forum Qualitative Sozialforschung / Forum: Qualitative. Social Research 2024; 25
• 13 Krueger RAK, Casey MA. Focus Groups. A practical guide for applied research. Thousand Oaks: Sage; 2014
• 14 Gale RC, Wu J, Erhardt T. et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implementation Science 2019; 14: 11
• 15 Taylor B, Henshall C, Kenyon S. et al. Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open 2018; 8: e019993
• 16 Neal JW, Neal ZP, VanDyke E. et al. Expediting the analysis of qualitative data in evaluation: A Procedure for the Rapid Identification of Themes From Audio Recordings (RITA). American Journal of Evaluation 2015; 36: 118-132
• 17 Eaton K, Stritzke W, Ohan J. Using scribes in qualitative research as an alternative to transcription. Qualitative Report 2019; 24: 586-605
• 18 Mbuagbaw L, Lawson DO, Puljak L. et al. A tutorial on methodological studies: the what, when, how and why. BMC Medical Research Methodology 2020; 20: 226
• 19 Bossert J, Mahler C, Boltenhagen U. et al. Protocol for the process evaluation of a counselling intervention designed to educate cancer patients on complementary and integrative health care and promote interprofessional collaboration in this area (the CCC-Integrativ study. PLoS One 2022; 17: e0268091
• 20 Arnold C, Hennrich P, Koetsenruijter J. et al. Cooperation networks of ambulatory health care providers: exploration of mechanisms that influence coordination and uptake of recommended cardiovascular care (ExKoCare): a mixed-methods study protocol. BMC Family Practice 2020; 21: 168
• 21 Arnold C, Hennrich P, Wensing M. et al. Keeping up with evidence-based recommendations – A qualitative interview study with general practitioners on information-seeking behaviour in cardiovascular care. BMC Primary Care 2023; 118
• 22 Michels JD, Meis J, Sturm N. et al. Prevention of invasive ventilation (PRiVENT)—a prospective, mixed-methods interventional, multicentre study with a parallel comparison group: study protocol. BMC Health Services Research 2023; 23: 305
• 23 Keller S, Forstner J, Weis A. et al. Interprofessionelle Weaning-Boards und Weaning-Konsile für Langzeitbeatmungspatient*innen: Eine qualitative Studie zum wahrgenommenen Potenzial für die Patientenversorgung. Pneumologie 2023;
• 24 Litke N, Weis A, Koetsenruijter J. et al. Building resilience in German primary care practices: a qualitative study. BMC Primary Care 2022; 23: 221
• 25 Senft JD, Fleischhauer T, Frasch J. et al. Primary care disease management for venous leg ulceration-study protocol for the Ulcus Cruris Care [UCC] randomized controlled trial (DRKS00026126). Trials 2022; 23: 60
• 26 Lawson DO, Puljak L, Pieper D. et al. Reporting of methodological studies in health research: a protocol for the development of the MethodologIcal STudy reportIng Checklist (MISTIC. BMJ Open 2020; 10: e040478
• 27 Khalil H, Munn Z. Guidance on conducting methodological studies – an overview. Current Opinion in Epidemiology and Public Health. 2023 2.
• 28 Braun V, Clarke V, Hayfield N. et al. Thematic Analysis. In: Liamputtong P, Hrsg. Handbook of Research Methods in Health Social Sciences. Singapore: Springer Singapore; 2019: 843-860
• 29 Gale NK, Heath G, Cameron E. et al. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Medical Research Methodology 2013; 13: 117
• 30 Emerson RM, Fretz RI, Shaw LL. Writing ethnographic fieldnotes. [2nd publ.]. Aufl. Chicago [u. a.]: Univ. of Chicago Pr.; 2011
• 31 Gravois T, Rosenfield S, Greenberg B. Establishing reliability for coding implementation concerns of school-based teams from audiotapes. Evaluation review 1992; 16: 562-569
• 32 Markle DT, West RE, Rich PJ. Beyond transcription: technology, change, and refinement of method. Forum Qualitative Sozialforschung / Forum: Qualitative. Social Research 2011; 12
• 33 Nevedal AL, Reardon CM, Opra Widerquist MA. et al. Rapid versus traditional qualitative analysis using the Consolidated Framework for Implementation Research (CFIR). Implementation Science 2021; 16: 67
• 34 Suchman L, Omoluabi E, Kramer J. et al. Analyzing fast and slow: Combining traditional and rapid qualitative analysis to meet multiple objectives of a complex transnational study. Front Sociol 2023; 8: 961202
• 35 Siiman LA, Rannastu-Avalos M, Pöysä-Tarhonen J. et al. Opportunities and challenges for AI-assisted qualitative data analysis: An example from collaborative problem-solving discourse data. In: Huang Y-M, Rocha T eds, Innovative technologies and learning. Cham: Springer Nature Switzerland; 2023: 87-96
• 36 Morgan DL. Exploring the use of artificial intelligence for qualitative data analysis: The case of ChatGPT. International Journal of Qualitative Methods 2023; 22 16094069231211248
• 37 Hamilton L, Elliott D, Quick A. et al. Exploring the use of AI in qualitative analysis: A comparative study of guaranteed income data. International Journal of Qualitative Methods 2023; 22 16094069231201504
• 38 Anis S, French JA. Efficient, explicatory, and equitable: Why qualitative researchers should embrace AI, but cautiously. Business & Society 2023; 62: 1139-1144

Publication History

Received: 12 July 2024

Accepted after revision: 13 December 2024

Article published online:
14 April 2025