Qualitative pretesting and item development
SBI contributors
Dr. Arne Bethmann, Charlotte Hunsicker
Project description
Qualitative Pretest Interviews (QPIs) are in-depth conversations with a small number of respondents that explore how survey questions are understood and interpreted. QPIs reveal reasoning behind answers, identify misunderstandings, and flag ambiguous or confusing wording. This approach is especially important in large-scale, cross-national surveys like SHARE, where differences in language, culture, and context can affect question interpretation.
Experience in SHARE shows the value of QPIs across three phases: conducting interviews, analysing results, and implementing findings. Trained interviewers with survey and subject-matter expertise are essential for generating meaningful pretest data. Analysis must balance interpretive depth with the practical demands of large, complex surveys, and SHARE has explored a range of approaches that vary in depth and complexity to make optimal use of resources.
A coding scheme has been developed to streamline QPI analysis and is currently being tested with actual interview material. Insights from QPIs are then used to make concrete improvements in survey instruments, ensuring that respondents’ feedback directly informs question design. This research demonstrates how qualitative methods can be integrated into the development of standardised surveys and provides practical guidance for improving cross-national survey methodology.
Status
Active
Selected publications
zur Kammer, K., Schütz, J., Hudelmayer, A., Hunsicker, C., Bethmann, A. (2025). Measuring Computer Skills of Older People in SHARE: Evaluation of the item IT003_PC_Skills (IT module) using qualitative pretest interviews. SHARE Working Paper Series 98-2025.
zur Kammer, K., Hudelmayer, A., Schütz, J. (2025). „Ich kann alles Mögliche, aber ins
Internet kann ich nicht – Doch in Google komme ich rein“, Erkenntnisse aus Qualitativen Pretest Interviews mit Offliner:innen zum IT-Modul in SHARE und Implikationen für die Umfrageforschung. BZPD Working Paper Series 10/2025. Kempten. https://doi.org/10.60785/OPUS-2997
Selected presentations
Hunsicker, C., Bethmann, A., Buschle, C., Reiter, H. (2025). QPIs in Large Scale Surveys – Lessons learned. 11th Conference of the European Survey Research Association (ESRA), Utrecht, The Netherlands, Zenodo. https://doi.org/10.5281/zenodo.16753387
Hunsicker, C., Bethmann, A., Buschle, C., Reiter, H. (2023). In-house pretesting with Qualitative Pretest Interviews (QPI) – the example of SHARE. 10th Conference of the European Survey Research Association (ESRA), Milan, Italy, Zenodo. https://doi.org/10.5281/zenodo.8177576