Machine learning for automatic scoring of cognitive tests

SBI contributors

Project description

Cube and figure drawing tasks are widely used in dementia screening to assess cognitive function, and SHARE has included three such tasks from the Addenbrooke’s Cognitive Examination III in its cognition module since Wave 8. In SHARE, drawings are scored by face-to-face interviewers rather than trained clinicians, which can reduce consistency and increase the risk of errors. Automated scoring using deep learning offers a way to improve reliability and data quality by reducing subjectivity in scoring.

Researchers have trained convolutional neural networks on SHARE Wave 8 drawings, using both interviewer scores and independently developed “ground truth” annotations. These models are designed to classify drawings as correct, partially correct, or incorrect, while addressing challenges such as noisy labels and limited training data.

To improve classification accuracy, curriculum learning strategies are being explored. Inspired by human learning, this approach presents simpler samples first and progressively introduces more complex or ambiguous ones. Methods for determining sample difficulty include annotator agreement and training loss from a reference model, while pacing strategies range from initially excluding the ‘partially correct’ class to staged multi-class introduction. These approaches aim to enhance the reliability of cognitive assessment in SHARE and demonstrate the potential of deep learning in large-scale survey research.

Status

Active

Selected publications

N/A