Examining the Neurocognitive Validity of Commercially Available , Smartphone-Based Puzzle Games

Cognitive assessment typically involves assessing a person’s cognitive performance in unfamiliar and arguably unnatural clinical surroundings. User-centred approaches to assessment and monitoring, driven by issues such as enjoyability and familiarity, are largely absent. Everyday technologies, for example, smartphones represent an opportunity to obtain an objective assessment of a person’s cognitive capabilities in a non-threatening, discreet and familiar way, e.g. by everyday puzzle games undertaken as a leisure activity at home. We examined the strength of relationships that exist between performance on common puzzle games and standard measures of neuropsychological performance. Twenty-nine participants, aged 50 65 years, completed a comprehensive neuropsychological test battery and played three smartphone-based puzzle games in triplicate: a picture puzzle [Matches Plus], a word puzzle [Jumbline] and a number puzzle [Sudoku]. As anticipated, a priori, significant correlations were observed between scores on a picture puzzle and visual memory test (r = 0.49; p = 0.007); a word puzzle and estimated verbal IQ (r = 0.53; p = 0.003) and verbal learning (r = 0.30; p = 0.039) tests; and a number puzzle and reasoning/problem solving test (r = 0.42; p = 0.023). Further analyses making allowance for multiple comparisons identified a significant unanticipated correlation (r = 0.49; p = 0.007) between number puzzle scores and a measure of nonverbal working memory. Performance on these smartphone-based games was indicative of relative cognitive ability across several cognitive domains at a fixed time point. Smartphone-based, everyday puzzle games may offer a valid, portable measure of assessing and monitoring cognition in older adults.


Introduction
Current methods for assessing and monitoring Alzheimer's disease (AD) can be lengthy and obtrusive and rely upon testing cognitive performance at fixed time-points, often in arguably unnatural clinical settings.The majority of assessment methodologies are developed with neurocognitive assessment principles as the starting point.
Clinical assessment and diagnosis of dementia in the UK is based upon a clinical interview and assessment of activities of daily living, often involving the input of a relative or friend; the administration of standardised assessment scales to assess the severity of impairment in the individual; and a physical examination and/or brain scan where considered appropriate by the physician.The most commonly used assessment scale is the 30-item Mini-Mental State Examination [MMSE] (Folstein et al., 1975).The MMSE includes key domains of cognitive functioning: orientation, registration, attention, recall, visuospatial capabilities and language delivered in a structured setting.Patient monitoring usually takes the form of follow-up clinic appointments with the physician or nurse, including further clinical interviews and formal cognitive assessment to identify change.
Interest is growing in new assessment methods that combine assessment and monitoring and which reduce the need for multiple, fixed time-point clinic visits while offering a more convenient, home-based alternative.Such user-centred approaches to assessment and monitoring, driven by issues such as enjoyability, familiarity, convenience and availability, are largely absent.
Information and communication technology (ICT), particularly the smartphone, represents an opportunity to obtain an objective assessment of a person's cognitive capabilities in a non-threatening, discreet and familiar way, e.g. by everyday puzzle games undertaken as a leisure activity.The mobile phone is an accessible, portable, everyday technology with 77% uptake among older adults in the UK, and smartphone market penetration is rising (Ofcom, 2011).If performance on everyday puzzles maps to scores attained on conventional neuropsychological tests, then smartphone-based puzzles may offer a valid portable measure of cognition.To investigate this idea further, we sought to examine the strength of relationships that exist between performance on three widely available puzzle games and standard measures of neuropsychological performance.
Twenty Neurocognitive tests were selected on the basis of having limited floor and ceiling effects, high tolerability for participants and applicability in clinical trials (Nuechterlein et al., 2008).
The three games were selected on the basis of having good face validity with regards to verbal memory [Jumbline], problem solving [Sudoku] and visual memory [Matches Plus], and satisfactory user ratings (minimum 3/5 stars) on the online "App Store".Jumbline is a word puzzle that involves finding as many words as possible within a set of given letters.Each level of the game presents a new set of letters and players work against a timer to try to complete each level in order to progress to the next.Sudoku is a number-placement puzzle that involves inserting numbers into a partially filled 9 × 9 grid.The player's objective is to complete the grid so that the digits 1 to 9 are contained within each row, each column and each 3 × 3 subgrid without duplication.Matches Plus is a matching game that involves matching pairs of identical symbols as quickly as possible.Players work against a timer to try to match all pairs to progress to the next level.
Based on game characteristics, we anticipated that the following would show moderate to large correlations: 1) Word game [Jumbline]  The relationships between puzzle game and neuropsychological test performance are reported in Table 1.In line with a priori predictions, significant correlations were observed between: word game scores and performance on the National Adult Reading Test [NART] measure of reading ability/estimated IQ and the Hopkins Verbal Learning Test™-Revised [HVLT-R™] measure of verbal memory; number game scores and the Neuropsychological Assessment Battery ® [NAB ® ] test of reasoning and problem solving; and picture game scores and the Brief Visuospatial Memory Test-Revised [BVMT-R™] measure of visual memory.Post-hoc exploratory analyses at the 1% significance level identified an additional, significant moderate correlation between performance on the number game and the Wechsler Memory Scale ® -Third Edition [WMS ® -III] measure of visuospatial working memory.
To our knowledge, this is the first study to examine the utility of everyday, smartphone-based puzzles for validly assessing cognitive abilities in this age group.Of note, in this middleaged, cognitively intact cohort, we were able to identify moderately sized associations between conventional cognitive tests and smartphone-based games in keeping with their face validity.Further work will explore the ability of smartphone-based puzzles to detect cognitive impairment, as well as the level of impairment that might limit smartphone use.
scores and tests of verbal memory; 2 Number game [Sudoku] scores and tests of problem solving and logical reasoning; 3) Picture game [Matches Plus] scores and tests of visual memory.

Table 1 .
Relationship between puzzle game scores and neurocognitive test performance.