Longitudinal Follow-Up of a Population with MCI: Predictive Value of the MIS Test with Delayed Recall for Progression to Dementia ()
1. Introduction
Early detection of dementia and the characterization of risk of progression stages are currently of great interest and of crucial importance for the implementation of preventive measures and treatment of the disease [1] . Alzheimer’s Disease (AD) is the most frequent cause of dementia and it is characterized by a slow and progressive neuropsychological decline. The presence of pre-symptomatic manifestations several years before its clinical diagnosis has been demonstrated [2] [3] . There is evidence showing that suffering from Mild Cognitive Impairment (MCI) poses a high risk of conversion to AD [4] . This clinical entity is characterized by changes in cognitive performance, reported by the patient or a relative, objective failure in one or more cognitive domains, with intact activities of daily living, and absence of dementia [5] .
Multiple longitudinal studies confirm a high percentage of patients with a diagnosis of MCI progress to dementia, with an annual conversion rate of 13%, while community study data indicate a conversion rate of 3% [6] [7] .
Other studies have shown that between 2.3% and 6.6% of older adults with a subjective memory complaint would convert to MCI or dementia every year [8] .
Episodic memory deficit appears to be one of the most reliable predictors of conversion. The combination of cognitive function assessments and neuroimaging predicts conversion in 80% of the cases [9] [10] [11] .
Since cognitive evaluation is a central pillar in the clinical investigation of MCI, it is paramount to determine the ability of specific memory tests to detect early changes which are predictive of progression to the disease [12] .
The research diagnostic criteria for Alzheimer’s Disease (AD) established by the International Working Group (IWG) has defined the neuropsychological profile of AD’s characteristic memory decline: objective evidence of significant episodic memory deterioration consisting of a recall deficit which does not significantly improve with cues after having controlled effective encoding of the information. It was observed that this characteristic correlates with the hippocampal volume and it is known as amnestic syndrome of hippocampal type [13] [14] . The test recommended by the IWG to evaluate this deficit is the Free and Cued Selective Reminding Test, which has the advantage of controlling an efficient semantic codification and facilitating the retrieval process using the same semantic cues [14] .
In turn, it has been verified that the most relevant cognitive measurements in memory tasks include a delayed recall (DR) phase, which seems to be an important cognitive marker in MCI detection. DR is the phase in amnestic processing that consists in reproducing a list of words stored after a certain period of time. Currently, this process by which a person can retrieve information is usually included in the Neuropsychological Evaluation and constitutes a predictive marker of AD in patients with MCI, as shown in various studies [11] [15] - [24] .
Higher scores in auditory verbal learning tasks seem to be “protective” regarding progression to dementia, as shown by a recent meta-analysis and a cohort study review which evaluated risk factors in subjects with MCI [25] .
These findings have implications in the dementia clinical field since they appear to confirm, to a certain extent, a continuum in the manifestations observed in patients, which would range from subjective memory complaint, to a subsequent diagnosis of MCI, to its probable progression to dementia.
Given the importance of the first visit in clinical practice, especially in the area of primary healthcare, it is vital to have cognitive screening instruments with predictive value for progression.
It should be noted that specialists or neuropsychology units are not always available to administer cognitive scales and tests, most of which are extensive and complex to register, and their results difficult to evaluate. In addition, imaging studies or biomarkers which may optimize the initial diagnosis are not readily available.
In this context, and considering its proven clinical usefulness, we have created the Río de la Plata Spanish version of the MIS (Memory Impairment Screen) and its guidelines for our population [26] whose verbal stimuli consist of medium tipicality words belonging to Spanish from Río de la Plata. This test was developed by Buschke in 1999 [27] and it was designed to provide better sensitivity and specificity to detect episodic memory disorders. It includes four free/facilitated recall items, which require learning using semantic cues. Its methodology improves encoding quality, thus enhancing retrieval capacity.
In order to optimize the sensitivity and specificity of the test for the detection of patients with MCI, we have added a delayed recall (DR) register of verbal stimuli to the original version. We will refer to this variation as MIS-DR, using the term MIS-IR (immediate recall) for the original version without delayed recall.
This modification optimizes the assessment of verbal episodic memory, enabling early detection of patients at risk of developing amnestic MCI (aMCI), which is considered an important contribution to the potential prevention and early treatment of the disease.
In a previous study [26] with 739 subjects, divided into 436 healthy controls and 303 with a diagnosis of aMCI, we verified the validity of MIS-DR in MCI detection. The comparison of the results obtained between the control population and the aMCI population using the receiver operating characteristic (ROC) curve yielded a value of 5.5 in the MIS with delayed recall, 97% of specificity and 76% of sensitivity. The softened ROC curve was also calculated, obtaining 94% for the area under the curve. The sensitivity value was 83% and specificity was 92%, with a positive predictive value (PPV) of 89% and a negative predictive value of 90%.
Building on this previous research work, and in order to verify the predictive value of MIS-DR for conversion to dementia, we performed a retrospective follow-up study with 502 patients with MCI diagnosis. We hypothesize that adding a delayed recall phase to MIS will enhance prediction of conversion to dementia.
The purpose of this work is to show the predictive value, sensitivity and specificity of the Memory Impairment Screen with delayed recall to predict conversion to dementia in a longitudinal study of a population of patients with a diagnosis of mild cognitive impairment.
2. Subjects and Methods
2.1. Retrospective Study
The study included 502 consecutive patients over 60 years of age, evaluated between January 2009 and July 2012, in the Older Adult Functional Evaluation Unit of Hospital Italiano de Buenos Aires, who sought consultation about cognitive complaints, with a clinical diagnosis of mild cognitive impairment (MCI).
Patients with a diagnosis of major depression, uncompensated sensory deficits (low vision, hypoacusia), epilepsy, and central nervous system tumors were excluded.
A comprehensive geriatric assessment (Older Adult Functional Evaluation) was performed on patients with cognitive complaint who were referred by their general practitioners. This assessment includes an evaluation of functional performance, emotional state and cognition through a detailed anamnesis and specific scales: ADL [28] , IADL [29] , Geriatric Depression Scale [30] , CDR (Clinical Dementia Rating Score) [31] , Folstein’s Mini Mental State Examination (MMSE) [32] , and cognitive assessment tests, which evaluate memory, language, attention, and visual-spatial and executive abilities. At this instance, the MIS in its original version with immediate recall (IR) and the delayed recall phase (DR) are administered to all patients.
Petersen’s criteria were used for the diagnosis of MCI [5] : 1) memory complaint, preferably corroborated by an informant, 2) evident amnestic decline in relation to age and level of education, confirmed by a formal test, 3) preserved general cognitive function, 4) intact activities of daily living, 5) without evidence of dementia.
In order to detect evolution, assessments were repeated every 12 to 24 months until July 2015.
The study was approved by the local institutional review board.
2.2. MIS-DR Test: Procedure
In the first step, the subject is presented with a poster containing four words belonging to different semantic categories, and is asked to read them aloud. The category learning process begins when the participant can associate each of the 4 semantic cues, randomly presented by the evaluator, with the corresponding word. For example, for the word “Bricklayer”, the patient is asked to “indicate which of the words in the poster is an Occupation”. The same procedure is followed with the 4 words. After a 2 - 3 minute interval, during which a non-semantic distracting task is given, such as counting from 0 to 20, saying the numbers from 20 to 0 by twos or the months of the year backwards, and repeating the task until the time has elapsed, free recall (Free R) of the items is asked, giving 5 seconds for each item, and waiting 10 seconds before moving on to the following step of facilitation. If any of the words is not retrieved through Free R, the same category cues used in the learning phase are used to facilitate retrieval (Fac. R). For example, if the subject did not recall the word Bricklayer, they are asked: “one of the words was an Occupation, which of the words was an occupation”? After 20 minutes of the immediate recall phase, during which functional evaluation is continued avoiding tasks which include verbal stimuli, the patient is asked to recall the 4 words again (DR) in the free and facilitated versions [26] .
Scoring: Scoring in this first stage (IR) is the same as in the original version (TOTAL MIS = [free recall × 2] + [facilitated recall]. Higher scores represent better memory performance.
The free recall score measures the level of spontaneous retrieval whereas facilitated recall measures episodic memory ability with category cues used during learning.
Two points are assigned for each word remembered in Free R and 1 point for each word recalled with category cues (Fac. R).
Considering that in Fac. R cues were selectively given for those verbal stimuli that were not retrieved in Free R, in the original version of the MIS it was assumed that the remaining stimuli would also be retrieved with the semantic cues. For this reason, Free R was multiplied by 2 in order to equate the scores. In this way, the range of scores obtained is distributed for Free R (0 - 8), Fac. R (0 - 4) and total recall (TR) (0 - 8) for MIS, IR. The same procedure was followed for Delayed Recall (MIS-DR).
2.3. Statistical Analysis
ROC curves were drawn for both MIS recall phases (IR and DR) to obtain the sensitivity and specificity of each cut-off point of the test, in order to predict progression to dementia.
The univariate and multivariate odds ratio (OR) was also calculated, adjusted by age, education level and vascular risk factors.
3. Results
Five hundred and two subjects with a diagnosis of mild cognitive impairment were included. The demographic characteristics of the population are shown in Table 1. Average age was 78 years. Seventy-four percent were women. Seventy-two percent of the population had received between 6 and 12 years of education.
During follow up, 144 participants developed dementia (28.6%, 95% CI 24.76 - 32.85).
![]()
Table 1. Characteristics of the study population.
n: number of patients; %: percentage; SD: standard deviation.
The average follow-up time was 47.9 months (SD 9).
The average time of progression to dementia was 23 months (SD 13.2).
About 66% of the sample had hypertension, with a low prevalence of type 2 diabetes and history of cerebrovascular event.
The cut-off point for the ROC curves was established below 6 for the MIS immediate recall (IR) and delayed recall (DR). Immediate recall shows a sensitivity of 45% (95% CI 36.33 - 53.94) and a specificity of 82% (95% CI 77.57 - 86.55), with a positive predictive value (PPV) of 50% (95% CI 43.64 - 56.72) and a negative predictive value (NPV) of 79% (95% CI 73.34 - 83.59) for prediction of progression to dementia. In turn, the delayed recall phase shows a sensitivity of 76% (95% CI 68.77 - 77.84) and a specificity of 56% (95% CI 50.44 - 61.58), with a PPV of 41% (95% CI 34.78 - 47.6) and a NPV of 85% (95% CI 80.53 - 91.11) (Table 2).
The area under the curve (AUC) was 0.67 (95% CI 0.62 - 0.72) for MIS IR and 0.74 (95 CI% 0.69 - 0.78) for DR.
Table 3 and Table 4 show sensitivity and specificity for each cut-off point of IR and DR of MIS. ROC curves graphs for IR and DR of MIS are shown in Figure 1 and Figure 2.
Table 5 shows the characteristics of the population according to their evolution. Patients who developed dementia were older and obtained a lower score in the MMSE and in both phases of the MIS test (IR and DR). There were no significant differences in terms of gender and education level or vascular risk factors.
Table 6 presents patient characteristics in relation to MIS-DR results. Patients in the group with a MIS-DR below 6 were older on average and obtained a lower
![]()
Table 2. Values of sensitivity (Se), specificity (Sp), positive predictive value (PPV) and negative predictive value (NPV) for MIS immediate and delayed recall, cut-off point below 6.
![]()
Table 3. MIS-IR: sensitivity and specificity values for each cut-off point.
![]()
Table 4. MIS-DR: sensitivity and specificity values for each cut-off point.
average score in the MMSE, MIS-IR and MIS-DR tests. There were no statistically significant differences in gender and education level between the two groups. Of those patients who obtained a MIS-DR score below 6, 41% developed dementia, in contrast to 14.53% of those with a score above or equal to 6, OR 4.09 (95% CI 2.64 - 6.34), p < 0.00001. After adjusting this value by level of education, an OR of 4.08 was obtained (95% CI 2.63 - 6.32), p < 0.00001, and after adjusting by level of education and age, the OR obtained was 3.71 (95% CI 2.37 - 5.81), p < 0.00001. The average time of progression to dementia for patients with a MIS-DR score below 6 was 21.34 months (SD 12.48), while for those with a
![]()
Table 5. Characteristics of the population according to their progression to dementia.
n: number of patients; %: percentage; SD: standard deviation.
![]()
Table 6. Population characteristics in relation to MIS-DR results.
n: number of patients; %: percentage; SD: standard deviation.
MIS-DR score above or equal to 6 was 28.5 (SD 13.88) months (p 0.002).
When comparing the predictive value of progression to dementia between the MMSE test with a cut-off point of 26 and the MIS-DR phase with the cut-off
![]()
Figure 1. Graph showing MIS-IR ROC curve.
![]()
Figure 2. Graph showing MIS-DR ROC curve.
point established in this work, it was observed that the MMSE OR was 2.36 (95% CI 1.54 - 3.61) and that of MIS-DR was 4.09 (95% CI 2.64 - 6.34).
4. Discussion
In the present research, we investigated the predictive value of conversion to dementia of a brief test of verbal episodic memory with delayed recall (MIS-DR) in a population of subjects with a diagnosis of mild cognitive impairment.
The conversion rate to dementia of the subjects with mild cognitive impairment was 28.6% in an average of two years. This is similar to the rate reported in previous studies conducted in memory clinics [6] [7] [23] [33] [34] [35] .
We consider that the cut-off point selected for MIS-DR (below 6) in this sample was the most adequate one to detect patients at risk of developing dementia, given the sensitivity and specificity of such value. A higher cut-off point would increase test sensitivity at the expense of reducing specificity, thus producing more false positives. In contrast, a lower cut-off point would reduce the rate of false positives but would significantly increase false negative rates, excluding too many patients at risk of developing dementia [26] .
In a review conducted by Gainotti et al. which studied the neuropsychological predictors of conversion from mild cognitive impairment to dementia, it was concluded that the more stringent the cut-off point of the memory test, the better the prediction of conversion to dementia [22] . We sought to detect the largest number of patients at risk of progression, given the clinical research context of the study.
For the MIS-DR cut-off point chosen, we obtained a sensitivity of 76% (95% CI 68.77 - 77.84) and a specificity of 56% (95% CI 50.44 - 61.58), comparable to other studies which evaluated the predictive value of conversion to dementia of different delayed recall memory tests. Perri et al. evaluated the predictive value of conversion to dementia of various tests of verbal and visual episodic memory with delayed recall, in patients with aMCI [36] . For the delay recall of a list of related words, they obtained a sensitivity of 77.2%. This means that of those patients who developed dementia, 77.2% had obtained a result below the cut-off point, compared to 41.4% in those who did not develop the disease, which is very similar to our results (see Table 5). In addition, they observed that the different tests assessed had a lowindividual specificity. However, when calculating a cumulative performance index among 4 tests with delayed recall, specificity reached 68.5%. We obtained a slightly lower specificity, but using only one memory test.
Similar to the previous results, Pozueta et al. report a sensitivity of 72.91% and a specificity of 70.31% for the CVLT-LDTR (California Verbal Learning Test-Long Delayed Total Recall) to predict conversion to Alzheimer’s disease dementia in patients with mild cognitive impairment, for the cut-off point chosen [19] .
Sano et al. reported relevant data in relation to the clinical value of using assessment instruments which include delayed recall tasks. They conclude that this benefit is associated with the diagnosis of MCI, and that including DR items in longitudinal studies of MCI patients optimizes detection of cognitive changes over time, especially in patients with high MMSE scores, such as our case [37] .
It should be noted that the OR value of MMSE for predicting conversion to dementia was significantly lower than that obtained for MIS-DR.
In relation to the variables studied, it should be mentioned that we found no association between vascular risk factors and risk of progression to dementia. This result is in line with other studies which did not find this association either in MCI patients [20] [38] .
We believe that the originality of this work lies in the fact that we used a short and easily administered memory test, which fulfils two essential conditions with amnestic deficit marker value. One of them is the inclusion of a learning process of a list of words with semantic cues, and the other, a delayed recall phase.
In addition, in a comparative analysis between the IR and DR phases of MIS, and in line with other studies mentioned above, we verified greater sensitivity of verbal episodic memory tasks that include a delayed recall phase as a predictive marker of progression to dementia.
However, we should point out that in our sample the probability of progression was 41%, which is similar to the values reported in previously mentioned studies. This requires the implementation of preventive measures and risk factors control in clinical follow-up. In spite of this, we believe that unnecessary alarming information should not be given to the patient and their family, since most subjects will not develop dementia. Instead healthy lifestyle habits should be promoted, related to nutritional factors, physical and cognitive activity and social interaction.
One of the limitations of this work is that the population studied belongs to a middle socio-economic class, who have the health insurance plan provided by Italian Hospital of Buenos Aires, and they may constitute a quite singular cohort given the homogeneity of the type of care they receive by their general practitioners, thus the results may not be extrapolated to other populations.
In conclusion, we believe that MIS-DR is a useful and valid test to detect episodic memory impairment and to identify patients at risk of progression to dementia.
Future work on this line of research will focus on establishing the characteristics of the type of dementia that patients develop with greater specificity, and on the identification of other early cognitive makers.
Acknowledgements
This research was carried out with the support of the Italian Hospital of Buenos Aires, Argentina.