Reproducibility of Test-Retest Cortical Evoked Responses in Patients with Focal Epilepsy ()
1. Introduction
Reliability refers to how similar, or reproducible, measurements are when repeated under the same conditions (test-retest). Test-retest reliability of cortical evoked responses has been found to be relatively poor in a variety of neurological disorders, including Fragile X syndrome [1], autism spectrum disorder [2] [3], and brain injury [4], suggesting that neuropathologic conditions are associated with neural instability. This finding has been referred to as the “neural unreliability thesis” [5] based on the hypothesis that trial-to-trial variability may exist for evoked sensory-neural responses within a single individual. However, other studies have refuted this hypothesis showing robust reliability compared to neurotypical controls [5] and over multiple time-points in individuals with neuropathologic conditions [6].
Seizure disorders are associated with increased neuronal hyperexcitability, potentially resulting in unstable neural responses. Moreover, reductions in anti-seizure medications for purposes of localizing seizures during scalp electroencephalography (EEG) monitoring could increase hyperexcitability, alter seizure propagation patterns [7], and further reduce test-retest reliability. Similarly, the location of a seizure focus, e.g. left hemisphere versus right hemisphere, temporal lobe versus extratemporal lobe, neocortical versus mesial, may also impact test-retest reliability. Although cortical evoked responses in individuals with intractable seizure disorders are thought to be unreliable due to increased neuronal hyperexcitability, this has not been investigated directly. Determining the reliability of evoked response recordings in this patient population has both clinical and research relevance. Planning for epilepsy surgery often involves recording over multiple days and assumes that the neural signals are reliable and reproducible, otherwise localization of a seizure focus could be inaccurate. Moreover, if cortical evoked responses are found to be reliable, they could ultimately be used to validate or identify regions of cortical dysfunction in individuals with focal epilepsy.
The goal of this exploratory study was to examine the short-term reliability of cortical auditory evoked responses recorded over multiple days from patients undergoing whole-head scalp EEG monitoring to assess their candidacy for surgical treatment of intractable focal seizures. In addition to evaluating test-retest reliability, we provide estimates of response variability for future studies.
2. Methods
2.1. Participants
Test-retest recordings were analyzed from 26 patients with focal epilepsy (12 female, 24 right-handed, mean age 35 years). A total of 28 patients were originally recruited for the study, however, two were excluded based on the poor quality of their recordings (see below). All patients were referred for continuous scalp EEG monitoring to localize their seizure onset for possible epilepsy surgery planning. Inclusion criteria were antiseizure medication-resistant focal epilepsy based on review of prior EEG recordings, neuroimaging, and/or clinical semiology and no history of hearing, language, or cognitive impairment. Normal hearing was confirmed by otoacoustic emissions screening (500 - 4000 Hz). Thirteen patients had seizures originating from the left hemisphere (LH seizures), although one patient had interictal left hemisphere slowing without a clear ictal pattern during focal aware seizures in the setting of left frontal cavernoma); thirteen had seizures originating from the right hemisphere (RH seizures). Mean age of seizure onset for all patients was 22 years. Patients’ antiseizure medications were reduced or discontinued during their admission to the Johns Hopkins Epilepsy Monitoring Unit (EMU) when clinically necessary. All patients gave informed written consent for participation in compliance with Johns Hopkins Medicine Institutional Review Board requirements.
2.2. Test-Retest Stimuli and Experimental Paradigm
The auditory stimuli were single-frequency, steady-state tones (1000 Hz, 1200 Hz) of 250 ms duration (5-ms rise/fall; NCH Tone Generator, Greenwood Village, CO). Stimuli were presented sequentially at inter-stimulus intervals of 1130 ms (offset-to-onset) in a 300-trial passive auditory oddball paradigm comprising a series of 1000 Hz tones (82%, N = 246 trials) interspersed infrequently and pseudo-randomly (e.g. non-consecutively) by 1200 Hz tones (18%, N = 54 trials). Patients were instructed to ignore the auditory stimuli and attend to a silent animated video. Stimuli were presented binaurally through insert earphones (ER-2, Etymotic Research, Elk Grove Village, IL) at a comfortable listening level (~40 dB SL). The oddball paradigm was adapted from a prior intracranial recording study [8] that did not include patients from the current study. We used a passive listening paradigm to avoid potential differences in attentional state across sessions and patients. The experimental paradigm was implemented using a portable TDT RZ6 System (Tucker-Davis Technologies, Alachua, FL) and was the same for both the test and retest sessions.
2.3. Test-Retest Auditory Recordings
Test-retest recordings were performed during each patient’s 4 - 8 day EMU admission (mean 5 days). At the time of admission, standard 10-mm Ag/AgCl electrodes were affixed to scalp with a semi-permanent adhesive (Collodion) and remained in place for the duration of the monitoring stay. Electrodes were placed based on the international 10-10 system and included midline (Fz, Cz, and Oz) and bilateral temporal lobe coverage. The total number of electrodes ranged from 32 - 74 (mean 48 electrodes). Continuous EEG recordings were acquired using a 128-channel Nihon Kohden system (Tokyo, Japan). The EEG signals were amplified (5 × 100) and recorded digitally using a referential montage, 1000 Hz A/D signal sampling, and a bandpass filter of 0.03 - 100 Hz. Electrode impedances were maintained below 5 kΩ. Stimulus onset markers were recorded simultaneously to separate EEG marker channels. Test-retest recordings were performed at intervals of one to six days (mean 2 days). The duration of each session was approximately seven minutes. Prior to initiating the recordings, each patient’s EEG record was inspected visually by an epileptologist [MCC] for the presence of interictal or ictal epileptiform activity or excessive multi-channel artifact. The test-retest recordings were saved in European Data Format (EDF) for offline analysis.
2.4. Signal Pre-Processing
Signal processing was performed offline using Matlab functions (Mathworks, Natick, MA, R2017b) and the EEGLAB toolbox [9]. Channels with excessive noise or artifact were excluded based on visual inspection and automatic channel rejection, leaving an average of 24 remaining channels in each recording montage. Test-retest recordings were low-pass filtered at 40 Hz and high-pass filtered at 0.5 Hz using a finite impulse response filter (Blackman window). High-pass filtering rather than baseline correction was used to reduce potential effects of pre-stimulus voltage differences on post-stimulus test and retest signals. Potential spurious effects from high-pass filtering were ruled out by re-computing latency and amplitude measurements for the early cortical auditory evoked N1 response based on a 0.1 Hz filter. Latency values were largely unchanged and less than 5% of the amplitude measurements differed. This suggests that any signal distortion resulting from high-pass filtering had negligible effects, consistent with prior reports that high-pass filtering mainly affects later cortical potentials, e.g. P300, N400, P600.
The continuous EEG signals were segmented into epochs (trials) using a 400-ms pre-stimulus to 1000 ms post-stimulus window. Individual trials containing artifact, including motion-related or epileptiform activity were excluded based on visual inspection and semi-automatic artifact rejection (EEGLAB). Trials with ocular or cardiac artifact were identified separately by independent component analysis [9]. Due to excessive artifact (>20% trials), EEG data from two participants (both with left hemisphere seizure onset patterns) were excluded from the analysis. Test and retest recordings from the remaining 26 participants (Table 1) comprised a minimum of 197 trials after trial rejection. The recordings were re-referenced to the vertex electrode (Cz) for within- and between-hemisphere comparisons of event-related potential (ERP) measurements.
2.5. Cortical Auditory Evoked Responses
Auditory evoked responses were computed separately for each electrode, session and stimulus by averaging across trials in the time-domain. Based on the relatively small number of 1200-Hz tone trials (N = 54), all evoked response measurements for test-retest analysis were computed from the 1000-Hz tone (N = 246) evoked response waveforms. Visual inspection of the single-channel 1000-Hz response waveforms identified prominent early (<250 ms post-stimulus) responses at posterior temporal lobe sites bilaterally across all patients and sessions. The four main electrodes sampling temporal lobe activity selected for
![]()
Table 1. Participant seizure lateralization and localization.
LH = left hemisphere; RH = right hemisphere; TL = temporal lobe; FL = frontal lobe, PL = parietal lobe; OL = occipital lobe; - = unknown (neocortical vs. mesial onset); *LH = Left hemisphere slowing and sharp transients noted with no definite ictal correlate on EEG during focal aware seizures, multifocal cavernous angiomas present including isolated left frontal lobe suspected to be underlying seizure focus.
test-retest analysis were: TP7, TP8, TP9, and TP10. For each patient and session, peak response latency and amplitude measurements (base-to-peak) were computed for the tri-phasic P1-N1-P2 components of the waveform in the 0 - 250 ms post-stimulus window. Waveform measurements were made by one rater (MR) and confirmed independently by a second rater (DBR) based on a randomly selected subset of 15 waveforms. Using a two-way mixed effect model, the intra-class correlation coefficient of 0.87 with a 95% CI [0.65, 0.96] indicated good-excellent inter-rater reliability [10]. The P1 was identified as the largest positive deflection between 30 - 70 ms post-stimulus; the N1 was the largest negativity following the P1 and occurring between 70 - 130 ms; and the P2 was the next largest positive deflection occurring between 120 - 220 ms. To account for the P1-N1-P2 polarity reversals observed at more inferior temporal lobe sites and after re-referencing, all amplitude measurements were represented as absolute values. For bi-modal or flat peaks, the mid-point latency and amplitude were used. For three patients, the P1 component was not identifiable in one or both recordings.
2.6. Test-Retest Analyses
For each participant, the same set of electrodes was used for test-retest comparisons. When one electrode was excluded from either session due to artifact, recordings were analyzed from a neighboring electrode for both test and retest. For test-retest analyses, we evaluated the N1 response as it was clearly visible across all patients and sessions. Test-retest reliability was assessed in two ways: by Pearson correlation coefficients and cross-correlation coefficients. We used Pearson correlation coefficients to assess the reliability of the single value peak latency and amplitude measurements. To assess the reproducibility of the overall post-stimulus response envelope (shape), we calculated the normalized cross-correlation function between test and retest evoked response waveforms using a sliding 1-ms window with a −100 ms to 100 ms lag (Matlab xcorr function). The maximum absolute value of the output, referred to as the cross-correlation coefficient, was Fisher z-transformed for comparisons across patients and sites.
To identify hemisphere-specific differences, we compared test-retest estimates for participants with left hemisphere seizure foci versus right hemisphere seizure foci using a two-tailed Mann-Whitney U test. We chose a non-parametric test because the group variances were found to be unequal (p < 0.001, F-test). To account for potential dependences between groups (repeated measures), we also computed a two-tailed Wilcoxon signed Rank test. Additional within-group comparisons were performed to determine whether test-retest reliability differed for the hemisphere containing the seizure focus versus the contralateral hemisphere by comparing across electrodes within each hemisphere (Friedman test, Kruskal-Wallis test) and by pairwise comparisons of individual electrodes (Wilcoxon Signed Rank test). A Bonferroni correction was applied to correct for multiple comparisons.
3. Results
For the test session, we analyzed recordings from a total of 686 electrodes across all 26 participants (mean 26; range 21 - 35). For the retest session, recordings from 648 electrodes were analyzed (mean 25; range 20 - 37). Grand mean auditory evoked response waveforms, computed by averaging across participants by electrode and session, are shown in Figure 1. Test-retest comparisons of waveform measurements (latency, amplitude) were based on four posterior temporal lobe electrodes (TP7, TP8, TP9, TP10) where evoked N1 responses were identifiable across all patients and sessions. Test-retest N1 latencies and amplitudes, computed for each patient by averaging across the four sites, are shown in Figure 2. The corresponding test-retest waveform plots are shown for a subset of patients in Figure 3. We first present test-retest results for all patients combined
![]()
Figure 1. Grand mean auditory evoked response waveforms, computed by averaging across participants by electrode and session during test (Blue solid) and retest (Red dashed) sessions are depicted using an international 10-10 electrode spacing headmap (top). Maximum amplitudes are seen in the temporal regions (bottom) on the right (even) and left (odd) and amplitudes are nearly identical for test-retest sessions and when comparing right and left.
![]()
Figure 2. Test-retest cortical auditory evoked N1 response (a) latencies and (b) amplitudes, computed for each patient by averaging across the four temporal lobe sites shown in Figure 1 (TP7, TP8, TP9 and TP10). The red diamond denotes the average (a) latency and (b) amplitude across all 26 patients. Red dashed lines in each plot denote standard deviation: horizontal for test and vertical for retest.
(population-level) and then for each seizure lateralization (right versus left) group separately (group-level). Within-group analyses by hemisphere are also presented (individual-level).
3.1. Test-Retest: Population-Level
N1 Latency. Comparing across participants and sites, the mean N1 latency for the test session was 92.79 ms ± 9.63. For retest, the mean N1 latency was 92.19 ms ± 8.05. The mean difference in test-retest latencies was 5.13 ms ± 4.32.
![]()
Figure 3. Trial-averaged test-retest waveform plots are shown for a subset of participants showing test-retest reproducibility by visual inspection and cross-correlations which account for both N1 latency and amplitude. For the test-retest waveform plots on the left, waveforms in black denote test and waveforms in red denote retest. These plots were selected to demonstrate that while across-subject waveforms were different by visual inspection, they yielded a high cross-correlation coefficient of rcc = 0.88 (p < 0.001), indicating high within-subject reproducibility of N1 waveform morphology (shape) across sessions.
Individual pairwise comparisons of test-retest latencies revealed no significant differences (p = 0.45; paired t-test). Similarly, we observed no associations or linear trends, increasing or decreasing, in latency values across sessions. The Pearson correlation coefficient (r = 0.61) indicated moderate test-retest association.
N1 Amplitude. The mean N1 amplitude for the test session was 3.55 µV ± 1.67 and for retest was 4.06 µV ± 2.30. The mean difference in test-retest amplitudes was 1.01 µV ± 0.66. Pair-wise comparisons showed no significant differences in individual test-retest amplitude values (p < 0.68, paired t-test). However, the Pearson correlation coefficient (r = 0.86) indicated a strong trend towards larger retest amplitudes.
N1 Latency and Amplitude. Test-retest comparisons using cross-correlation to account for both N1 latency and amplitude yielded a coefficient of rcc = 0.88 (p < 0.001), indicating high within-subject reproducibility of N1 waveform morphology (shape) across sessions. This is consistent with visual inspection of the test-retest waveforms that show good reproducibility within subjects despite considerable variability in waveform morphologies between patients (Figure 3).
3.2. Test-Retest: Comparison by Seizure Lateralization
We compared test-retest reliability for participant with left hemisphere seizures versus right hemisphere seizures. Because N1 amplitude test-retest variances across the two groups were found to be unequal based on the F-test (F = 0.463, p = 0.007), we used the non-parametric Mann-Whitney U test. Results showed no significant test-retest differences in N1 latencies or amplitudes for either group (p > 0.05, in all cases). Pearson correlation coefficients calculated separately for each seizure lateralization group indicated strong test-retest associations for N1 amplitudes in both the left hemisphere (r = 0.92) and right hemisphere (r = 0.85) seizure lateralization groups. Conversely, N1 latencies showed only moderate test-retest correlation for both left hemisphere (r = 0.66) and right hemisphere (r = 0.56) seizure lateralization groups. These group results are largely consistent with the population-level results (see above).
3.3. Test-Retest: Within Group Comparison
To determine whether test-retest reliability differed between the hemisphere containing the seizure focus and the contralateral hemisphere in each of the two seizure lateralization groups, we used the non-parametric Friedman test with the post-hoc Conover test. No significant hemisphere asymmetries in N1 test-retest latencies were found for either the left hemisphere or right hemisphere seizure lateralization groups (F = 2.19, p = 0.10 and F = 2.27, p = 0.09). However, the left hemisphere seizure lateralization group showed significantly smaller N1 amplitudes in the left hemisphere (containing the seizure focus) compared with the right (non-seizure focus) hemisphere on both test and retest after Bonferroni correction (F = 5.11, p = 0.003). This hemisphere asymmetry in N1 amplitudes was not evident in the right hemisphere seizure lateralization group (F = 0.18, p = 0.91).
3.4. Impact of Prior Epilepsy Surgery on N1 Amplitudes
Four participants in the left hemisphere group had prior focal resections and two in the right hemisphere group. When analyses were repeated excluding participants with prior resections from the left hemisphere group, N1 amplitudes were no longer significantly smaller in the left hemisphere compared to the right.
3.5. Impact of Seizure Localization on N1 Amplitudes
Subgroup analyses were performed for participants with right or left neocortical versus mesial temporal lobe onset seizures and temporal versus extratemporal lobe onset seizures, excluding those participants in whom localization was uncertain and patients who received prior epilepsy surgery. Median and mean N1 amplitudes were significantly higher in participants with temporal lobe neocortical onset seizures compared to those with mesial temporal onset seizures when combining those with right and left seizure lateralization (p = 0.0001). There were no significant differences in N1 amplitudes between participants with left and right hemisphere extratemporal lobe neocortical onset seizures. There were significant differences in N1 amplitudes between participants with temporal lobe versus extratemporal lobe onset seizures with higher N1 amplitudes in the extratemporal lobe group (p = 0.0002). When comparing all participants with neocortical (temporal or extratemporal lobe onset) seizures to those with mesial temporal onset seizures, those with neocortical onset seizures had higher N1 amplitudes (p < 0.0001).
4. Discussion
In this study of 26 patients with focal epilepsy who underwent continuous video electroencephalography for epilepsy surgery planning, test-retest reliability of N1 cortical auditory response latencies and amplitudes was unexpectedly high at both the population and group levels. These findings challenge the view that neural responses are intrinsically unstable (unreliable) in patients with neuropathologic conditions [2] [3]. The mean test-retest variability was estimated to be 5 milliseconds for N1 latency and 1 µV for N1 amplitude. In patients with focal epilepsy, seizure lateralization was not associated with asymmetries in N1 latencies or amplitudes. An N1 amplitude asymmetry (right > left) in patients with focal seizures originating from the left hemisphere was initially observed but disappeared when patients with prior resections were excluded, suggesting that reduced left hemisphere tissue volume may have accounted for the smaller N1 amplitudes. This finding may be helpful in the selection of patients for future evoked potential studies and in the interpretation of results. Likewise, interpretation of cortical evoked potentials may be less reliable in individuals with history of prior focal cortical resection or other cortical lesions, and should be taken into consideration when interpreting neurophysiologic studies for clinical purposes such as future epilepsy surgery planning.
When considering the impact of these findings and comparing them to prior findings in the literature, the few evoked response studies that have examined test-retest reliability in patients with focal epilepsy have relied on intracranial recordings often from only one hemisphere: the hemisphere of seizure origin sampled based on clinical necessity [11] [12]. Some studies have found relatively poor test-retest reproducibility, and it is not known whether this reflects instability in the underlying neural activity due to seizures or other factors, including effects of intracranial electrode implantation such as cortical hyperexcitability and potential hemispheric asymmetries. There are a growing number of research studies based on clinical intracranial recordings and these findings are challenging to try to replicate given the unique clinical circumstances. This study showed that test-retest reliability of auditory evoked responses is robust over multiple days using scalp EEG recording, which is reassuring given the common practice of using recorded EEG data to lateralize and localize seizures in planning for intracranial monitoring or epilepsy surgery.
Of interest, patients with neocortical onset temporal lobe seizures had significantly larger N1 amplitudes than patients with mesial onset temporal lobe seizures. This finding was initially puzzling as dysfunction of the neuronal network causing neocortical seizures might be predicted to disrupt the phase-locked neural activity involved in generating evoked potentials resulting in smaller N1 amplitudes. In addition, patients with neocortical temporal lobe seizures had shorter N1 latencies than patients with mesial temporal lobe onset seizures, although this finding was not consistent across both test and retest sessions. In clinical practice, these observed differences could be used to help predict whether seizure onset zones within the temporal lobe are mesial or neocortical prior to intracranial monitoring if replicated in larger populations of patients with temporal lobe epilepsy.
This study has several limitations. Only two patients in the study were left-handed (one in each hemisphere group), so the effect of handedness could not be assessed. One participant was included who had clinical seizures with no ictal correlate although left hemisphere abnormalities on interictal EEG and multifocal lesions including a solitary left frontal cavernoma. There was considerable variability between which days of admission test and retest took place, based on availability and willingness of patients to participate in the study. There was therefore also variability in time from last seizure, antiseizure medications at the time of recordings, and tapering of antiseizure medication at the time of each testing session which may have affected results. Larger studies are needed to assess the impact of these variables.
In conclusion, cortical auditory evoked responses recorded from patients with medication-resistant focal seizures were highly reproducible on test-retest recordings. These findings support use of repeated-measures recordings with this patient population over multiple days for clinical and research studies. There may be potential strategies for using this method to validate seizure focus localization (mesial versus neocortical) if results are shown to be reproducible in a larger patient population.
Acknowledgements
The authors wish to acknowledge Dr. Robert Webber and Dr. Christophe Jouny for their assistance with data acquisition for this project.
NOTES
*Indicates authors who contributed equally to the manuscript.