Test-Retest Reliability of Sensory Evoked Potentials in Low-Functioning Autism: A Case Report (Case Report)
1. Introduction
Autism spectrum disorder (ASD) is a neurodevelopmental disorder characterized by impaired social interaction and communication, stereotyped behaviors, and restricted interests [1]. Sensory abnormalities, including atypical responses to auditory stimuli and impaired visual processing, are common in ASD and estimated to affect more than 60% of individuals [2] - [7]. Despite the high comorbidity, sensory abnormalities in ASD remain challenging to evaluate especially in individuals with low-functioning autism who are often unable to comply with behavioral testing. The classification of low-functioning autism (LFA) is associated with more severe ASD symptoms, including minimally verbal or non-verbal and inability to function independently in terms of daily living, and is also referred to as ASD-level 3 based on DSM-5 criteria [1]. LFA is one of the two widely used classifications based on autism severity, the other being high-functioning autism which refers to individuals who are moderately or highly verbal and require minimal or no support for daily living.
1.1. Sensory Evoked Potentials
Sensory evoked potentials provide objective, non-invasive electrophysiologic measures of sensory processing that could be useful for evaluating individuals with LFA. Cortical evoked responses to sensory stimuli can be recorded from the scalp and are known to correlate with behavioral detection thresholds [8] [9]. The utility of sensory evoked potentials is dependent on establishing reliability, namely they produce consistent results when the same individual is retested under the same conditions. Although evoked potentials have been shown to be highly reliable in test-retest studies of typically developing individuals [10] [11] [12]. reliability has not been established for individuals with LFA.
1.2. Neural Response Variability in ASD
Some studies have suggested that neural sensory responses are more variable in individuals with autism than in typically developing individuals [13] [14]. Increased intra-individual variability would be expected to reduce test-retest reliability and contribute to the sensory processing abnormalities commonly observed in ASD [4] [15].
Evidence for greater intra-individual variability in ASD is based mainly on electrophysiology and fMRI studies showing increased trial-to-trial variability in neural sensory responses. Visual electrophysiology studies reported greater trial-to-trial variability in amplitude for individuals with ASD as compared with typically developing individuals [13]. Similarly, fMRI studies of visual, auditory, and somatosensory processing showed greater trial-to-trial variability in ASD participants [14] [16]. Studies further suggest that increased intra-individual variabilty in ASD is specific to sensory processing as it localizes to sensory cortex [14]. is not evident in the ongoing background neural activity [14]. and is also present during passive tasks that do not require attention or behavioral responses [17] [18] [19].
Increased neural variability, also referred to as internal neural noise, in ASD has been attributed to a number of sources [15]. One explanation is an excess of connectivity in sensory cortex [20] [21]. Alternatively, abnormal neural synchrony, resulting from an imbalance of cortical excitation and inhibition, has been proposed [17] [18] [22]. It has also been suggested that the degree of intra-individual neural variability may be associated with severity of ASD symptoms [14]. Because individuals with LFA have more severe autism symptoms, they might be expected to have high intra-individual variability and, therefore, show poor test-retest reliability. Taken together, these studies underscore the need to measure reliability of sensory evoked potentials directly in individuals with LFA.
1.3. Evaluating Test-Retest Reliability in Low-Functioning Autism
Although electrophysiologic studies can be challenging for individuals with LFA, our experience and that of others [23] suggest that the likelihood of success can be increased by incorporating pre-study practice sessions, electrode tolerance training, and experimental tasks and stimuli that allow for shorter recording sessions. Test-retest reliability studies pose additional methodologic and logistic challenges, including ensuring that test-retest experimental conditions are the same, controlling for potential differences in behavioral performance, as well as scheduling logistics.
In this study, we investigated the feasibility of measuring reliability of sensory evoked potentials based on repeat recordings from a minimally verbal adult with LFA. Auditory and visual evoked potential recordings were acquired during a series of short (10 - 20 minutes) sessions repeated under the same experimental conditions over 2.5- and 6-month intervals. We tailored the test-retest paradigm to an individual with LFA by using a passive stimulus presentation paradigm with auditory and visual stimuli known to elicit robust responses. We further individualized the test-retest paradigm by incorporating photos of family and friends as visual distractors to enhance compliance during the auditory recordings and by modifying the duration of recording sessions and breaks based on caregiver feedback.
2. Case Report
2.1. Participant History
Our participant was a 32-year-old male diagnosed with regressive ASD at age 5 years. Pregnancy and birth history were unremarkable; early developmental milestones were met: sitting by 6 months, walking by 12 months, producing spontaneous, single-word utterances by 12 to 15 months. Based on parental report, language regression and decreased social interaction began at age 2.5 years. At age 3.5 years, he was diagnosed with pervasive developmental delay. Social interaction was limited to family members, and language abilities were rated at a 9-month-old level. Repetitive, stereotyped behaviors were observed, including hand wringing and toe walking, and he was reported to be intermittently unresponsive to sounds. His neurological evaluation was otherwise normal; gross motor skills were age-appropriate; EEG and MRI were normal. A hearing test was attempted using behavioral sound field testing. Results suggested largely normal hearing thresholds (≤25 dB) for at least one ear but were considered inconclusive due to non-compliance. The participant was subsequently enrolled in a speech-language therapy program in a therapeutic school.
At age 5 years, he was diagnosed with autism based on medical history, physician evaluation, and an Autism Diagnostic Observation Schedule (ADOS-2) score > 10 [24]. He was minimally verbal and required full support for daily living consistent with classification criteria for LFA [1]. Re-evaluation at age 10 years revealed decreased gross motor skills and minimal spontaneous speech (verbal) output. At age 12 years, he transitioned to home instruction. At the time of testing, he continued to receive home instruction supplemented by educational and community activities and was not on any medication.
The participant’s parents provided written consent for his participation prior to the start of the study in accordance with the Institutional Review Board (IRB) of the Johns Hopkins Medical Institutions. The IRB-approved protocol covers a number of different research paradigms, including evoked potential studies. The participant’s parents were given the opportunity to read through the consent form and ask questions of the investigators before signing the consent. The investigators also discussed the risks (minimal) and potential benefits of the study in accordance with IRB requirements.
2.2. Evoked Potential Recordings
Test and retest recordings were acquired using the same experimental set-up. The participant was seated in front of a computer screen. Recordings were acquired using a 32-channel Geodesic Sensor electrode net he had been trained previously to wear. Continuous EEG recordings were acquired at sampling rates of 1000 Hz (auditory) and 250 Hz (visual), using a vertex reference electrode and a filter cut-off frequency of 4 kHz with a Geodesic EEG system (Electrical Geodesics, Inc., Eugene, Oregon). Electrode impedances were ≤50 kΩ. A caregiver accompanied the participant to all sessions and remained in the room during the recordings.
2.2.1. Auditory Stimuli and Task
For the auditory recordings, two single-frequency tones (1000 Hz, 1200 Hz), 250 ms in duration, were presented passively in a 300-trial oddball paradigm. Both tone frequencies are within the speech range (500 - 4000 Hz), and 1000-Hz tones are routinely used in clinical studies and typically elicit robust responses. The 1200-Hz tone was interleaved infrequently (11% of trials) and non-conse- cutively among a series of 1000-Hz tones. The tones were presented binaurally through headphones at a comfortable listening level (50 dB), as determined by a listening pre-check, and were well tolerated by the participant. For the duration of the recording, the participant watched a video montage comprised of family and friend photos and short animations without sound. He completed the 10-minute auditory recording without a break. The test-retest interval for the auditory recordings was 6 months.
2.2.2. Visual Stimuli and Task
Visual evoked potential recordings were acquired during practice sessions to familiarize and train the participant for participation in future electrophysiologic visual studies. Two practice sessions, occurring 2.5 months apart, were conducted to train the participant to discriminate between simple shapes (circles, squares) using a computer mouse. Interspersed randomly among the shape discrimination trials were 100-ms duration checkerboard trials that were presented passively (e.g., no response required). The passive checkerboard trials were presented in a 5:1 ratio to shape discrimination trials in two blocks of 78 trials each that were separated by a short break during which the electrode net remained in place. The duration of each block was approximately 20 minutes. Checkerboard stimuli are used routinely in clinical studies and are known to elicit robust cortical visual evoked responses with as few as 10 trials [25]. Only the passive checkerboard trials were used in the visual evoked response analysis.
2.3. Data Analysis
Signal processing was performed using MATLAB (R2018b; Mathworks) and the EEGLAB toolbox [26]. The EEG signals were band-pass filtered at 0.1 - 38 Hz; channels with excessive noise or movement artifact were rejected. The EEG signals were re-referenced to an average reference and segmented into trials using either a 1200-ms window (auditory: 400 ms pre-stimulus) or a two-second window (visual: 1000-ms pre-stimulus).
Visual and auditory evoked responses were computed separately by session (test, retest) and passive stimulus: checkerboard stimuli for visual trials; 1000-Hz and 1200-Hz tone stimuli for auditory trials. After individual trials with excessive artifact were excluded, evoked response waveforms were generated by trial-averaging in the time domain.
2.3.1. Auditory Evoked Potentials
Test and retest auditory evoked waveforms were generated from 1000-Hz tone trials based on 138 and 285 trials, respectively. We focused solely on the 1000-Hz tone trials because the number of 1200-Hz tone trials remaining after trial rejection was too small (N ≤ 22) to generate measurable evoked responses. For all auditory test-retest waveform measurements, we used a midline electrode (Cz) to capture the bilateral contributions from auditory cortex.
2.3.2. Visual Evoked Potentials
Visual evoked test and retest waveforms were generated from 18 and 24 checkerboard trials, respectively. Although the number of trials analyzed was relatively small, the total number of trials in each session exceeded the minimum number (N = 10) considered necessary to generate measurable visual evoked potentials [25]. All test-retest visual waveform measurements were computed from the midline electrode (Oz) corresponding to visual (occipital) cortex.
2.3.3. Evoked Potential Measurements
The early cortical P1-N1-P2 components of the evoked responses were identified visually, and their peak latencies and amplitudes were measured. The P1-N1-P2 components are obligatory, automatic evoked responses generated from primary sensory cortex, reflecting cortical detection of sensory stimuli [27] [28]. The auditory N1 response is also known to correlate with hearing thresholds [8] [9].
2.3.4. Test-Retest Reliability Measurements
Test-retest reliability was estimated based on cross-correlation coefficient analysis, which takes into account the entire waveform shape (latency and amplitude) rather than single time-point measurements and is commonly used to estimate similarities between time series signals [29] [30] [31]. Cross-correlation coefficients were computed for a post-stimulus response period of 0 - 200 ms using a sliding 1-ms window with a lag time of ±100 ms (xcorr function).
2.4. Results
2.4.1. Auditory and Visual Waveform Measurements
Auditory and visual test and retest evoked waveforms are shown in Figure 1. Early cortical auditory evoked P1-N1-P2 waveform components were identifiable for both test and retest. The visual waveforms appeared noisier than auditory waveforms, consistent with the smaller number of trials analyzed; however, early visual P1-N1-P2 components were also identifiable.
Peak waveform measurements (latencies, amplitudes) for auditory and visual cortical evoked responses are shown in Table 1. Auditory and visual P1-N1-P2 waveform measurements, for test and retest, were within the normal range.
![]()
Table 1. Test and retest cortical auditory and visual evoked potential measurements.
![]()
Figure 1. Test-Retest Auditory and Visual Evoked Potentials. (a) Plot shows auditory evoked response waveforms for test (red) and retest (blue), with P1-N1-P2 components labeled. Time is on the x-axis in milliseconds (ms); vertical line at zero denotes stimulus onset. Amplitude is on the y-axis in microvolts (µV). (b) Plot shows visual evoked response waveforms for test (red) and retest (blue), with P1-N1-P2 components labeled. Time is on the x-axis in milliseconds (ms); vertical line at zero denotes stimulus onset. Amplitude is on the y-axis in microvolts (µV).
2.4.2. Test-Retest Reliability
Test-retest reliability, as measured by cross-correlation, was high for both auditory and visual evoked responses. For auditory evoked responses, the cross-correlation coefficient was 0.91 (lag −7 ms); for visual evoked responses, the cross-correlation coefficient was 0.80 (lag 0 ms).
Because of the noisy appearance of the visual waveforms and relatively small number of trials analyzed (≤24), we recalculated the cross-correlation after first subtracting the mean of the signal, a method known as cross-covariance [32] [33] to provide a zero-mean “normalized” estimate of test-retest reliability. Cross-covariance was computed for visual and auditory evoked responses using a 200-ms time window (xcov function). This yielded cross-covariance estimates of 0.87 (−8-ms lag) for auditory and 0.51 (24-ms lag) for visual test-retest.
3. Discussion
This case study demonstrates the feasibility of establishing test-retest reliability of sensory evoked responses in individuals with LFA. Repeat auditory and visual evoked potentials recorded from an adult with LFA showed good agreement over intervals up to six months, with cross-correlation coefficients ≥ 0.80. This finding suggests that sensory evoked potentials could be a viable alternative to behavioral testing in clinical and investigative studies and neural sensory responses may be more stable in individuals with LFA than previously thought. Several prior studies have suggested that neural responses may be intrinsically unstable and, therefore, unreliable in individuals with ASD [13] [14]. However, those studies measured trial-to-trial variability and not test-retest variability. Although we did not investigate trial-to-trial variability directly in our study, the high test-retest reliability observed suggests that, even if confirmed, trial-to-trial variability did not impact the overall test-retest reliability of neural responses. Another difference is that most prior studies included only individuals with high-functioning autism, raising the question of whether reliability of neural responses may differ across ASD subtypes. The high test-retest reliability observed for up to six months in our study also suggests that evoked potentials could be useful for measuring and monitoring therapeutic effects in longitudinal interventional studies of individuals with LFA.
3.1. Auditory Evoked Potentials
Auditory evoked waveforms showed very high test-retest reliability, as measured by the cross-correlation coefficient (0.91). The cross-correlation coefficient and cross-covariance estimate (0.87) were similar, even though different time windows were used (250 ms and 200 ms, respectively), suggesting that our test-retest estimates are relatively stable.
Auditory waveform measurement values were in the normal range, suggesting that basic neural processes underlying sound perception are intact. Although no formal threshold testing was done with evoked potentials in this study, robust N1 responses were elicited for stimuli presented at normal conversational levels (~50 dB). The frequency of the tone stimulus, 1000 Hz, is within the speech range (~500 - 4000 Hz), suggesting intact perception of speech frequencies presented at normal conversational levels. These evoked potential findings supplement prior unsuccessful attempts to evaluate the participant’s hearing using behavioral testing.
Test-retest auditory evoked waveforms showed good agreement despite different numbers of trials in each session. Because each session had over 100 trials, differences in number of trials may not have impacted the overall response waveforms sufficiently to reduce reliability, as also indicated by the post-hoc, cross-covariance analysis that yielded high test-retest agreement (0.87).
3.2. Visual Evoked Potentials
Despite their noisy appearance, the test and retest visual evoked waveforms showed good agreement as measured by the cross-correlation coefficient (0.80). Although checkerboard stimuli can elicit large robust evoked responses with as few as 10 trials in typically developing individuals [25]. our results suggest that establishing reliability of visual evoked responses in individuals with LFA may require larger numbers of trials. This view is supported by the post-hoc, cross-covariance analysis that yielded a lower cross-covariance coefficient (0.51), suggesting the test-retest estimate is less stable due to the smaller number of trials analyzed.
The visual evoked waveform measurements were within the range of normal values, suggesting basic visual processing is intact. Although the participant’s visual processing abilities had not been evaluated previously, there were no reported concerns about visual perception or recognition abilities. The evoked potential results suggest normal visual processing, consistent with prior observations.
Two take-away lessons from this study may be useful in guiding future clinical and research investigations involving individuals with LFA. First, individualizing electrophysiology paradigms for LFA participants can facilitate their inclusion in test-retest reliability studies and does not reduce the validity of the reliability estimates, which are based on within-subject, not between-subject, comparisons. A second take-away is the high test-retest reliability we observed supports the utility of sensory evoked potentials as diagnostic tools in clinical evaluations and as outcome measures in therapeutic and investigative studies of individuals with LFA.
3.3. Conclusion
Our results demonstrate that sensory evoked potentials can be highly reliable in individuals with LFA, offering a viable complement to behavioral tests of sensory processing in clinical and investigative studies. Evaluating auditory and visual function in individuals with LFA is important to rule out abnormalities that could contribute to the atypical language development and social interaction impairments characteristic of ASD.
Acknowledgements
This work was supported by the Therapeutic Cognitive Neurology Fund (BG). The authors thank the participant and his family for their participation and Nancy Grund for editorial assistance.
NOTES
*Co-first authors; equal contributions. #Corresponding author.