Selective Samples and the Accuracy of Screening for Intellectual Disabilities : Learning Disability Screening Questionnaire

The Learning Disability Screening Questionnaire (LDSQ) was developed primarily to provide a short and effective screening instrument to indicate possible intellectual disability amongst persons who come in contact with the service system. This study examined the validity of the Norwegian version of LDSQ in a sample of 35 subjects. All participants were outpatients referred for neuropsychological examinations. WAIS-III full-scale score was used for criterion validity. At a stated cut-off score of 46.6%, LDSQ showed several false negatives and false positives but no true positives. This raises the question of whether this test represents a valid screening instrument for learning disabilities in a clinical sample.


Introduction
Persons with intellectual disabilities or learning difficulties may experience challenges in daily life, including more formal situations like seeing a doctor, understanding official letters or understanding their role in society.Persons with these problems may beat a greater risk of experiencing difficult situations, being victims of crimes and experiencing abuse [1], as well as also being at risk of committing crimes.One study [2] showed that 10.8% of persons in prison, had an IQ lower than 70.We also know that there is a hidden population of persons with an intellectual disability, meaning that the "true" prevalence of this population is higher than the "administrative" prevalence [3].The statistical prevalence based on IQ-scores defines 2.3% of the normal population as having a mild intellectual disability (IQ below 70) or a more severe form of disability.This indicates that there is a chance of meeting patients in the health system, or offenders in the legal systems, with an undiagnosed state of intellectual disability.The results of this phenomenon can mean that trials in court proceed as usual, with a client who does not understand the procedures.It can mean that certain persons do not follow a doctor's or a health-adviser's recommendations, they can be at risk of getting little help from the local community service, and they may be in danger of establishing unhealthy relationships, or being abused.The lack of useful screening assessments for identifying persons with an intellectual disability in the Norwegian health care system results in unaddressed needs and insufficient health services [4].The Learning Disability Screening Questionnaire (LDSQ) was created in Great Britain (GB) to help staff in social services to identify people with a learning disability [5].The British concept "learning disability" corresponds to the more international concept of "intellectual disability" (ID).The correlation between LDSQ and WAIS-III in the GB studies was found to be significant (r = 0.751, p < 0.0001 [5] and r = 0.71, p = 0.01 [6]).The questionnaire can be used in many settings and by a wide range of professionals.The questionnaire can be completed in clinical interviews, by the client him or herself or by a person who knows the client well.
The LDSQ contains background questions that are often already known to the health system or social worker.The construct of a screening instrument based on this information was the rationale behind the LDSQ.The LDSQ involves collecting background information about previous contact with social services, earlier special education-programmes, independent living, employment, reading and writing abilities and telling the time.All the items can be administered quickly; the test (administration and scoring) can be completed within 10 minutes.The items resemble some of the themes found in the Vineland Adaptive Behaviour Scales (VABS) [7], but it is still unknown if the two scales are correlated.
This study aimed to validate LDSQ with WAIS-III [8] in a Norwegian clinical sample.The screening has shown promising results in previous studies [6] and the lack of identification routines call for proper screening at a primary level.No previous research comparing these two instruments in Norway was located, and the agreement between them can be important for future assessments.

Sample and Participants
All individuals were referred for examinations in a clinical neuropsychological practise in Norway.There were 18 male and 18 female subjects.The subjects' ages ranged from 16 to 66 years (M = 34; SD = 13.9).IQ score for the participants ranged from 67 to 100 (M = 86.6,SD = 8.5), two of the participants had an IQ below 70.9 had an IQ below 80.We choose not to include the participants with a total IQ-score above 100, because of the roof-effect of the LDSQ-Scale.The LDSQ describes non-linear results, with 100% as the highest score possible.

Procedure
The data were collected by a specialist in clinical neuropsychology, and all subjects were tested with both instruments.The instruments were administered in a random order.All subjects were tested at the test administrator's offices.After collection, all data were anonymised.Each subject gave informed consent for taking part in this study.The study was approved by the regional ethical committee for medical research.

Measures
The Norwegian translation of the LDSQ was undertaken by the present authors and included the complete version.The final Norwegian version was back-translated into English and reviewed by the original author.The LDSQ comprised seven questions.Three of the LDSQ items addressed academic skills relating to understanding functions of a clock, reading and writing, two items addressed adult independency: living independently or not, having employment or not, and two addressed the experiences of receiving assistance at school or at home.
The scores were dichotomously rated 1 or 0, with 0 indicating a functional loss, and 1 indicating no loss.The total scores were summed, and divided by the total number of questions answered.This again gave a percentage score as a result.The minimum number of questions answered cannot be lower than five.
Indication of an ID occurred when less than 46% of the items were scored according to a functional loss based on statistics indicating that LDSQ identifies persons without ID with an accuracy of 87%, and a person with ID with 91% accuracy [5].

Data Analysis and Statistic Procedure
We analysed the specificity and sensitivity of the LDSQ by using ROC-analysis.Spearman's rho was used to compare the total scores of the LDSQ and WAIS, and to compare the LDSQ and the verbal/performance parts of WAIS.Further, a scatter-plot was used to illustrate any co-variance between the LDSQ and WAIS.The data were analysed using the SPSS software programme, version 19.

Descriptive Statistics
In this study, the total percentage scores of the LDSQ were not statistically significantly correlated with WAIS-III, full-scale IQ (r = 0.207, p = 0.23).The correlation between the LDSQ and verbal score (WAIS-III) as well as for the performance score (WAIS-III) were not correlated (see Table 1).

Sensitivity and Specificity
The ROC-analysis showed a low degree of sensitivity and specificity for the LDSQ (area under the curve = 0.49), which was not significant (Figure 1).The scatter-plot showed a weak covariance for the scores from LDSQ with WAIS-III.None of the scores under the LDSQ cut-off (score 46) co-varied with the lower scores from WAIS-III (total IQ-score of 70 and below).
Figure 2 illustrates the distribution of the scores showing that 7 persons had a LDSQ-score lower than cut-off (46%).All of these persons had at the same time an IQ-score of 80 and higher.Two of the persons had an IQ below 70.Their LDSQ-score was above the cut-off.Eight persons had an IQ at or below 80 but their LDSQ score did not fall below the cut-off.With a cut-off score at 46%, sensitivity of the LDSQ was 0%, specificity 83%.

Discussion
By analysing the correlation results between the LDSQ and the WAIS, we found that LDSQ did not correlate well with WAIS III, when we compared the scores from WAIS (from IQ 100 and below), with the scores of the   LDSQ.The ROC-analysis further showed that the LDSQ did not recognise subjects with an IQ below 80 and this represents a statistical lack of sensitivity and specificity for the purpose of discovering persons with an IQ-score in this area.
The LDSQ was developed to indicate the possible presence of ID, but not to be used for diagnostic purposes.The question is whether this test is a good predictor and valid alternative to other brief questionnaires to recognise learning disabilities or intellectual challenges.It is important to stress that the persons included in this study were referred to a neuropsychological assessment for different reasons, and that this study did not include persons from a normative sample.With the given cut-off score, the LDSQ does not appear to be a valid scale for identifying people with a need for further assessment (such as full scale neuropsychological examinations, diagnostic procedures etc.).
The present study was conducted with a convenience sample, as described above.Contrary to the samples in previous studies [5] [6] using stratified low functioning samples, this study used subjects already identified with ID when studying the screening tool.Our results pointed out that LDSQ is not valid and reliable for identifying persons with intellectual difficulties in our population/sample.This study included 36 persons, of whom 9 had an IQ of 80 and below.This is perhaps a low number, but indicates that one should be careful in using this screening to identify IDs in other populations/samples as well.The significant difference in the sample applied in the GB and the Norwegian studies may explain most of the different outcomes.Screening a sample of people with identified ID in Norway, would possibly confirm the British results.
A previous study [9] made a similar comparison between the Hayes Ability Screening Index (HASI) [10] which demonstrated better and more valid results.HASI is a more test-like scale, and because of that, may be more useful because it eliminates bias such as living conditions, having a job, etc.This stresses the importance of adding some cognitive elements or tests (such as the Trail Making Test in HASI) to the screenings, to ensure a better and valid measure of intellectual functions.These point to the cultural challenges that exist, concerning all "objective" screenings, assessments and tests.This also highlights that it is important to be careful about generalizing the results from one sample or study alone, to a whole population, and that two countries can have different social and educational support systems, that can provide elements directly relevant to some of the test items, for example having a job or not, managing to read and write.This perhaps reflects issues concerning the Norwegian welfare system, which provides social, economic and educational support to the main part of the population who is in need, thereby eliminating the differences of socioeconomic status which may exist to a higher degree in other countries (e.g.UK), or indicate the differences between two models of social care for persons with challenges like learning difficulties.Governmental proposals indicate that persons with intellectual difficulties have rights concerning independent living, having a job, and education [11].With that perspective, LDSQ can be seen as a screening of received social or economic support in our population or sample.
Our study found that the LDSQ does not identify persons with learning disabilities in a selective sample of persons with an IQ-score of 100 and below.The screening tool has been applied in several studies in GB during the last few years [12]- [14] with adequate validity in contrast to findings from the present study.The usefulness of the LDSQ in a Norwegian setting does not seem to be as good, and is thus not recommended until after it has been examined further.

Figure 1 .
Figure 1.Roc curve presenting scores on WAIS and values over and under cut-off for LDSQ (AUC = 0.49).

Table 1 .
Correlations (Spearman's rho) between the LDSQpercentage score and the WAIS-III scores.