ABSTRACT

A range of different language systems for nursing diagnosis, interventions and outcomes are currently available. Nursing terminologies are intended to support nursing practice but they have to be evaluated. This study aims to assess the results of an expert survey to establish the face validity of a nursing interface terminology. The study applied a descriptive design with a cross-sectional survey strategy using a written questionnaire administered to expert nurses working in hospitals. Sample size was estimated at 35 participants. The questionnaire included topics related to validity and reliability criteria for nursing controlled vocabularies described in the literature. Mean global score and criteria scoring at least 7 were considered main outcome measures. The analysis included descriptive statistics with a confidence level of 95%. The mean global score was 8.1. The mean score for the validity criteria was 8.4 and 7.8 for reliability and applicability criteria. Two of the criteria for reliability and applicability evaluation did not achieve minimum scores. According to the experts’ responses, this terminology meets face validity, but that improvements are required in some criteria and further research is needed to completely demonstrate its metric properties.

1. INTRODUCTION

As the nursing discipline develops, there is a clear need for the design, validation, implementation and evaluation of standardized vocabularies to describe nursing phenomena and actions. Nursing language plays an important role in defining what nurses do and why they do it and helps to develop, express and understand concepts in the discipline. In the words of Clark and Lang [1]: “If we cannot name it, we cannot control it, finance it, research it, teach it or put it into public policy”, as a result, language systems have become an important issue for nursing international agenda. The increasing emphasis on overall efficiency in healthcare systems places nurses under growing pressure to demonstrate their contribution to quality and cost in terms of the health problems they resolve or prevent and to health outcomes in patients, communities and societies [2-4].

Nursing terminologies have been implemented as interface terminologies at the point of care and as administrative or management terminologies to retrieve nursing clinical data and information that is useful for supporting decision-making on nursing activity and productivity, staffing, skill mix and assignment, quality of care, clinical safety and costs [5-7]. As Müller-Staub et al. [8] stated: “standardized computer-compatible professional terminology is becoming a requirement, especially by institutions and healthcare systems that bear the costs of health care”.

Substantial work has been done in the development of nursing language systems for nursing practice since the early 1970s in order to define the professional identity of the discipline and to include nursing data in healthcare information systems [9-11]. Developers pioneering nursing vocabularies did not generally consider informatics and semiotics in their studies, with the result that some essential features of standard terminologies for use in computer-based information systems are lacking [12].  

In 1989, Graves and Corcoran published the “Data, Information and Knowledge Framework”, a conceptual work widely recognized throughout the international nursing community [9,13]. The same year, Nelson and Joos [14] proposed the addition of the concept “Wisdom” to this continuum. Later, in 2008 the American Nurses Association (ANA) included “Wisdom” in defining the meta-structures of nursing informatics, providing a basis for linking theory and practice [14,15].

Currently, ten nursing terminologies and two data sets are recognized by the ANA for supporting nursing practice: the NANDA International Taxonomy (NANDA-I), the Nursing Interventions Classification (NIC), the Clinical Care Classification (CCC), formerly Home Health Care Classification, the Omaha System, the Nursing Outcomes Classification (NOC), the Nursing Management Minimum Data Set (NMMDS), the Perioperative Nursing Data Set (PNDS), SNOMED Clinical Terms from the the International Health Terminology Standards Development Organization (IHTSDO), the Nursing Minimum Data Set (NMDS), the International Classification for Nursing Practice (ICNP) from the International Council of Nurses, ABCcodes and Logical Observation Identifiers Names and Codes (LOINC) [12,16].

Historically, other international efforts to advance in nursing vocabularies are reflected in the Canadian Health Outcomes for Better Information and Care (C-HOBIC) project, a standardized nursing information program for inclusion in EHR [17]; the nursing diagnosis project from the Centre for Development and Research (ZEFP as the German acronym) of the University of Zurich in Switzerland, based on Käppeli’s model [8]; the Wellbeing, Integrity, Prevention and Safety model, yielding the acronym VIPS in the Swedish spelling, originally designed to provide a structure for nursing documentation and to support individualized approach to nursing care based on written care plans [18] and implemented in Sweden and other Scandinavian and north European countries [18-20]; and the European Nursing Care Pathways (ENP), a classification system developed in Germany and currently in use at many hospitals and other healthcare settings in this country [21].

In any case, according to Bakken et al. [5], some issues should be considered: “First, although some might wish for a single terminology with broad coverage of health care domain, it is clear that in the near future multiple terminologies will continue to exist. Second, the acceptance of standardized nursing terminologies continues to grow, but their use is not yet universal. Third, nurses routinely use terms other than those in standardized nursing terminologies in the care documentation process. Fourth, no single existing terminology can serve all purposes equally well; the level of granularity of data required for decision support is very different than that required for billing or for examining disease patterns in a population over time”.

In order to facilitate an integrated approach to further development and implementation of nursing terminologies, various models, sets of criteria and features have been proposed for the evaluation of the validity and usefulness of clinical terminologies [5,8,15,22,23] and recently the NANDA_I has proposed updated criteria for the evaluation of nursing classifications [24].  

This article focuses on a new nursing interface multiaxial terminology for representing nursing phenomena, implemented in the electronic health records (EHR) in eleven hospitals in Catalonia [25,26]. First, the evolving status of the coverage and general structure of this terminology is briefly described. Second, the results of an expert survey to test face validity are presented and discussed.

Brief Description of the Evaluated Terminology

The name of this nursing interface terminology is based on six key concepts: Architecture, Terminology, Interface-Information-Nursing (Infermeria) and Knowledge (Coneixement) yielding the acronym ATIC in the Catalan spelling.

The main goals of this interface terminology are to simplify the organization of current nursing knowledge in the EHR, to facilitate systematized clinical data entry and to promote nursing information retrieval and exchange to contribute to the generation of new knowledge.

The ATIC terminology is designed as a nursing concept-oriented, multi-axial, interface controlled vocabulary to reflect health status, problems, situations and responses for which nurses are accountable, the interventions and actions they perform, the clinical findings they assess and the outcomes they evaluate as well as their specifications in different axes.  

Concepts in this terminology were generated from the study and the analysis of the natural language that nurses use in their clinical practice. The concepts were labeled, assigned to an axis, coded, defined, dissected and are currently being mapped to other nursing terminologies.

The original terminology was written in Catalan and Spanish, the author’s normal languages of use. Terms from the nursing diagnosis axis are also available at request in English, French, Italian, Russian and Portuguese. The Arabic and Chinese translations are ongoing and the translation of the other axes of the terminology to all these languages is under consideration.  

Since 2008, the ATIC terminology has been implemented in the EHR systems in eleven hospitals in Catalonia: three large metropolitan teaching centers, three urban university facilities, three community hospitals, one rural hospital and one in-patient adult cancer centre joined the project. Overall, more than 3500 adult and pediatric acute in-patient beds (including step-down units), representing around 120,000 patient care episodes per year and more than 7000 registered nurses that use this language system in the daily practice.  

Information on the evolving status of the terminology, its philosophical and theoretical foundations based on an interpretative conceptualization of the meta paradigmatic concepts of the discipline (individual, health, environment and nursing), an analysis of its conceptual framework, including the nursing process, diagnosis, outcomes and interventions, and some other studies have been published elsewhere [25-28].

The primary purpose of this study was to evaluate the results of an expert survey on the face validity of the ATIC terminology.

The research questions for this study were:

Does the ATIC terminology meet the criteria of a nursing controlled vocabulary?  

To what extent does it meet these criteria?

2. METHOD

2.1. Design

The study applied a descriptive design, with a cross-sectional survey strategy, using a written questionnaire to elicit data.

2.2. Sample

Sample size was determined considering this face validity evaluation study as a pilot test for future validation research protocols. In pilot testing, 30 to 50 participants are usually recommended or at least a greater number of participants than the number of questions included in the survey [29].

Using a convenient sampling technique, experts responsible for nursing methodology in their facilities were invited to participate. The only inclusion criteria required were a minimum of five years’ professional experience in the use of the EHR and responsibility for the nursing process implementation in a public hospital.

Thirty-five nurses from the different Catalan provinces agreed: 22 nurses from Barcelona, five from Girona, four from Tarragona and four from Lleida. Most of them (85%) held advanced nursing degrees (postgraduate education), 40% held master’s degrees and 15% were on doctoral programs.

The panel included experts from six university hospitals, two community hospitals and an adult in-patient cancer centre; 45% had previous experience in other healthcare settings (mainly community care, home health care, nursing homes and private medical clinics); 30% were associate lecturers in nursing schools at private and public universities.

2.3. Data Collection

Criteria for evaluating validity, reliability and applicability of health controlled vocabularies had been previously defined by Bakken et al. [5], Müller-Staub et al. [8], ANA [15] and Trent Rosenbloom et al. [23]. These criteria were applied in a short standardized questionnaire constructed for the survey. The survey included 24 questions organized within the following topic areas: 1) Theoretical basis; 2) Relevance; 3) Orientation to nursing phenomena; 4) Consistency; 5) Coherence; 6) Potentiality for linking elements; 7) Non-overlapping; 8) Nonredundancy; 9) Non-ambiguity; 10) Understandable for nurses; 11) Multi-usability; 12) Potentiality for mapping; 13) Simplicity; 14) Context free identifiers; 15) Synonyms; 16) Attributes; 17) Concept-orientation and 18) Concept permanence (Table 1).

Each question in the survey could be answered in a scale from 0 (Totally disagree) to 10 (Totally agree). Any question could be answered as “non-applicable”, “do not wish to respond” or “not known”.

Ethical issues related to anonymity and data confidentiality were guaranteed. Participants were accordingly informed in a cover letter and were also informed of the nature of the study and the method for responding to the questionnaire. Subjects who completed and returned the questionnaire between October 1st and December 30th 2011, were considered to have agreed to take part voluntarily in the survey.

2.4. Data Analysis

The analysis for the main outcomes measured the mean global score which included all the topics evaluated and the percentage of criteria scoring 7 or more. Secondary outcome measures included mean score for each topic. Data were processed onto an Excel spreadsheet (Microsoft Office 2007) and revised to identify potential processing errors or inconsistencies. Depending on the properties of the data, frequencies in percentages, medians, means and standard deviations were calculated for description. Confidence interval was calculated for a confidence level of 95%.

3. RESULTS

The final analysis included 35 questionnaires. Detailed response rate reached 99.3%. Participants in the survey were mainly female nurses (89%), aged between 25 and 59, with extensive professional experience (Mean 18.5, CI ± 2.9, range 5 - 38 years). Additional information on the participants’ professional experience is presented in Table 2.

In testing the validity, reliability and applicability criteria, the mean global score for the ATIC terminology was 8.1 (SD 0.9; CI ± 0.3) with all criteria. In 91.6% of responses, scores on the criteria evaluated were 7 or above.

The mean result for validity criteria analysis achieved a score of 8.4. The validity criterion with the highest score was “Orientation to nursing phenomena” (Mean 8.8, CI ± 0.3).

None of the validity criteria assessed had a mean score under 7.

The mean result for reliability and applicability criteria was 7.8. The reliability criterion with the highest score was “Multi-usability” (Mean 8.8, CI ± 0.3). The topics “Concept-orientation”, “Concept permanence” and “Context free identifiers” also achieved mean scores higher

than 8.

Two of the criteria for reliability and applicability evaluation did not achieve the minimum score of 7: “Non-redundancy” (Mean 6.8, CI ± 0.7) and “Synonyms” (Mean 4.5, CI ± 1.0). Table 3 presents detailed results for each criterion evaluated.

4. DISCUSSION

The results suggest the interface terminology evaluated for face validity seems to meet the criteria required for a nursing controlled vocabulary, although some important issues should be taken into account.

First, face validity is a metric property aimed to determine if the terminology represents what is intended to represent, but it is considered “weak evidence” that might support construct validity; to quote Suttleworth [30], “Whilst face validity is a weak measure of validity its importance cannot be underestimated, because it offers a contrast to content validity”.

Face validity only means that the terminology looks like it works, not that it has been proven to work; so it represents a first step in the validation process and at this point, if the measure seems to be valid, further research can be planned and conducted to determine full content validity, criterion validity, reliability and other properties.

Second, as shown in the results, none of the criteria evaluated achieved a score of excellence (9 or higher), clearly indicating the existence of an improvement threshold.

Third, the two reliability criteria that did not achieve the minimum scoring, redundancy and synonyms are related because synonyms may represent a type of redundancy.

Redundancy refers to the condition in which the same information can be stated in different ways. Whilst some redundancy may be inevitable and it is considered desirable, like the type of redundancy generated by the presence of synonyms [31], some authors consider redundancy as an indicator of ambiguity [12,32] or an indicator of the complexity of a terminology [33].  

In terms of this evaluation, the insufficient number of “Synonyms” is the main problem in this terminological system. Synonyms “help users to find formal terms that match users’ informal descriptions; the presence of adequate synonyms increases the usability of the interface terminology” [23]. However, it should be emphasized that this is a “young” nursing interface terminology which is still evolving and which can be enriched in the future.

According to the evaluation study by Müller-Staub et al. [8], only NANDA-I meets the validity criteria of a nursing diagnosis classification, but as these researchers state: “criteria may reflect a NANDA bias because NANDA literature dominates the field because of its 32-year history”. This situation has probably changed in the last years, as a great deal of research has been produced on the ICNP [34-37], although ICNP is not intended to be an interface terminology nor a classification, but a reference terminology or a Unified Nursing Language System [38]. More recently, other nursing classifications have been recognized as meeting these criteria [24].

Nursing taxonomies and classifications are being used around the world as interface terminologies for healthcare computer-based systems [39-41]. Probably, the nursing community should rethink whether classifications and taxonomies are constructed to reach this or other goals, since “not all terminologies serve all purposes equally well” [5]. The degree of specificity of the concepts in an interface terminology may be more appropriate for direct patient care than that found in the classification systems. The level of abstraction of the later ones may better serve retrieval and statistical purposes; so probably a balance should be found in-between.

While controlled vocabularies and computer-based systems must contribute to promote the visibility and influence of the nursing profession [16], the social mandate of our discipline and practice has been constructed to meet human health-care related needs with an individualistic approach, to address the health of the collective and to participate and support the health care systems [42-44], so nursing terminologies used at the point of care have to contribute to assure patients’ safety through knowledge-based practices and to warrant clear communication among professionals. One of the main benefits of the development and implementation of the ATIC terminology is that it provides nurses with systematized data and information, in a controlled, “closeto-natural” language system essential to planning and evaluating patient care and status. Other health care professionals could also benefit from this implementation, because it provides nurses with a new way to communicate with physicians, assistants and other health agents. Previous studies have shown that improved communication between nurses and physicians is reflected in improved patient’s outcomes [45-47].

This study has some significant limitations that should be mentioned; those inhered to a descriptive design and a face validity evaluation, as previously introduced, and others as follow. Difficulties have been found in identifying a tool to measure the properties of controlled nursing vocabularies. In the absence of a validated tool to use, the evaluation criteria and assignment of a criterion to the validity or reliability group are mainly based on the studies of Bakken et al. [5] and Müller-Staub et al. [8] although, as previously explained, criteria from other sources have also been considered; this fact may have unwillingly introduced a bias in the results.  

No pilot testing of the questionnaire was performed and this should also be considered a limitation. The sample size, although appropriate according to the literature, might have influenced the results and the non-random sampling technique selected reduced the power of the study and prevented extended statistical analysis.

This survey, although a multicenter study, included only participants from Catalan public hospitals. The high response rate and the long professional experience of the participants could act as compensatory factors, but nationwide or international studies are probably needed.

Further research is needed to demonstrate that the ATIC terminology meets the criteria for a nursing controlled vocabulary in terms of system terminology attributes, metric properties and usability factors such as efficiency or user satisfaction.

According to the experts’ survey, in terms of face validity, this nursing interface terminology meets the criteria for a nursing controlled vocabulary, except for “Synonyms” and “Redundancy”. Overall, the rest of criteria evaluated presented high scores.

The inclusion of this nursing interface terminology in electronic health record systems may contribute to facilitate data entry, promote patients safety and continuity of care across the healthcare system and provide useful data to facilitate aggregation and analysis of relevant information for decision-making to clinicians, managers and policy-makers. Valid, reliable, comprehensive, easyto-use, nursing entry terminologies are needed for the immediate future nursing practice worldwide

5. ACKNOWLEDGEMENTS

My sincere thanks to Dr. Jordi Carratalà Fernandez and to Dr. Maria Teresa Icart Isern for the revision of this manuscript.

REFERENCES