Testing of the Content Validity of a Modified OPCq Instrument — A Pilot Study in Norwegian Home Health Care

Aim: To test the content validity of a modified Oulu Patient Classification instrument (OPCq), part of the RAFAELA Nursing Intensity and Staffing system in home health care (HHC) in Norway. Background: Due to the growing number of patients in HHC, a Patient Classification System (PCS) whereby the systematic registration of patients’ care needs, nursing intensity (NI) and the allocation of nursing staff can occur is needed. The validity and reliability of the OPCq instrument have been tested with good outcomes in hospital settings, but only once in an HHC setting. In this study, the OPCq is tested for the first time in HHC in Norway. Methods: A pilot study with a descriptive design. The data were collected through a questionnaire (n = 44). Both qualitative and quantitative analyses were used. Results: The OPCq fulfills the requirements for validity in HHC, but the manual may need some minor adjustments. Discussion: The OPCq seems to be useful for measuring nursing intensity in HHC. Staff training and guidance, high-quality technological solutions and that all technology works satisfactorily are important when implementing a new PCS. Further research is needed in regard to NI and the optimal allocation of nursing staff in an HHC setting.


Introduction
The percentage of older people in the population of many countries is rising, concur-How to cite this paper: Flo, J., Landmark rent with a widespread trend to refocus health care services away from hospital care and into municipal-based care.The number of beds in hospitals and nursing homes facilities has decreased in the European Union [1], which has resulted in a significantly increased need for home health care (HHC) services and, consequently, an increased need for nursing resources.Until now, research on the allocation of nursing resources in HHC has been scarce [2], and relatively few instruments for classifying and measuring patients' care needs in HHC have been developed and regularly used [3] [4] [5] [6].
As delineated by the Norwegian Ministry of Health and Care Services in the Coordination Reform, municipalities are now responsible for the care of individuals with complex medical and psychosocial needs [7].A person-centered approach in primary health care is also recommended [8].During the last decade, several researchers have found that a person-centered, holistic approach improves the quality of care that older and vulnerable patients receive [9] [10].In a person-centered approach, one important objective is the fulfillment of patients' physical, psychological, sociocultural and spiritual needs [11].Yet organizational structures also affect whether patients' needs can be fulfilled, especially in regard to staff resources: both in terms of educational level and number of nurses and/or nursing hours.An optimal nursing workload ensures that nurses can meet patients' needs.Aiken et al. [12] found associations between higher mortality in hospitals and fewer nurses qualified at bachelor's degree level.In a recent study, a clear association between a nursing workload above the optimal level and mortality was found [13].
We maintain that if nursing resources are not matched to patients' care needs and nursing intensity (NI), adverse events and mortality will increase in HHC.It is therefore essential that the continual classification and measuring of patients' care needs and NI occur.New instruments and systems for the systematic monitoring of NI are needed, so that nurse staffing resources can be purposely planned and quality of care ensured.
NI as a concept is closely related to the concepts "patient dependency", "acuity" and "severity" [14] [15] [16].NI can be defined as how nursing-intensive a situation is and how dependent a patient is on the care provided: how much care, help and support a patient receives [14] [15].
The RAFAELA Nursing Intensity and Staffing system is a classification system developed in Finland in the early 1990s for hospital settings [17] [18].The RAFAELA system provides a rational, systematic and objective foundation for evidence-based human resource management [19].It is a well-functioning, tested administrative tool for nurse managers, which has been used in Finland, Norway and Iceland and on different health care and hospital organizational levels [17] [18] [20] [21] [22].In an integrative review of PCS, Fasoli and Haddock [23] found that RAFAELA was one of a small number that met the criteria for validation and reliability in hospital settings.
RAFAELA is composed of two instruments, the Oulu Patient Classification/Qualisan (OPCq) instrument and the Professional Assessment of Optimal Nursing Care Intensity Level (PAONCIL) instrument.The OPCq was developed for hospital use and incor-porates a holistic approach to care, measuring basic physical needs, emotional needs and nursing care activities.The validity of the OPCq has been tested in HHC once in Finland [4].In this paper, we describe the process whereby the OPCq was modified to suit a Norwegian HHC setting and present the results of a pilot study of the content validity of the modified OPCq, assessed through nurses' evaluation of the instrument.

Background
PCS and NI instruments were first developed in the USA in the 1940s for use in hospital settings; similar development and research in the Nordic countries started first in the early 1970s.Of those designed for use with older patients in HHC settings, the majority have been developed in the USA.We found several tools: Clinical Care Classification (CCC) [24], Resident Assessment Instrument (inter RAI), Resource Utilization Groups (RUG III) [5], Community Health Intensity Rating scale (CHIRS), Easely-Storefjell Patient Classification Instrument (R-ESPCI) [3] and Caseload Intensity Tool (CIT) [6].Some instruments estimate functional capacity instead of NI, for example the Katz Index of Independence in Activities of Daily Living [25] and the modified Katz ADL [26].
In Sweden, the Time in Care instrument (TiC) has been used in some municipalities [27].In Norway the Individbasert statistikk for pleie-og omsorgstjenesten i kommunene (IPLOS) register, a central health register that forms the basis for national statis- tics for the nursing and care services, is used in Norwegian municipalities to catalogue individual patient's resources and need for assistance [28].The IPLOS register is not all-encompassing and mainly classifies functional capacity; it does not register sociocultural or spiritual needs.Norwegian nurses consider it to be a technology-driven register whereby interpersonal skills are diminished [29].The need exits to test an instrument that captures all dimensions of nursing care, such as the RAFAELA system, which is the most commonly used system in the Nordic countries [14] [22] [30] [31].

Description of the RAFAELA System and the Modification of the OPCq for a HHC Setting
When using the RAFAELA system, it is possible to gather information on each patient's need for individual care and ensure the realization of a person-centered care.HHC Nurse Managers can use the RAFAELA system to balance patients' needs and nurse staffing resources and realize an optimal nurse staffing level.The RAFAELA system is used to ensure that the workload per nurse (expressed in NI points per nurse) is on the optimal NI level.This makes it possible to ensure the quality of nursing, good patient outcomes, good working conditions and the effective use of available resources [21].
The validity and reliability of RAFAELA in hospital settings has been assessed in several dissertations [14] [30] [31] and in primary health care in one dissertation [4].With RAFAELA it is possible to respond to the constant variation in patients' needs, and it consists of the following components: 1) Daily registration of patients care needs using the OPCq instrument; 2) Daily registration of actual nurse staffing resources; 3. Periodical determination of optimal NI level using the PAONCIL instrument.
In the OPCq, nursing care and care needs are organized into the following six subareas: 1) Planning and co-ordination of nursing care; 2) Breathing, blood circulation and symptoms of disease; 3) Nutrition and medication; 4) Personal hygiene and secretion; 5) Activity, sleep and rest; 6) Teaching, guidance in care and follow-up care, emotional support.Using the OPCq, nurses measure the six sub-areas at regular intervals, with A = 1 point (independent), B = 2 points (partial need of help), C = 3 points (repeat need of help, complex) or D = 4 points (constant need of help, very complex); the sum provides a total NI per patient per day in hospital or per HHC visit.Total NI can thus vary from 6 to 24 points.This project was a collaboration between a municipality in southeast Norway and a regional University College and lasted from 2012-2014.The Finnish Consulting Group Ltd. (FCG) [32] supplied the RAFAELA system and led a two-day introduction (educational program) to the RAFAELA system for all nursing staff at the participating HHC units in October 2012.The FCG also provided a manual for the OPCq instrument, which included instructions for its use and descriptions of the six sub-areas, classification system and key terms.Prior to the start of the project, the OPCq manual was modified for use in an HHC setting.Two workgroups consisting of 8 people in total (including registered nurses (RNs), practical nurses (PNs) and organizational leaders) met 4 -5 times to discuss modifications to the OPCq manual with the internal project leader.Modifications were based on what staff considered to be relevant concerning clinical practice in HHC.The internal project leader regularly discussed the modifications that the workgroup had agreed upon with an external project leader/professor from the University College, and the experts at FCG thereafter approved all modifications.

Modifications were made as follows. Examination program at regular intervals B-C
was removed from sub-area 1 (Planning and co-ordination of nursing care).The requirement that nursing staff assess electrolyte and acid-base disturbances or increased intracranial pressure was removed and patient positioning was changed to bedridden in sub-area 2 (Breathing, blood circulation and symptoms of disease).Management of prophylactic medication was changed to continuous medication in sub-area 3 (Nutrition and medication).The need for advice prior to discharge from hospital was removed from sub-area 6 (Teaching, guidance in care and follow-up care, emotional support), because the patients were already living in their own homes.Modifications were additionally made to the key terms listed in the manual: "occasional" was adjusted to "need for occasional help" in sub-areas 2 -6.
Prior to implementation of the instrument, all nursing staff at the two participating HHC units were given an introduction to the modified OPCq instrument.The project leader was responsible for all subsequent education related to the project and/or use of the OPCq instrument.
While in hospital settings measurement of the OPCq occurs daily, this was not considered feasible in an HHC setting.Instead, measurement of the modified OPCq occurred after each HHC visit.Following each visit, the nurses first wrote down their classifications by hand and then entered the data into the RAFAELA database after-wards.However, due to the high number of visits per nurse, the daily classifications were assessed as being too time consuming and the FCG and the municipality decided to develop a mobile OPCq classification application.While the final mobile application saved time, during its development and whenever there was poor mobile network coverage the participants were required to continue to write down their classifications by hand, which caused additional stress.

Aim
The aim of this present study was to test the content validity of the modified OPCq instrument, part of the RAFAELA Nursing Intensity and Staffing system, in HHC in Norway.

Ethical Considerations
Approval was sought from and provided by the Norwegian Social Science Data Services (NSD) prior to commencement of the study and appropriate permission was sought from the municipality.A license from the FCG to use the RAFAELA system was sought by the municipality and granted.

Design and Settings
This is a pilot study with a descriptive design.Validity testing of the OPCq instrument through the use of a summative questionnaire was carried out on two HHC units in a medium-size city, about 70,000, in southeast Norway during 2013 and 2014.The study was a part of a municipal research and development program and realized in collaboration with a regional University College during 2012-2014.

Participants and Data Collection
The data collection was conducted in two phases.Inclusion criteria were that participants worked 50% or more, worked day or evening shifts and had participated in the RAFAELA educational program for instruction in the use of the OPCq instrument.In spring 2013 the head nurses at two HHC units handed out 31 questionnaires.The HHC units had a total of 36 staff members, 24 RN and 12 PN or assistants.The head nurses and the coordinators were not included in the study.The questionnaire was answered anonymously and were returned, sealed in a reply envelope, to the same head nurses with a response rate of 71% (n = 22).In order to garner more participant responses, Of the participants (n = 44), 23 (52.3%) were RNs with bachelor degrees, 18 (40.9%)were PNs with vocational degrees and one was an assistant without formal competence (2 missing).A total of 27 (61.4%)had ten years or more work experience, 5 (11.4%) between 5 -10 years, 3 (6.8%) between 3 -4 years and 7 (15.9%) between 1 -2 years (2 missing).The mean age was 40.8 years (MD 39), with a range from 19 -69 years.The majority were women, with only two men.The participants had classified patients' NI about 7 months before the 2013 data collection and 18 months before the 2014 data collection.
The OPCq has been evaluated using the same questionnaire in two earlier studies: once in a hospital setting [14] and once in a primary health care setting [4].For this study, the questionnaire was translated from Swedish into Norwegian and slightly modified to suit an HHC setting.The face validity was tested by six RNs at the municipal research unit prior to data collection.The internal consistency was measured using Cronbach's alpha, with a reliability of 0.96 [33] [34].A reliability coefficient of 0.70 or higher is considered acceptable [34].
The questionnaire comprised 13 questions with set answers and the possibility to comment on eight of the questions.Ten questions had a five-point Likert scale with the variables: 1 = not at all, 2 = partly, 3 = pretty well, 4 = well, 5 = very well: as well as the alternative 0 = cannot say.One question had a five point Likert scale with the variables: 1 = not motivated, 2 = partly motivated, 3 = motivated, 4 = very motivated and 5 = highly motivated.The remaining two questions pertained to demographic variables (gender, work experience, educational level) and whether the OPCq's six measurement sub-areas should be modified.One question was excluded from the questionnaire in that it had different content in the first and second data collections.

Data Analysis
IBM Statistical Package for Social Sciences (SPSS) Version 22 was used for descriptive analyses.Pearson's r and Spearman's rho correlations were also used: both are recommended for use when calculating ordinal scales [33].Inductive content analysis [33] [35] was used in a simplified form to analyze qualitative comments.

Results
The data findings are presented quantitatively and qualitatively below.Note that in the tables, but not the analysis, the questionnaire scoring options were sorted into four categories: very well/well, pretty well, partly/not at all and cannot say.Q2: In your opinion, how well are the sub-areas 1 -6 described in the OPCq instrument?About 80% of participants scored sub-areas 1, 2, and 4 using very well/well or pretty well.Sub-area 5 was given the lowest score (Table 1).Fourteen participants (n = 44) replied that additional sub-areas were needed, 14 that none were needed and 15 cannot say (1 missing).
Qualitative findings.Nineteen participants left written comments, from which two categories were discerned: some sub-areas do not match and poorly adapted to HHC.
In some sub-areas do not match, participants specified that some sub-areas did not match and should be more clearly defined: a degree of overlapping existed and there was uncertainty in regard to the OPCq's NI classification levels B and C. Participants also noted that there were too many situations included in each sub-area.Nevertheless, participants considered some sub-areas to be well described, full of detail and as having good coverage.Still, the use of more suitable keywords was sought.
In poorly adapted to HHC, participants mentioned that they lacked the ability to classify practical things such as: garbage, activities, the washing of garments, support stockings, weather conditions, driving conditions, phone calls, interdisciplinary collaboration and unexpected events.
Q5: How well do sub-areas 1 -6 describe the patient's total NI?More than half of the participants 25 (56.8%)indicated that sub-areas 1 -6 describe NI very well, well or pretty well, while 18 (40.9%)scored this partly or not at all (1 missing) (Figure 1).
Qualitative findings.Eight (n = 44) participants left written comments.While some indicated that the sub-areas were well described, one replied (without further elaboration) that they should be more specific.Some sought better keywords and the ability to register unexpected events and better express concepts such as emotional support and persuasion.
Q8: How well does the OPCq's interpretation of the patient's NI correspond to the interpretation that your experience leads you to? Twenty-one (47.7%) participants replied using well or pretty well, 14 (31.8%)partly, six (13.6%) not at all and two cannot say (1 missing).Q6: How well do the sub-areas 1 -6 differentiate from one another?Twenty-three (52.3%) participants replied using very well, well or pretty well while 19 (43.2%) replied partly or not at all (2 missing) (Figure 2).Two thirds of participants replied using very well, well or pretty well.The highest estimated sub-area was personal hygiene and secretion (sub-area 4) and the lowest estimated was nutrition and medication (sub-area 3) (Table 2).
Qualitative findings.Fourteen participants left written comments from which two categories were discerned: some unclear and time.In some unclear, participants noted that some NI levels were unclear and difficult to understand: there were only slight differences between the levels, making classification difficult; it was difficult to address nuances when selecting a level; and it was difficult to distinguish between levels C and D. A few mentioned that the instrument was not suited for use in HHC.In time, participants noted that they could not properly register the time they spend with patients, e.g., making phone calls to doctors or other authorities: "It is difficult to account for the More than half of the participants replied using very well, well or pretty well in regard to the question's three sub-categories: instrument instructions (manual's written instructions), concepts and support words, NI levels A-D (Table 3).Q9: In your opinion, has the training been sufficient?More than half of the participants replied using very well, well or pretty well in regard to the question's four sub-categories: electronic scheduling, OPCq as a method used to measure, sub-areas 1-6 and NI (Table 4).Q10: Has the training provided you with practical skills in the use of the OPCq instrument?
Twenty-six participants (59.1%) replied using very well, well or pretty well, thirteen (29.5%) replied partly or not at all and five (11.4%) cannot say.
Qualitative findings.Nine participants left written comments.A number considered the educational program to be good.

Q12: How motivated are you to classify patients' NI?
Twenty-six participants (59.1%) replied using motivated, very motivated or highly motivated and 17 (38.7%)partly motivated or not at all (1 missing).Additional analyses revealed a moderate correlation (0.36; p < 0.05) between Q12 (How motivated are you to classify patients' NI?) and Q13 (How do you like working in HHC?).While no correlation was seen between Q12 and work experience, PNs (a lower educational level) were more motivated than RNs (0.34; p < 0.05).
Qualitative findings.Eleven participants left written comments from which two categories were discerned: motivation and time.In motivation, participants noted that they were motivated to use the OPCq but that some technical problems (PC-to-instrument software compatibility, password issues) lowered their motivation.In time, participants mentioned a lack of time as one of the factors that made using the OPCq difficult: "The motivation is certainly present, but out of everything that should be done RAFAELA is prioritized last".Q13: Do you enjoy working in HHC?
The majority of participants 43 (97.7%)replied using very well, well or pretty well; only one replied using partly.
Qualitative findings.Thirteen participants left written comments from which two categories were discerned: working environment and relationship with the patients.
Participants mentioned positives including a good working environment of high professional quality, contact with the patients and a variable workday.Still some mentioned negatives, including a lack of time and high workload: "The environment is good, but the workload and intensity are too great after the new coordination reform."

Discussion
The RAFAELA system's OPCq instrument has been tested for the first time in an HHC setting in Norway.The content validity of the modified OPCq instrument, evaluated using a summative questionnaire, was estimated as being quite good.The modified OPCq instrument's sub-areas were overall assessed favorably (very well, well or pretty well), though some disagreement was seen.Sub-area 1 (Planning and coordination), sub-area 2 (Breathing, blood circulation and symptoms of disease) and sub-area 4 (Personal hygiene and secretion) were given the highest scores while sub-area 5 (Activity, sleep and rest) was given the lowest score.The sub-areas given the highest scores may be those areas that the participants feel confident classifying.Sub-area 5 may be difficult to classify, because of the short time spent with the patient.Furthermore, the low score given to sub-area 5 could result from that decisions related to activities and/or psychosocial needs are not common in the care of HHC patients.Instead, those care services directly related to illness/disease, elimination, medication or hygiene are common.
In this study, only day and evening shift nursing staff used the modified OPCq instrument.Some participants expressed a need for additional sub-areas, which may indicate that the instrument should be further adjusted for use in HHC.Participants specifically mentioned items such as support stockings, garbage, weather and driving conditions.While a few expressed that the sub-areas did not describe patients' NI at all, more than two-thirds considered the assessment of the NI levels A-D to be very well, well or pretty well.Still, participants indicated that some ambiguity exists between the levels, especially between C and D, which may reflect the lower level of education among the staff.Similar results were seen in a study by Frilund and Fagerström [4] in Finland.More than half assessed the OPCq manual as being practical and clear, but critique in regard to the written instructions may indicate that some adjustments may be needed.Participants assessed the educational programs lead by the FCG and the project leader as being good.
The participants were primarily RNs and PNs, with only one assistant without formal education.The educational level of nursing staff is relevant, because professional assessments often correspond to educational level [36] [37] [38].In Norwegian HHC, RNs and PNs often perform the same tasks and help with personal activities of daily living (PADL).While this reflects the HHC context, RNs are nonetheless more often responsible for acute care needs and specialized nursing interventions [39].The participants had quite a lot of work experience, which may be a benefit when implementing a new PCS.While the majority here were motivated to use the OPCq instrument, the PNs were more motivated than the RNs.A lack of time was mentioned as a negative factor, also seen in a study by Flöjt, Hir and Rosengren [40].Likewise, Gautun and Bratt [41] showed that when nurses experience great pressure in regard to time, not enough time could be given to individual patients.According to Tønnessen, Nortvedt and Førde [42], nurses ration care on a daily basis due to time constraints, consequently prioritizing medical and physiological needs over psychosocial and spiritual needs.This is not congruent with a person-centered approach [11], in which each patient's emotional, sociocultural and spiritual needs are supported.The OPCq, which does include a holistic approach congruent with a person-centered approach, includes emotional support and dialog in its sixth sub-area [14] [17] [18].A lack of time may negatively affect one's ability to engage in dialog or cooperation with patients and, as such, may increase a task oriented way of working [42] [43] [44].When the participants in this study felt that they did not have sufficient time to complete all tasks, they ceased prioritizing using the OPCq, which has an adverse effect.If patients' care needs are not systematically monitored, a correct and complete depiction of nurses' workload is not possible and the calculation of staff resources will, accordingly, be incorrect.
There was a high level of work satisfaction among the participants, despite their lack of time and a high workload.Nübling et al. [45] showed that HHC staff evaluate their psychosocial work situation more positively than other employees in professional geriatric care and that a high rate of part-time workers in HHC could affect results.While an indication of this was also seen in our study, exact data is unavailable.
This was a pilot study with relatively good results.The use of a PCS that measures NI is relatively new in an HHC setting, and it takes time to introduce a new system.Further clinical projects and research are needed to guarantee care and care results (outcome) and for the optimal allocation and calculation of nursing staff resources.
Based on the presented results, nurse leaders on varying levels in HHC can use the OPCq instrument, after some slight modifications, to measure and classify NI and as a workforce planning tool for nurse staffing.Use of the OPCq makes leaders aware of actual care needs and need for resources, but more focus should be placed on training nurse leaders to use systematic data in the allocation of nurse staffing resources.The shift from institutional to municipal-based care [7] [8] and the growing population of older people [2] make this essential.

Methodological Considerations
For more reliable results, a larger study is needed; this was a pilot study comprised of two HHC units with a limited number of participants.Due to low participant response in 2013, a new data collection was assessed as necessary and conducted in 2014.Technological problems such as poor mobile network coverage prevented the participants from using the mobile classification application, which caused stress and could thereby have affected the findings negatively.
One strength was that the summative questionnaire had been used in earlier studies [4] [17].The questionnaire used in a study in Finnish primary health care for older people [4] showed a reliability of 0.89 (Cronbach's Alpha), while the questionnaire used in this study showed a reliability of 0.96 (Cronbach's Alpha).Another strength is that open responses from self-completed questionnaires can complement frequency distribution.

Conclusion
The results showed that the modified OPCq instrument, one of two instruments' part of the RAFAELA Nursing Intensity and Staffing system developed for use in a hospital setting, seems to fulfill the requirements for validity in an HHC setting.However, the OPCq manual should be improved and some instrument aspects changed to better correspond to the specific needs in HHC.Based on the findings in this study, our recommendation is to improve the manual slightly to better adapt to HCC, both in regarding to sub areas 1 -6 and the NI levels A-D.It might be a need for more clearly defined levels A-D and keywords that are more suitable.Staff training and guidance are important when implementing a new PCS and that all technology works satisfactorily.Given that the complexity of care and the number of patients are increasing in HHC, further research is needed in regard to NI and the optimal allocation of nursing staff in an HHC setting.
, B., Hatlevik, O.E., Tønnessen, S. and Fagerström, L. (2016) Testing of the Content Validity of a Modified OPCq Instrument-A Pilot Study in Norwegian Home Health Care.Open Journal of Nursing, 6, 1012-1027.http://dx.doi.org/10.4236/ojn.2016.612097 nursing students from the University College collected data in spring 2014 through the use of structured interviews, with interviewers basing their questions on the same questionnaire previously used.Twenty-two participants responded this time.The questionnaires, sealed in a reply envelope, were returned to the external project leader/professor leading the research project.The main items in the questionnaire concerned background variables (age, gender, education and work experiences), questions about the sub areas 1 -6 and NI, education and training in OPCq classification and motivation to classify.All participants provided written informed consent for participation in the study and were informed that they could withdraw from the study at any time.

Table 1 .
Q2: In your opinion, how well are the sub-areas 1 -6 described in the OPCq instrument?

Table 2 .
Q4: In your opinion, how well are the NI levels A-D described in the following sub-areas in the OPCq instrument?
Q7: In your opinion, how practical and concrete is the OPCq instrument?

Table 3 .
Q7: In your opinion, how practical and concrete is the OPCq instrument?Very well/well Pretty well Partly/not at all n = 44 Mean Median SD

Table 4 .
Q9: In your opinion, has the training been sufficient?