Preliminary Receiver Operating Characteristic Analysis on Voice Handicap Index of Laryngeal Inflammation in Greek Patients

Dionysios Tafiadis¹, Meropi E. Helidoni², Spyridon K. Chronopoulos^3,4*, Evangelia I. Kosma^5,6, Vasiliki Liagkou⁴, Louiza Voniati⁷, Nafsika Ziavra¹, George A. Velegrakis⁸

¹Department of Speech & Language Therapy, Technological Educational Institute of Epirus, Ioannina, Greece.
²ENT Department, University of Crete, Crete, Iraklion, Greece.
³Department of Informatics and Telecommunications Engineering, University of Western Macedonia, Kozani, Greece.
⁴Department of Computer Engineering, Technological Educational Institute of Epirus, Arta, Greece.
⁵Psychologist, Private Practice, Mihail Aggelou 18, Ioannina, Greece.
⁶Faculty of Medicine, School of Health Sciences, University of Thessaly, Biopolis, Larissa, Greece.
⁷Department of Health Sciences, Speech and Language Therapy, European University, Nicosia, Cyprus.
⁸Department of Otolaryngology, University of Crete, Crete, Iraklion, Greece.

DOI: 10.4236/ijohns.2018.73014                

Abstract

Objective(s): Laryngeal inflammations lead to voice disorders. Medical conditions such as chronic laryngitis, gastroesophageal reflux, laryngopharyngeal reflux, Reinke edema and/or vocal folds hemorrhage, result in diverse symptoms including chronic cough, throat cleaning and dysphonia (e.g. hoarseness). In turn, the dysphonic symptoms can be evaluated via subjective and objective procedures. The objective procedures usually include self-perceived questionnaires like the Voice Handicap Index (VHI). Studies reported that VHI can distinguish objectively dysphonic and non-dysphonic populations using the cut-off points of Receiver Operating Characteristic Curves. The purpose of this study was to calculate the cut-off points for individuals exhibiting voice symptoms which had been developed from laryngeal inflammatory diseases in Greece. Methods: One hundred and twelve participants (90 non-dysphonic and 22 dysphonic) filled in the Hellenic Voice Handicap Index (VHI) and the Greek translated version of Voice Evaluation Template (VEF) were administrated. All subjects were evaluated by an Otolaryngologist and a Speech-Language Pathologist. Results: The group with voice disorders exhibited higher VHI scores (in total and in its 3 subdomains) compared to non-dysphonic subgroup. Statistical significant differences were found for all VHI’s total cut-off point of 19.50 (sensitivity: 0.882, 1-specificity: 0.011) and for its three subdomains [functional 6.50 (sensitivity = 0.636, and 1-specificity = 0.022); physical 9.50 (sensitivity = 0.636, and 1-specificity = 0.000); emotional 6.50 (sensitivity = 0.455, and 1-specificity = 0.133)]. Conclusion: The preliminary results showed that VHI could discriminate individuals having voice disorders from laryngeal inflammations. The Voice Handicap Index can be used as a primary health care tool and a self-monitoring procedure in acute and sub-acute phases of the laryngeal inflammation.

Keywords

Receiver Operating Characteristic, Statistical Analysis, Cut-Off Scores, Voice Handicap Index, Voice Disorders, Laryngeal Inflammatory Diseases

1. Introduction

The laryngeal pathogenesis mechanism, leading to voice problems, is complicated due to the contribution of sufficient aetiologies (anatomic, physical, habitual-functional, and/or environmental) [1] - [9] . A category of aetiologies is laryngeal inflammations which lead to voice disorders [10] - [24] . Examples of inflammatory medical conditions can be Gastroesophageal Reflux (GERD) [10] [11] [12] [13] [14] , Laryngopharyngeal Reflux (LRP) [12] [13] [14] [15] [16] , Reinke’s edema [17] [18] [19] and/or vocal folds haemorrhage [20] [21] [22] [23] [24] . The aforementioned, usually result in diverse symptoms including sore throat, hoarseness, chronic cough and throat cleaning [10] - [25] .

Those vocal and laryngeal symptoms affect a person’s daily living and soon enough he/she will have to seek for professional evaluation. The evaluation of voice disorders according to European Laryngeal Society (ELS) [26] , includes objective [27] - [36] and subjective [37] [38] assessment methods. Particularly, the subjective assessment is usually conducted with the use of self-reporting questionnaires which determine subjects’ standpoints for their voice disorder and its impact on their way of living [37] [38] .

In the literature, one of the most used self-perceived questionnaires is Voice Handicap Index (VHI) which was developed and standardized in 1997 by Jacobson et al. [39] . The VHI is a tool which acquires sufficient and evident data relevant to voice symptoms and the impact on the quality of life [39] . This data can be retrieved from the answers to 30 questions of VHI. The questionnaire is split into three sets of ten questions. Every question is based on a 0 to 4 Likert-type scale. Three sets of ten questions correspond to the three VHI’s subdomains (Functional, Physical and Emotional). The total score of VHI varies from 0 (minimum) to 120 points (maximum).

The VHI scale was standardized and cross-culturally validated in many languages [40] - [50] , including Greek [51] and recently in Danish [52] . Furthermore, VHI was administered in different populations [40] - [65] and it was proven that it could adequately distinguish between people of and without voice disorders [40] - [65] . This distinguishing capability has been reported in many studies using ROC analysis [66] [67] [68] [69] . The ROC curve methodology was developed during the World War II and was later adapted by health sciences for determining the threshold of symptoms-pathology due to different medical conditions [70] [71] . Based on the aforementioned significant findings and capabilities of ROC analysis, the aim of this study was to determine the cut-off points of VHI for patients experiencing voice disorders due to laryngeal inflammatory diseases in Greece.

2. Materials and Methods

2.1. Participants

The sample was consisted of one hundred and twelve (112) individuals. Particularly, ninety (90) individuals exhibited no vocal complaints (control group) and twenty-two (22) patients suffered from voice disorders due to laryngeal inflammatory disease (LID). The control group was assembled by patient’s caregivers and their accompanied persons, stuff and personnel of the ENT clinic and individuals attending the ENT department for other medical reasons. Moreover, were recruited people from the School of Health and Welfare Professions, TEI of Epirus. The LID group included 22 patients [ten (10) with Reinke’s edema, three (3) with vocal fold haemorrhage, and nine (9) with chronic laryngitis]. The majority of LID patients attended the ENT clinic of the University Hospital of Crete (15 subjects) and seven (7) subjects a private medical office and SLP office in Ioannina-Greece.

The patients of the LID subgroup, which had previous laryngeal surgery or had other recent surgeries, were not included in this study. All LID patients were diagnosed via video laryngeal endoscopy and stroboscopy. The subjects which served as the control group were recruited based on the absence of the following: 1) history of alcohol or drug abuse, 2) history of gastroesophageal reflux (GERD) and/or laryngopharyngeal reflux disease (LPR), 3) lived or still living under affecting environmental conditions for voice (smoke, dust, chemicals, etch.), 4) upper or lower respiratory system disorder the last two weeks, 5) any laryngeal/vocal symptoms or past voice disorder/s and 6) a previous voice therapy

The eighteen (18) participants, out of the one hundred and eight (108) non-dysphonic participants, did not meet the study’s inclusion criteria. The eleven (11) of them had a history of gastroesophageal reflux (GERD) and/or laryngopharyngeal reflux (LRP) disease with history of voice disorders in the past, three (3) cases had a history of alcohol abuse and in one (1) case the use of recreational drugs. Finally, three (3) cases had a history of exposure to noisy-dusty industrial environments.

2.2. Data Collection

All out of the one hundred and twelve (112) participants after signing a written consensus letter, they filled in the Voice Evaluation Template (VET) [72] (which had been translated in Greek) and the Hellenic version of VHI [48] . The VHI is a self-reported questionnaire which consists of 30 items, which are split into three domains as already aforementioned [functional (VHI-F), physical (VHI-P) and emotional (VHI-E)]. All three domains sub-scores (0 - 40 points equally) additively lead to the VHI total score (VHI-T) with a maximum of 120 points. The VHI assess persons’ perception level (awareness) of their voice problem’s impact on daily living. The VET is a voice history form originated from the American Speech Hearing Association consensus committee. The VET was used to collect clinical data about the participants. This study was approved by the Ethical Committee of the Medical School of the University of Crete.

2.3. Statistical Analysis

The distribution of the variables was examined using the Kolmogorov-Smirnov and Shapiro-Wilk tests. All skewed variables were expressed through medians (interquartile range) and all normal distributed variables were expressed with means and standard deviations. The comparison between the control group and the LID patients was conducted with Mann-Whitney U test. Finally, ROC curve analysis was employed in order to estimate the cut-off values for VHI and its three domains. All reported P values were two-tailed and the statistical significance was set at P < 0.005. The analysis was conducted using SPSS statistical software (version 19.0, Armonk, NY, USA).

3. Results

The sample consisted of 112 participants (90 controls and 22 LID). The LID group had a mean educational level of M = 11.27 yrs. (SD = 4.17) and the control group had a mean of 13.01 yrs. (SD = 5.09). The participants of the LID group smoked for an average of M = 12.23 yrs. (SD = 56.78) while the participants from the control group did not smoke [M = 0.00 (SD = 0.00)]. The most of the subjects were married and they lived in urban areas (Table 1).

Table 1. Participants demographic data.

The control group exhibited a median of VHI-T at 15.00, for VHI-F at 5.00, for VHI-P at 5.50 and for VHI-Ε at 5.00. The median score of NVD group’s VHI-T was found equal to 25.50, of VHI-F was found as 7.50, for VHI-P was 22.00 and of VHI-E was 16.00. The NVD exhibited higher scores in all comparisons to control group for VHI total score and its three domains: VHI-T [U = 467.000, P < 0.001]; VHI-F [U = 575.500, P < 0.005]; VHI-P [U = 519.500, P < 0.001 and for VHI-E (U = 699.500, P < 0.001) (Table 2). The results of VHI (Figure 1) and its three domains (Figures 2-4) are presented as box plots.

The determination of the cut-off points for VHI and its three domains was accomplished by conducting ROC analysis. A statistically significant positive discrimination between control group and NVD was revealed. Specifically, an excellent effect value was noted for VHI-T (AUC 0.764, P < 0.001), VHI-F (AUC 0.709, P < 0.005), VHI-P (AUC 0.738, P = 0.001) and VHI-E domain (AUC 0.647, P < 0.001) (Table 3).

The cut-off point of VHI-T score was equal to 19.50 with sensitivity of 0.636

Table 2. Comparisons of medians between controls and LID for VHI Total score and its domains.

*p level at P < 0.05. Abbreviations: IQR, interquartile range; VHI, Voice Handicap Index.

Figure 1. VHI box plot―total score of the two subgroups.

Figure 2. VHI box plot―functional domain’s score of the two subgroups.

Figure 3. VHI box plot―physical domain’s score of the two subgroups.

and 1-specificity of 0.022 (Figure 5), VHI-F cut-off point was equal to 6.50 with sensitivity of 0.591 and 1-specificity of 0.111 (Figure 6). Also, VHI-P cut-off point was equal to 9.50 with sensitivity of 0.636 and 1-specificity of 0.000 (Figure 7) while VHI-E cut-off point was equal to 6.50 with sensitivity of 0.455 and 1-specificity of 0.133 (Figure 8).

Figure 4. VHI box plot―emotional domain’s score of the two subgroups.

Table 3. Details of IVD and non-dysphonic curve for VHI Total score and its 3 domains.

*p level at P < 0.05. Abbreviations: AUC, area under curve; CI, confidence interval; SE, standard error; VHI, Voice Handicap Index.

Figure 5. ROC curve of Voice Handicap Index―Total Score (VHI-T).

Figure 6. ROC curve of Voice Handicap Index―Functional Domain (VHI-F).

Figure 7. ROC curve of Voice Handicap Index―Physical Domain (VHI-P).

Figure 8. ROC curve of Voice Handicap Index―Emotional Domain (VHI-E).

4. Discussion

This study showed that LID patients exhibited higher median scores in VHI compared to the control group. The same result was reported from other studies that compared diverse voice disordered population to non-dysphonic subjects [39] - [52] and other populations [60] [61] [62] [63] [64] . Particularly, the physical domain had the highest mean score in comparison to functional and emotional domain [39] - [52] . This is in agreement with other cross-validated versions of the VHI and probably reflects that patients are aware of their vocal physical symptoms better in contrast to emotional and functional symptoms [39] - [52] .

Additionally, the VHI scores for Greek LID patients with voice disorders are different in comparison to other studies [39] - [52] . Those differences, were expected since similarly results appeared in studies of populations with diverse vocal symptoms [39] - [59] . Furthermore, the aforementioned indicate the dynamics/ability of the questionnaire to obtain data about the impact of voice disorder on Quality of Life (QoL) from different real conditions that someone experiences [39] - [59] . Lastly, the above are consistent with other cross-cultural versions of VHI questionnaires [39] - [59] signifying the potentiality of the VHI being probably a strong screening tool [38] - [59] [68] [69] . According to the above is that Cohen, Dupont and Courey [73] in a Meta-analysis study presented that patients with inflammatory voice disorders exhibited more changes in their social functioning than patients without vocal symptoms. Albeit this comparison had no statistically difference between the two subgroups, nevertheless, the voice disordered individuals had a decrease in their social interaction due to inflammatory processes. Moreover, it is reported that populations, with different etiology of voice disorders, perceived variations in the level of QoL handicap when compared to those with LID [73] .

The cut-off point of VHI-T (19.50) for this study was within the range of 12 to 20 that other studies reported with satisfactory AUCs as this study did [68] [69] . Particularly, this study is in agreement with the Swedish [43] and Norwegian [47] versions of VHI with VHI-T scores of 20 (sensitivity of 0.77 and 1-specificity of 0.87) and 19 (95% sensitivity) respectively. Likewise, Solomon et al. [59] set a cut-off value of VHI total score of 18 points for people with organic dysphonia after thyroidectomy. Moreover, Tafiadis et al. found a cut-off point of 19.5 [68] [69] for female and male smokers which is equal to the estimated score of this study.

Furthermore, other researchers proposed lower VHI-T cut-off scores which were not in agreement to the score of this preliminary study. Gräßel et al. [42] (for the German Version of VHI) calculated a cut-off value of 12 points. Equally, for the Polish version of VHI, Niebudek-Bogusz et al. suggested a VHI-T of 12 points (sensitivity = 0.98 and 1-specificity = 0.95) [46] . Similarly, to the above Behrman, Rutledge, Hembree and Sheridan (which studied only women under combined therapeutic program) computed a VHI-T cut-off score of 11.5 [53] which is obviously lower in comparison to the proposed cut-off value of this research for LID patients. Also, for the Persian version of VHI, the cut-off value was equal to 14.5 (sensitivity = 92% and 1-specificity = 95%) which has again not in agreement to the score of this research [67] . Lastly, Van Gogh et al. [65] set a threshold of VHI-T equal to 15 (sensitivity = 0.97 and 1-specificity = 0.86) by comparing patients, exhibiting cancer in glottic area, to subjects with laryngeal lesions of early stages. The aforementioned conflicts of scores are probably due to recruitment differences (size of sample, diverse voice pathologies and different cross-cultural adaptations) of the studied populations [47] .

Finally, as many studies calculated the cut-off points of VHI total score [42] [43] [46] [47] [53] [59] [65] [66] [67] [68] [69] , this research is one of the few that computed and discussed thresholds based on ROC analysis for the three VHI domains (VHI-F, VHI-P, VHI-E) as other studies did [68] [69] . Tafiadis et al. underlined the adequate diagnostic value of the three domains [68] [69] . Similarly, Karlsen et al. [47] underlined that VHI-F seems to have good discrimination between dysphonic and non-dysphonic patients. Additionally, Sorensen et al. [53] in their study suggested that VHI-F and VHI-P has also this probable discriminant ability. The above results are in agreement to this research. Nevertheless, this study is the only one so far which provides and has already processed the data regarding the group of LID patients with vocal symptoms. Nevertheless, the daily clinical practice needs more specialization (i.e. the health professionals need more evidence based data). In turn, this preliminary research delivers to an adequate point the previous necessary information. This information will lead to better screening and monitoring of LID populations as it is already suggested by other studies regarding other populations [68] [69] .

5. Conclusion

The main aim of this study was to estimate the VHI’s total cut-off point as well as the cut-off points of the three subdomains for dysphonic populations with laryngeal inflammatory diseases. As it is shown, VHI can distinguish the perception scores between the LID dysphonic and non-dysphonic populations. Additionally, it can serve as a screening tool and it could provide data for better re-evaluation and therapy planning. The VHI’s cut-off points for LID patients have the limited validity of the small sample of this study. Further research is suggested to be conducted in larger populations with LID. In the long run, this could help in order to specify better and to enrich the VHI’s thresholds for LID patients with different etiologies. Also, a future application would be the electronic screening evaluation of potential voice disordered patients [74] by using automatic decision making based on this study’s results. This technology could be even implemented inside LEED buildings [75] utilizing enhanced wireless and electronic technologies [76] [77] [78] .

Conflict of Interest

All authors declare no conflicts of interest in this manuscript.

Abbreviations List

ASHA = American Speech Hearing Association

ENT = Ear Nose Throat

ELS = European Laryngeal Society

GERD = Gastroesophageal Reflux

LID = Laryngeal Inflammatory Disorders

LRP = Laryngopharyngeal Reflux

ROC = Receiver Operating Characteristic

SLP = Speech Language Pathologist

TEI of Epirus = Technological Educational Institute of Epirus

VET = Voice Evaluation Template

VHI = Voice Handicap Index

VHI-E = Voice Handicap Index―Emotional

VHI-P = Voice Handicap Index―Physical

VHI-F = Voice Handicap Index―Functional

VHI-T = Voice Handicap Index―Total score

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Awan, S.N. and Morrow, D.L. (2007) Videostroboscopic Characteristics of Young Adult Female Smokers vs. Nonsmokers. Journal of Voice, 21, 211-223. https://doi.org/10.1016/j.jvoice.2005.10.009
[2]	Damborenea, J.T., Fernández, R.L., Llorente, E.A., Naya, M.G., Marín, C.G., Rueda, P.G. and Ortiz, A.G. (1999) The Effect of Tobacco Consumption on Acoustic Voice Analysis. Acta Otorrinolaringologica Espanola, 50, 448-452.
[3]	Gonzalez, J. and Carpi, A. (2004) Early Effects of Smoking on the Voice: A Multidimensional Study. Medical Science Monitor, 10, CR649-CR656. http://www.MedSciMonit.com/pub/vol_10/no_12/4738.pdf
[4]	Roy, N., Merrill, R.M., Thibeault, S., Parsa, R.A., Gray, S.D. and Smith, E.M. (2004) Prevalence of Voice Disorders in Teachers and the General Population. Journal of Speech, Language, and Hearing Research, 47, 281-293. https://doi.org/10.1044/1092-4388(2004/023)
[5]	Sala, E., Laine, A., Simberg, S., Pentti, J. and Suonpää, J. (2001) The Prevalence of Voice Disorders among Day Care Center Teachers Compared with Nurses: A Questionnaire and Clinical Study. Journal of Voice, 15, 413-423. https://doi.org/10.1016/S0892-1997(01)00042-X
[6]	Titze, I.R., Lemke, J. and Montequin, D. (1997) Populations in the US Workforce Who Rely on Voice as a Primary Tool of Trade: A Preliminary Report. Journal of Voice, 11, 254-259. https://doi.org/10.1016/S0892-1997(97)80002-1
[7]	Vilkman, E. (2004) Occupational Safety and Health Aspects of Voice and Speech Professions. Folia Phoniatrica et Logopaedica, 56, 220-253. https://doi.org/10.1159/000078344
[8]	Rengaraja, D., et al. (2017) Primary Laryngeal Amyloidosis: A Case Report. International Journal of Otolaryngology and Head & Neck Surgery, 6, 1. https://doi.org/10.4236/ijohns.2017.61001
[9]	Yiotakis, I., Georgolios, A., Charalabopoulos, A., Hatzipantelis, P., Golias, C., Charalabopoulos, K. and Manolopoulos, L. (2009) Primary Localized Laryngeal Amyloidosis Presenting with Hoarseness and Dysphagia: A Case Report. Journal of Medical Case Reports, 3, 1. https://doi.org/10.4076/1752-1947-3-9049
[10]	Deveney, C.W., Benner, K. and Cohen, J. (1993) Gastroesophageal Reflux and Laryngeal Disease. Archives of Surgery-New York, 128, 1021-1021. https://doi.org/10.1001/archsurg.1993.01420210085011
[11]	Hamdan, L., Sharara, A.I., Younes, A. and Fuleihan, N. (2001) Effect of Aggressive Therapy on Laryngeal Symptoms and Voice Characteristics in Patients with Gastroesophageal Reflux. Acta Oto-Laryngologica, 121, 868-872. https://doi.org/10.1080/00016480152602357
[12]	Ramzy, I., El Shazly, M., Marzaban, R., Elbaz, T., Safwat, M. and Latif, B. (2014) Laryngopharengeal Reflux in Gastroesophageal Reflux Disease: Does “Silent Laryngopharengeal Reflux” Really Exist? Open Journal of Gastroenterology, 4, 130. https://doi.org/10.4236/ojgas.2014.43020
[13]	Tauber, S., Gross, M. and Issing, W.J. (2002) Association of Laryngopharyngeal Symptoms with Gastroesophageal Reflux Disease. The Laryngoscope, 112, 879-886. https://doi.org/10.1097/00005537-200205000-00019
[14]	Toohill, R.J. and Kuhn, J.C. (1997) Role of Refluxed Acid in Pathogenesis of Laryngeal Disorders. The American Journal of Medicine, 103, 100S-106S. https://doi.org/10.1016/S0002-9343(97)00333-1
[15]	Belafsky, P.C., Postma, G.N. and Koufman, J.A. (2001) Laryngopharyngeal Reflux Symptoms Improve before Changes in Physical Findings. Laryngoscope, 111, 979-981. https://doi.org/10.1097/00005537-200106000-00009
[16]	Khan, A.M., Hashmi, S.R., Elahi, F., Tariq, M. and Ingrams, D.R. (2006) Laryngopharyngeal Reflux: A Literature Review. The Surgeon, 4, 221-225. https://doi.org/10.1016/S1479-666X(06)80063-6
[17]	Marcotullio, D., Magliulo, G. and Pezone, T. (2002) Reinke’s Edema and Risk Factors: Clinical and Histopathologic Aspects. American Journal of Otolaryngology, 23, 81-84. https://doi.org/10.1053/ajot.2002.30961
[18]	Mohyuddin, A. and Sandhu, G. (2017) Reinke’s Oedema Presenting as Stridor: Implications for Otolaryngologists in Difficult Airway. Head & Neck Surgery, 6, 11-15. https://doi.org/10.4236/ijohns.2017.62003
[19]	Yonekawa, H. (1988) A Clinical Study of Reinke’s Edema. Auris Nasus Larynx, 15, 57-78. https://doi.org/10.1016/S0385-8146(88)80010-5
[20]	Abitbol, J. (1988) Vocal Cord Hemorrhages in Voice Professionals. Journal of Voice, 2, 261-266. https://doi.org/10.1016/S0892-1997(88)80084-5
[21]	Lennon, C.J., Murry, T. and Sulica, L. (2014) Vocal Fold Hemorrhage: Factors Predicting Recurrence. The Laryngoscope, 124, 227-232. https://doi.org/10.1002/lary.24242
[22]	Lin, P.T., Stern, J.C. and Gould, W.J. (1991) Risk Factors and Management of Vocal Cord Hemorrhages: An Experience with 44 Cases. Journal of Voice, 5, 74-77. https://doi.org/10.1016/S0892-1997(05)80166-3
[23]	Neely, J.L. and Rosen, C. (2000) Vocal Fold Hemorrhage Associated with Coumadin Therapy in an Opera Singer. Journal of Voice, 14, 272-277. https://doi.org/10.1016/S0892-1997(00)80035-1
[24]	Postma, G.N., Courey, M.S. and Ossoff, R.H. (1998) Microvascular Lesions of the True Vocal Fold. Annals of Otology, Rhinology & Laryngology, 107, 472-476. https://doi.org/10.1177/000348949810700604
[25]	Branski, R.C., Bhattacharyya, N. and Shapiro, J. (2002) The Reliability of the Assessment of Endoscopic Laryngeal Findings Associated with Laryngopharyngeal Reflux Disease. The Laryngoscope, 112, 1019-1024. https://doi.org/10.1097/00005537-200206000-00016
[26]	Dejonckere, P., Bradley, P., Clemente, P., Cornut, G., Crevier-Buchman, L., Friedrich, G., et al. (2001) A Basic Protocol for Functional Assessment of Voice Pathology, Especially for Investigating the Efficacy of (Phonosurgical) Treatments and Evaluating New Assessment Techniques. European Archives of Oto-Rhino-Laryngology, 258, 77-82. https://doi.org/10.1007/s004050000299
[27]	Emilia Elias, M., Thayer Sataloff, R., Rosen, D., Heuer, R. and Spiegel, J. (1997) Normal Strobovideolaryngoscopy: Variability in Healthy Singers. Journal of Voice, 11, 104-107. https://doi.org/10.1016/S0892-1997(97)80030-6
[28]	Gaskill, C.S., Awan, J.A., Watts, C.R. and Awan, S.N. (2017) Acoustic and Perceptual Classification of Within-Sample Normal, Intermittently Dysphonic, and Consistently Dysphonic Voice Types. Journal of Voice, 31, 218-228. https://doi.org/10.1016/j.jvoice.2016.04.016
[29]	Joshi, A. and Watts, C.R. (2017) Phonation Quotient in Women: A Measure of Vocal Efficiency Using Three Aerodynamic Instruments. Journal of Voice, 31, 161-167. https://doi.org/10.1016/j.jvoice.2016.06.007
[30]	Lovato, A., De Colle, W., Giacomelli, L., Piacente, A., Righetto, L., Marioni, G. and de Filippis, C. (2016) Multi-Dimensional Voice Program (MDVP) vs Praat for Assessing Euphonic Subjects: A Preliminary Study on the Gender-Discriminating Power of Acoustic Analysis Software. Journal of Voice, 30, 765.e1-765.e5. https://doi.org/10.1016/j.jvoice.2015.10.012
[31]	Rosen, C.A., Lombard, L.E. and Murry, T. (2000) Acoustic, Aerodynamic, and Videostroboscopic Features of Bilateral Vocal Fold Lesions. Annals of Otology, Rhinology & Laryngology, 109, 823-828. https://doi.org/10.1177/000348940010900907
[32]	Sapienza, C. (1997) Aerodynamic and Acoustic Characteristics of the Adult African American Voice. Journal of Voice, 11, 410-416. https://doi.org/10.1016/S0892-1997(97)80036-7
[33]	Sataloff, R.T., Mandel, S., Mann, E.A. and Ludlow, C.L. (2003) Laryngeal Electromyography: An Evidence-Based Review. Muscle & Nerve, 28, 767-772. https://doi.org/10.1002/mus.10503
[34]	Sataloff, R., Spiegel, J., Carroll, L., Schiebel, B., Darby, K. and Rulnick, R. (1988) Strobovideolaryngoscopy in Professional Voice Users: Results and Clinical Value. Journal of Voice, 1, 359-364. https://doi.org/10.1016/S0892-1997(88)80012-2
[35]	Woo, P. (1996) Quantification of Videostrobolaryngoscopic Findings-Measurements of the Normal Glottal Cycle. The Laryngoscope, 106, 1-27. https://doi.org/10.1097/00005537-199603001-00001
[36]	Yamasaki, R., Madazio, G., Leão, S.H., Padovani, M., Azevedo, R. and Behlau, M. (2017) Auditory-Perceptual Evaluation of Normal and Dysphonic Voices Using the Voice Deviation Scale. Journal of Voice, 31, 67-71. https://doi.org/10.1016/j.jvoice.2016.01.004
[37]	Deary, I., Wilson, J., Carding, P. and MacKenzie, K. (2003) VoiSS: A Patient-Derived Voice Symptom Scale. Journal of Psychosomatic Research, 54, 483-489. https://doi.org/10.1016/S0022-3999(02)00469-5
[38]	Hogikyan, N.D. and Sethuraman, G. (1999) Validation of an Instrument to Measure Voice-Related Quality of Life (V-RQOL). Journal of Voice, 13, 557-569. https://doi.org/10.1016/S0892-1997(99)80010-1
[39]	Jacobson, B.H., Johnson, A., Grywalski, C., Silbergleit, A., Jacobson, G., Benninger, M.S. and Newman, C.W. (1997) The Voice Handicap Index (VHI): Development and Validation. American Journal of Speech-Language Pathology, 6, 66-70. https://doi.org/10.1044/1058-0360.0603.66
[40]	Behlau, M., dos Santos, L.D.M.A. and Oliveira, G. (2011) Cross-Cultural Adaptation and Validation of the Voice Handicap Index into Brazilian Portuguese. Journal of Voice, 25, 354-359. https://doi.org/10.1016/j.jvoice.2009.09.007
[41]	Bonetti, A. and Bonetti, L. (2013) Cross-Cultural Adaptation and Validation of the Voice Handicap Index into Croatian. Journal of Voice, 27, 130.e7-130.e14. https://doi.org/10.1016/j.jvoice.2012.07.006
[42]	Gräßel, E., Hoppe, U. and Rosanowski, F. (2007) Graduierung des Voice-Handicap-Index. HNO, 56, 1221-1228. https://doi.org/10.1007/s00106-007-1594-9
[43]	Karlsen, T., Grieg, A.R.H., Heimdal, J.H. and Aarstad, H.J. (2012) Cross-Cultural Adaption and Translation of the Voice Handicap Index into Norwegian. Folia Phoniatrica et Logopaedica, 64, 234-240. https://doi.org/10.1159/000343080
[44]	Malki, K.H., Mesallam, T.A., Farahat, M., Bukhari, M. and Murry, T. (2010) Validation and Cultural Modification of Arabic Voice Handicap Index. European Archives of Oto-Rhino-Laryngology, 267, 1743-1751. https://doi.org/10.1007/s00405-010-1296-x
[45]	Moradi, N., Pourshahbaz, A., Soltani, M., Javadipour, S., Hashemi, H. and Soltaninejad, N. (2013) Cross-Cultural Equivalence and Evaluation of Psychometric Properties of Voice Handicap Index into Persian. Journal of Voice, 27, 258.e15-258.e22. https://doi.org/10.1016/j.jvoice.2012.09.006
[46]	Niebudek-Bogusz, E., Kuzańska, A., Woznicka, E. and Sliwinska-Kowalska, M. (2011) Assessment of the Voice Handicap Index as a Screening Tool in Dysphonic Patients. Folia Phoniatrica et Logopaedica, 63, 269-272. https://doi.org/10.1159/000324214
[47]	Ohlsson, A.C. and Dotevall, H. (2009) Voice Handicap Index in Swedish. Logopedics Phoniatrics Vocology, 34, 60-66. https://doi.org/10.1080/14015430902839185
[48]	Schindler, A., Ottaviani, F., Mozzanica, F., Bachmann, C., Favero, E., Schettino, I. and Ruoppolo, G. (2010) Cross-Cultural Adaptation and Validation of the Voice Handicap Index into Italian. Journal of Voice, 24, 708-714. https://doi.org/10.1016/j.jvoice.2009.05.006
[49]	Taguchi, A., Mise, K., Nishikubo, K., Hyodo, M. and Shiromoto, O. (2012) Japanese Version of Voice Handicap Index for Subjective Evaluation of Voice Disorder. Journal of Voice, 26, 668.e15-668.e19. https://doi.org/10.1016/j.jvoice.2011.11.005
[50]	Trinite, B. and Sokolovs, J. (2014) Adaptation and Validation of the Voice Handicap Index in Latvian. Journal of Voice, 28, 452-457. https://doi.org/10.1016/j.jvoice.2014.01.008
[51]	Helidoni, M.E., Murry, T., Moschandreas, J., Lionis, C., Printza, A. and Velegrakis, G.A. (2010) Cross-Cultural Adaptation and Validation of the Voice Handicap Index into Greek. Journal of Voice, 24, 221-227. https://doi.org/10.1016/j.jvoice.2008.06.005
[52]	Sorensen, J.R., Printz, T., Mehlum, C.S., Heidemann, C.H., Groentved, A.M. and Godballe, C. (2018) Cross-Cultural Adaption and Validation of the Danish Voice Handicap Index. Journal of Voice, in press. https://doi.org/10.1016/j.jvoice.2018.01.010
[53]	Behrman, A., Rutledge, J., Hembree, A. and Sheridan, S. (2008) Vocal Hygiene Education, Voice Production Therapy, and the Role of Patient Adherence: A Treatment Effectiveness Study in Women with Phonotrauma. Journal of Speech, Language, and Hearing Research, 51, 350-366. https://doi.org/10.1044/1092-4388(2008/026)
[54]	Bouwers, F. and Dikkers, F.G. (2009) A Retrospective Study Concerning the Psychosocial Impact of Voice Disorders: Voice Handicap Index Change in Patients with Benign Voice Disorders after Treatment (Measured with the Dutch Version of the VHI). Journal of Voice, 23, 218-224. https://doi.org/10.1016/j.jvoice.2007.08.007
[55]	da Rocha, L.M. and de Mattos Souza, L.D. (2013) Voice Handicap Index Associated with Common Mental Disorders in Elementary School Teachers. Journal of Voice, 27, 595-602. https://doi.org/10.1016/j.jvoice.2012.10.001
[56]	Meulenbroek, L.F.P., Van Opstal, M.J.C.M., Claes, L., Marres, H.A.M. and de Jong, F.I.C.R.S. (2012) The Impact of the Voice in Relation to Psychosomatic Well-Being after Education in Female Student Teachers: A Longitudinal, Descriptive Study. Journal of Psychosomatic Research, 72, 230-235. https://doi.org/10.1016/j.jpsychores.2011.11.016
[57]	Musio, D., Felice, F., Bigelli, C., Bulzonetti, N., Guarnaccia, R., Tombolini, M., et al. (2014) An Analysis of the Global and Partial Voice Handicap Index in Patients with Early Glottic Carcinoma Treated with High Level of Irradiation. Journal of Cancer Therapy, 5, 21-27. https://doi.org/10.4236/jct.2014.51003
[58]	Rosen, C.A., Murry, T., Zinn, A., Zullo, T. and Sonbolian, M. (2000) Voice Handicap Index Change Following Treatment of Voice Disorders. Journal of Voice, 14, 619-623. https://doi.org/10.1016/S0892-1997(00)80017-X
[59]	Solomon, N.P., Helou, L.B., Henry, L.R., Howard, R.S., Coppit, G., Shaha, A.R. and Stojadinovic, A. (2013) Utility of the Voice Handicap Index as an Indicator of Postthyroidectomy Voice Dysfunction. Journal of Voice, 27, 348-354. https://doi.org/10.1016/j.jvoice.2012.10.012
[60]	Tafiadis, D., Chronopoulos, S.K., Siafaka, V., Drosos, K., Kosma, E.I., Toki, E.I. and Ziavra, N. (2017) Comparison of Voice Handicap Index Scores between Female Students of Speech Therapy and Other Health Professions. Journal of Voice, 31, 583-588. https://doi.org/10.1016/j.jvoice.2017.01.013
[61]	Tafiadis, D., Kosma, E.I., Chronopoulos, S.K., Papadopoulos, A., Toki, E.I., Siafaka, V. and Ziavra, N. (2017) Acoustic and Perceived Measurements Certifying Tango as Voice Treatment Method. Journal of Voice, 32, 256.e13-256.e24. https://doi.org/10.1016/j.jvoice.2017.05.016
[62]	Tafiadis, D., Tatsis, G., Ziavra, N. and Toki, E.I. (2017) Voice Data on Female Smokers: Coherence between the Voice Handicap Index and Acoustic Voice Parameters. AIMS Medical Science, 4, 151-163. https://doi.org/10.3934/medsci.2017.2.151
[63]	Tafiadis, D., Toki, E.I., Miller, K.J. and Ziavra, N. (2017) Effects of Early Smoking Habits on Young Adult Female Voices in Greece. Journal of Voice, 31, 728-732. https://doi.org/10.1016/j.jvoice.2017.03.012
[64]	Toki, E.I., Plachouras, K., Tatsis, G., Chronopoulos, S.K., Tafiadis, D., Ziavra, N. and Siafaka, V. (2017) The Design of a Mobile System for Voice e-Assessment and Vocal Hygiene e-Training. In: Auer, M. and Tsiatsos, T., Eds., Interactive Mobile Communication Technologies and Learning. IMCL 2017. Advances in Intelligent Systems and Computing, Vol. 725, Springer, Cham, 167-174.
[65]	Van Gogh, C.L., Mahieu, H.F., Kuik, D.J., Rinkel, R.N., Langendijk, J.A. and Verdonck-de Leeuw, I.M. (2007) Voice in Early Glottic Cancer Compared to Benign Voice Pathology. European Archives of Oto-Rhino-Laryngology, 264, 1033-1038. https://doi.org/10.1007/s00405-007-0313-1
[66]	Behlau, M., Madazio, G., Moreti, F., Oliveira, G., dos Santos, L.D.M.A., Paulinelli, B.R. and de Barros Couto Junior, E. (2016) Efficiency and Cutoff Values of Self-Assessment Instruments on the Impact of a Voice Problem. Journal of Voice, 30, 506.e9-506.e18. https://doi.org/10.1016/j.jvoice.2015.05.022
[67]	Moradi, N., Pourshahbaz, A., Soltani, M. and Javadipour, S. (2013) Cutoff Point at Voice Handicap Index Used to Screen Voice Disorders among Persian Speakers. Journal of Voice, 27, 130.e1-130.e5. https://doi.org/10.1016/j.jvoice.2012.08.007
[68]	Tafiadis, D., Kosma, E., Chronopoulos, S., Papadopoulos, A., Drosos, K., Siafaka, V., et al. (2018) Voice Handicap Index and Interpretation of the Cutoff Points Using Receiver Operating Characteristic Curve as Screening for Young Adult Female Smokers. Journal of Voice, 32, 64-69. https://doi.org/10.1016/j.jvoice.2017.03.009
[69]	Tafiadis, D., Chronopoulos, S.K., Kosma, E.I., Voniati, L., Raptis, V., Siafaka, V. and Ziavra, N. (2017) Using Receiver Operating Characteristic Curve to Define the Cutoff Points of Voice Handicap Index Applied to Young Adult Male Smokers. Journal of Voice, in press. https://doi.org/10.1016/j.jvoice.2017.06.007
[70]	Lehr, R. and Pong, A. (2003) ROC Curve. In: Encyclopedia of Biopharmaceutical Statistics, Informa Healthcare.
[71]	Morrison, A.M. (2005) Receiver Operating Characteristic (ROC) Curve Preparation—A Tutorial. Massachusetts Water Resources Authority, Environmental Quality Department.
[72]	American Speech Hearing Association. Voice Evaluation Template. https://www.asha.org/uploadedFiles/slp/healthcare/AATVoiceEvaluation.pdf
[73]	Cohen, S.M., Dupont, W.D. and Courey, M.S. (2006) Quality-of-Life Impact of Non-Neoplastic Voice Disorders: A Meta-Analysis. Annals of Otology, Rhinology & Laryngology, 115, 128-134. https://doi.org/10.1177/000348940611500209
[74]	Tafiadis, D. (2016) Study of Communication Disorders in Patients with Different Types of Dementia with Use of Speech and Language Scales and Assistance from Modern Neuroimaging Methods. Ph.D. Thesis, University of Ioannina, Ioannina. http://olympias.lib.uoi.gr/jspui/handle/123456789/27985
[75]	Chronopoulos, S.K., Kosma, E.I., Tafiadis, D., Dimopoulos, D., Raptis, V., Karvounis, E.C., Angelidis, P. and Kostarakis, P. (2018) Reduced Ecological Footprints of Modern Facilities Introducing the Implementation of Advanced Wireless Technologies, and Human Resources’ Benefits. Communications and Network, 10, 11-29. https://doi.org/10.4236/cn.2018.101002
[76]	Chronopoulos, S.K., Christofilakis, V., Tatsis, G. and Kostarakis, P. (2016) Performance of Turbo Coded OFDM under the Presence of Various Noise Types. Wireless Personal Communications, 87, 1319-1336. https://doi.org/10.1007/s11277-015-3055-1
[77]	Chronopoulos, S.K., Tatsis, G. and Kostarakis, P. (2012) Turbo Coded OFDM with Large Number of Subcarriers. Journal of Signal and Information Processing, 3, 161-168. https://doi.org/10.4236/jsip.2012.32021
[78]	Chronopoulos, S.K. (2014). Study and Impact Considerations of Various Aspects in OFDM Signals. Ph.D. Thesis, University of Ioannina, School of Sciences, Department of Physics, Electronics-Telecommunications and Applications Laboratory. http://olympias.lib.uoi.gr/jspui/handle/123456789/5651 http://dx.doi.org/10442/hedi/37578