Quality of Service (QoS) Evaluation in Wideband Code Division Multiplexing Access (WCDMA) Networks Using Empirical Analysis Methods

Gloria A. Chukwudebe¹, Emmanuel N. Ekwonwune², Florence O. Elei²

¹Department of Electrical and Electronics Engineering, Federal University of Technology, Owerri, Nigeria.
²Department of Computer Science, Imo State University, Owerri, Nigeria.

DOI: 10.4236/cn.2020.122004                

Abstract

Performance evaluation is essential in maintaining the Quality of Service (QOS) of the Wideband Code Division Multiplexing Access (WCDMA). This work was motivated by the reception of the poor signals, increase call drop, failure rate which was a poor QoS Reception. The aim is to survey WCDMA services in Owerri environs and establish that there are degradation and the level of debasement in the network. The methodology involved an Empirical Analysis through Drive Test across Owerri City in Imo State. The work adopted the empirical approach and deduction of some Standard performance metrics like call drop rate, failure rate, call success rate, call completion rate, Handover success rate and handover Failure Rate, compare with expected KPI(key performance indicator) threshold. From the assessment, it was found that only one out Four Networks (“GLO”) met the target Call Drop Rate (CDR), Call Completion Success rate (CCSR), Call Setup Success Rate (CSSR) and Call Blocked Rate (CBR) and the Handover was better in “GLO” and 9 mobile than in the “MTN” and Airtel.

Keywords

WCDMA, KPI, Drive Test, QoS

1. Introduction

The telecommunication industry around the world is facing notable challenges from the unending customer demands and competition from other operators.

Customer satisfaction is critical to gain a sustainable competitive edge in the market [1]. The increasing need for efficient high-speed internet and multimedia application further resulted in the invention of Universal Mobile Telecommunication Services (UMTS), a 3rd Generation network (3G), widely known as Wideband Code Division Multiple Access (WCDMA) operating at 2.1 GHz [2].

WCDMA covers a larger area and provides greater capacity and a greater number of variable parameters to monitor. Information obtained from Nigeria Communication Commission (NCC) report: 2012, 2017, 2018 on the four networks (MTN, GLO, 9 Mobile, and Airtel) established the fact that the WCDMA network is currently experiencing some challenges such as poor accessibility and retain ability. In the telecommunication industry, network performance is measured using some key performance indicators (KPI). KPI is the pressure unit metrics commonly used to evaluate and define how good/successful a Network is. KPI is used to consider the QoS structure of network through some metrics such as call drop rate (CDR), Call Competition Success Rate (CCSR), Call Success Set up Ratio (CSSR), Network Accessibility Ratio (NAR), Network Retainability Ratio (NRR), recall termination rate (CTR), etc. The performance of network and service quality as frequently accessed with the drive test method [1].

In this research work, the drive test was conducted for the four (4) Networks in some part of Owerri Municipal and Owerri North (the Primary Research Area in Imo State University and its Environments) is densely populated during the busy hours. This study deployed KPI to evaluate the quality of service for the four Networks using this KPI metrics such as Call Drop Rate (CDR), Call Blocked Rate (CBR), Call Setup Success Rate (CSSR), Drop Call Rate (DCR), Call Success Rate (CSSR), Call Retainability (CR) and Call Handover Rate (CHR).

This work will be significant to the NCC’s performance evaluations of the Network operators since four (4) Networks in Owerri Municipal and Owerri North were evaluated and the Recommendation of an enhanced technique as a template for improving the overall performance of the Network based on the KPI evaluations.

2. Literature Review

Key Performance Indicators (KPI)

A Key Performance indicator (KPI) is a Metric used to measure Network performance. KPI measurements are commonly used by the organization to define and evaluate QoS. It is typical gear toward Operator’s long-term organizational goal. Some selected KPI’s that are used for voice measurement will be discussed [2].

Call Success Setup Ratio (CSSR)

This is the ratio of the total number of completed calls to the total number of call attempts.

$\text{CCSR}=\frac{\text{Successfully completed callsetups}}{\text{Call setups attempts}}\times 100$ (2.1)

Call Completion Success Rate (CCSR)

It is a parameter for evaluating the network accessibility, retainability and can be deduced from drive test statistics. The indicator is derived using the expression:

$\text{CCSR}=\frac{\text{Total Number of Completed Call}}{\text{Total Nummber of Call Attempted}}\times 100$ (2.2)

Call Dropped Rate (CDR)

It is the total number of calls dropped (not ending as desired by the user) to the total Call Set up

$\text{CDR}=\frac{\text{Number of dropped call}}{\text{Number of Successully Completed Call Setup}}\times 100$ (2.3)

Call Block Rate (CBR)

This is the ratio of the total number of blocked calls to the total number of call attempts.

$\text{CBR}=\frac{\text{Number of Call blocked}}{\text{Total Nummber of Call Attempted call}}\times 100$ (2.4)

Handover Success Rate (HOSR)

This is the ratio of successful handover call attempts. This KPI is intended to indicate successful user mobility in the network.

$\text{HOSR}=\frac{\text{Number of handover Call}}{\text{Total Nummber of Attempted handover Call}}\times 100$ (2.5)

Received Signal Level (RXLEV)

This is referred to as the received signal level at the input of the mobile device. For the WCDMA network, it is in RSCP and Table 1 shows the RSCP rating for the WCDMA signal.

$\text{RSCP}=\frac{\text{Summation of all data that fall in}>-85\text{dBm}}{\text{Summation of all data}}\times 100$ (2.6)

Endpoints Service Availability (ESA)

This indicator measures service availability, which is the QoS during service use. It is defined as the percentage of time a usable call can be established and maintained between two endpoints.

$\text{ESA}=\frac{\text{NAttempts}-\text{Nblock}-\text{Ndrop}}{\text{Nattempts}}\times 100$ (2.7)

where,

Nattempts = No. of call attempts;

Nblock = No. of call blocks;

Ndrop = No. of call drop.

Table 1. RSCP rating for WCDMA signal.

Network Accessibility Ratio (NAR)

This is the probability that a mobile user will establish successful voice communications between the two ends of the network within a given condition. It is also expressed as the call setup rate. It is expressed as:

$\text{NAR}=\frac{\text{Number of successul call setup}}{\text{Number of Call Attempts}}\times 100$ (2.8)

Network Retainability Ratio (NRR)

It is the ratio between the number of successful calls and the number of normally terminated calls. It is also the probability that an active call comes to an end successfully in a network.

$\left(\text{1}-\text{CDR}\right)\times 100$ (2.9)

where, CDR is defined in Equation (2.3)

Call Success Failure Rate (CSFR)

The indicator also takes into account the fact that all failures are either drops or unsuccessful call setups. The indicator is derived using the expression:

$\text{CSFR}=\frac{\text{Total Number of successul call}}{\text{Total Number of Attempted Call}}\times 100$ (2.10)

Handover Failure Rate (HOFR)

This is the ratio of the failure rate of handover call attempts. This KPI is intended to indicate user failure mobility in the network.

$\text{HOFR}=\frac{\text{Number of failed handover calls}}{\text{Total Number of Attempted Call}}\times 100$ (2.11)

Ec/No Service Quality

Technically Ec/No is usually used to measure equipment capability. In the WCDMA network, Ec/No and Ec/Io are often used interchangeably. This is the ratio of the received energy per chip (=Ec) and the interference level (Io), usually given in dBm. It is used to obtain values for coverage quality of service. Ec/No Rating for WCDMA Signal is shown in Table 2.

$\text{Coverage quality}=\frac{\text{smmation of all data that falls in}>-11\text{dBm}}{\text{Smmation of data}}\times 100$ (2.12)

Table 3 shows the NCC Threshold for all the Network Performance KPI’s.

Table 2. Ec/No rating for WCDMA signal.

Table 3. NCC threshold for network performance KPI’s.

Source: Ugwuoke F 2015.

The authors in [3] analyzed the performance of the four networks using the client (Mobile) in accessing the live GSM network in FCT. Five KPIs were evaluated which are CSSR, CDR, HOSR, Call completion rate (CCR), and Standalone Dedicated Control Channel (SDCCH). From their investigations, it was observed that none of the operators met the NCC threshold for HOSR, most of the operators met the NCC target for Call Completion Rate (CCR) and in the overall performance, GLOBACOM was found to be the best. They used the software engineering approach to develop an optimization drive test plant to improve the QoS thresholds by service providers. They concluded that there is a need for KPIs metrics to be normalized for better QoS.

The work of [4] used an experimental approach and structured questionnaire from six selected locations in Owerri metropolis Imo State which include Aba Road, Obinze Road, Okigiwe Road, Onitsha Road, Orlu Road, and Wethdral Road. Their work titled “Performance Evaluation of WCDMA Networks in selected geolocations using a structured questionnaire and empirical Analysis methods”. They studied the Quality of service of the two predominate Networks (MTN and Airtel) in Owerri and analyzes the performance of the Networks using the Drive test. The six selected KPI’s are Call Setup Success Rate (CSSR), Drop Call Rate (DCR), Call Success Rate (CSSR), and Call Retainability (CR). From their assessment, none of the Networks met the NCC threshold based on the KPIs selected which shows the state of art conditions of the Network rollout in the five geographical locations understudies.

The work of [1] formulated benchmarking of the cellular Network in Ankara, turkey, by comparing the customer’s perceptive on GSM and UMTS Network for three operators A, B, and C to determine which Network renders the best Network performance in Ankara. They study six KPIs which include CSSR, Call Setup Time (CST), CDR, Speech Quality (SQ) Perceptual Evaluation of Speech Quality (PESQ) and Received Signal level (RxL). The end-user perception of Service quality was integrated into QoS assessment. For GSM; operator B was the overall best followed by C and A with 73.00, 74.14 and 72.57 scores respectively.

When KPIs for RX level were compared, operator B had the highest signal strength. Operator C follows operator B with a better signal level than operator A.

Their results showed that operator B had the best performance for accessibility and retainability. The authors did a further analysis of measurement conducted on speech quality using Perceptual Evaluation Speech Quality (PESQ). They analyzed the speech quality (SQ) of the Network operators using Probability Density Function and Cumulative Density Function (PDF/CDF) analysis, which according to them was the most popular KPI. They used PESQ to measure the SQ and evaluated with Chi-Square and fisher’s exact tests. The researchers, therefore, were able to determine which network operator provided the highest Speech quality in Ankara. Their results after comparing UMTS and GSM networks showed that UMTS had better signal levels and voice quality than GSM.

The work of [4], conducted an experimental measurement campaign and drill down the analysis. The aggregate of RSCP and Ec/No, MOS and PESQ was used to deduce the mean, min, max and STD of the UMTS and the aggregate of Rxlev Sub, RxQual Sub, MOS (LQ) and PESQ for GSM in their work on Speech and Video Quality Assessment of GSM and WCDMA Rollout Mobile Radio Access Networks in a Regulated and Competitive Market. The five selected KPI’s are Call Setup time (CST), DCR, Call Success Rate (CSR), Access Failure Rate (AFR), Service Success Rate (SSR), voice and voice quality for GSM and UMTS. Finally, they evaluated Video quality by an assessments algorithm with a range of MOS, PSNR parameter which has a range from 0 to 110 dB, Jerkiness/Pausing, Freezing due to video framed skipping, Jitter which is computed as the standard deviation of all frame intervals according to the typical frame interval and Matched frames.

The authors in [5] employed a data mining process which is also referred to as the nontrivial technique of extracting implicit and potentially useful information from existing data sets in their work, titled “End-User Satisfaction Assessment Approach for Efficient Network Performance Monitoring in Wireless Communication Systems”. The performance statistical data was obtained from the NCC QoS KPIs database. A total of eight months data within the periods of February to September 2013 was obtained and the selected Network was named A, B, C, and D. The researchers analyzed the two key performance measures which are QoS and Grade of Service (GoS). The KPIs used were CSSR, CDR. From the results obtained it none of the operators except operator D in March to September met the NCC target of 98% of CSSR standard within the evaluation period.

The work of [6] studied the “Data on the key performance indicators for quality of service of GSM Networks in Nigeria” The authors empirically assessed and analyzed the Comprehensive data obtained from Nigeria Communication Commission (NCC) The KPIs used are CDR, CSSR, SDCCH Congestion and traffic channel TCH congestion for the GSM Network operators in Nigeria (Airtel, Etisalat, Glo, and MTN).

The authors in [7] used Artificial intelligence through the use of fuzzy Policer (FP) to solve the problem of congestion in WCDMA Networks. The experimental approach was conducted locally through empirical analysis from six selected locations in Owerri metropolis Imo State

From the review of literatures, [2] observed that among the two prominent Network assessed, none of the networks met the NCC threshold based on the KPIs selected, [1] and [5] showed that one out of the Networks assed met the threshold, [1] and [4] assessed voice quality as a measure for Network integrity using MOS(LQ) and PESQ for GSM. [3] observed that there was minimal congestion in the four Network studied, none of the operators met the NCC threshold for HOSR, most of the operators met the NCC target for Call Completion Rate (CCR) and in the overall performance, “GLO” was found to be the best. The software engineering approach was used after evaluation and analysis to optimize the raw data to improve the QoS thresholds by service providers. [7] after the Analysis used fuzzy logic to improve the performance of WCDMA Network caused by congestion of the Network.

This paper study will tend to investigate and evaluate the performance of the four Networks using drive test in Owerri Municipal and Owerri North of Imo State. The experiment will involve the collection of data, analyzing the data to asses Network Availability, Accessibility and integrity for the four networks through some KPI metrics like (CCSR), (CSSR), (CDR), (CBR), (HOSR) and (HOFR). the other objectives were to identify the Network that met the NCC threshold on the listed performance metrics, identify black spots in the WCDMA coverage Area and compare the Received Signal Code Power (RSCP) of GSM to the Ec/No of WCDMA for the four Networks.

3. Methodology

In this performance evaluation assessment, two approaches were used which include the Empirical and Analytical Approach. The Empirical Approach for QoS Analysis was through drive test macro mobility. Drive test was carried out in Owerri Municipal and Owerri North to evaluate the WCDMA Network performance in the four Networks (MTN, Airtel, 9mobile and “GLO”).

3.1. Experimental Setup for Drive Test

Experimental Setup Tools comprises Laptop, Transmission Evaluation, and monitoring system (TEMS) discovery Software, TEMS-enabled mobile stations (Sony Ericsson and Samsung Phones), GPS, Inverter, TEMS Software, and Car. A laptop with TEMS Investigation software installed in it was needed for the test and because of mobility. TEMS Phone was used to test the Network and the software on the phone can synchronize with that of the laptop to monitor the Network. Global Position System (GPS) was used to know the time and it gave a pictorial representation of the route. With the help of a car inverter, the AC (car Battery) can be converting to DC to charge the Laptop. TEMS Software enables monitoring and evaluating the Network when a device is connected to it. The hardware setup was shown in Figure 1.

3.2. Measurement Environment

The measurement was carried in some parts of Owerri Municipal and Owerri North Zone. The Primary Measurement Areas are Imo State University Owerri, Cementary Road, Bishop Court, Ikenegbu, Okigwe Road and Orji Road. The Experiment focuses on the densely populated area and it avoided disjointed data, avoiding traffic jams and with complete information regarding the four Networks. The test was carried out within the busy hours, between 3.00 pm to 5 pm.

Data obtained from the mobile phone (log file) was transferred for post-processing. Evaluation is then carried out using the Tem development Software.

The drive test route that was used in the analysis was clearly shown in Figure 2 and Figure 3. The site identification numbers, cell identification codes, antenna

Figure 1. Experimental setup for drive test.

Figure 2. Pictorial view of the environment of coverage area.

Figure 3. Road map of the drive route.

positions and the GPS showing the latitude, longitude, altitude, speed, etc. are indicated in the figure. The drive was at a very low speed of about 20 - 31 km/h along the study areas to be able to connect to the base station subsystems (BTS and BSC). Calls were placed at the durations 120 sec to be able to have more samples for the analysis.

4. Result

The data were obtained from TEM discovery software during the drive test include Received signal strength for WCDMA Network, Azimuth position of the Antenna, Neighboring cell information and other statically and pictorial information of the WCDMA parameters.

5. Discussion

From the Average received Signal Strength of the four Networks, the Signal received from “GLO” was satisfactory compared to the other Networks.

It was found that in WCDMA all the Networks expect “GLO” did not reach the NCC threshold of 98% and 96% for CSSR and CCSR respectively. The statistical values of the experiment conducted were used to deduce the KPI’s. These standard KPIs were used for benchmarking. From the Statically Analysis in Table 4 it was found that only “GLO” met the target CDR, CCSR, CSSR, and CBR.

In Figures 4-7 and from Table 5, it was found that the handover was smooth in “GLO” and 9 mobile than in “MTN” and Airtel (Figure 8 and Figure 9).

Figure 4. Pictorial view of handover for Airtel.

Figure 5. Pictorial view of handover for MTN.

Table 4. KPI’s statically analysis.

Figure 6. Pictorial view of handover for 9MOBILE.

Figure 7. Pictorial view of handover for “GLO”.

Table 5. Showing KPI’s analytical result.

Figure 8. Pictorial presentation of the received signal along the route.

Figure 9. The KPI’s and NCC threshold.

6. Conclusion and Future Work

This study investigated the Network performance and the Quality of Service delivered by the Radio Networks of WCDMA deployed in Owerri City. The findings of this research work showed that both radio network technologies investigated should be optimized for the better quality of service delivery in some parts of the city. The poor radio coverage and degradation in the quality of service identified in the affected areas of the city were traced to resource utilization, code management and other physical factors like topology and the presence of physical obstructions present in the propagation environment. Also, poor quality samples collected were due to poor coverage in the area. The base stations should be Regularly Monitored and Optimized by Radio Engineers and vendors to enhance radio coverage in the affected areas. The Hand-Over (HO) failures were mainly due to Base Station Controller (BSC) harmonization problems and some black spots observed. This should be resolved appropriately and within three days of occurrences according to the NCC requirement. If the Radio Network is constantly monitored and optimization is successfully performed, the Network Integrity, Availability, Retain ability will be enhanced and that means more customers and more Gain to the Mobile Operators.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Kadioglu, R., Dalveren, Y. and Kara, A. (2015) Quality of Service Assessment: A Case Study on Performance Benchmarking of Cellular Network Operators in Turkey. Turkish Journal of Electrical Engineering & Computer Sciences, 23, 584-559. https://doi.org/10.3906/elk-1302-191
[2]	Onyenwe, E.M. and Nnosiri, A.N. (2018) Performance Evaluation of WCDMA Using Structured Questionnaire and Empirical Analysis. International Advances in Scientific Research and Engineering (IJASRE), 4, 62-65.
[3]	Ugwuoke, F.N., Okafor, K.C., Onwusuru, I.M. and Udeze, C.C. (2014) Using Software Engineering Approach in Mitigation QoS Challenges in Mobile Communication Network in Nigeria. Computing Information Systems Development Informatics & Allied Research Journal, 15, 131-148.
[4]	Pitas, C., Moraitis, N., Panagopoulos, A. and Constantinous, P. (2009) Speech and Video Quality Assessment of GSM and WCDMA Rollout Mobile Access Networks in a Regulated and Competitive Market.
[5]	Isabona, J. and Ekpenyong M. (2015) End-User Satisfaction Assessment Approach for Efficient Network Performance Monitoring in Wireless Communication Sys1-2tems. African Journal of Computing & ICT, 8, 1-2.
[6]	Popoola, S.I., Atayero, A.A., Faruk, N. and Badejo, J.A. (2017) Data on the Key Performance Indicators for Quality of Service of GSM Networks in Nigeria. Data in Brief, 16, 914-928. https://doi.org/10.1016/j.dib.2017.12.005
[7]	Nnosiri, O.E.N. and Ekwueme, E. (2019) Fuzzy Logic Implementation for Enhanced WCDMA Using Selected KPIs. https://www.researchgate.net/