QUIC vs TCP: A Performance Evaluation over LTE with NS-3

In the vast majority of mobile applications, the Transmission Control Protocol (TCP) is still leveraged at the transport layer of the Internet’s protocol stack. But, in many cases, the performance of TCP over mobile networks has been proven sub-optimal in practice, thus causing substantial bottlenecks. Quick UDP Internet Connections (QUIC) is a new protocol, currently being standardized by the Internet Engineering Task Force (IETF), that aims at solving some of the inherent problems of TCP. The purpose of this paper is to provide a comprehensive overview of QUIC and compare the performance of QUIC and TCP in wireless networks. To compare QUIC with TCP under various transmission scenarios over LTE networks, the ns-3 network simulator has been employed. The simulations performed showed that 1) under good or average transmission conditions, QUIC is characterized by a better steady state throughput at the same time achieving quite lower file download times; and 2) under poor transmission conditions, the two protocols exhibit a similar performance.


Introduction
Ever since its creation, the Internet has experienced an impressive rise both in the number of users base and the volume of data traffic. According to a report issued by Cisco [1], this trend is expected to continue in the following years. The report foresees that 1) the volume of IP traffic will be 235 Exabytes per month in 2021, that is three times the corresponding traffic in 2016; and 2) the number of Internet users is expected to reach 5.3 billion by the end of 2021.
To support the rapidly increasing mobile and desktop traffic over the Internet, changes are required both in network infrastructure and communication protocols. The majority of web applications today leverage TCP and Hyper Text Transport Protocol (HTTP) to perform functionalities, such as connection establishment, reliable data transmission and congestion control. However, the original design of TCP and HTTP did not predict the increasing complexity of modern web pages and the dynamic nature of Internet traffic. Consequently, the TCP and HTTP protocols currently constitute a major performance bottleneck in the modern Internet. In wireless networks, particularly, these protocols have an adverse effect on web access times, thus deteriorating the Quality of Experience (QoE) offered to the users.
To address this challenge, the Internet research community has developed an intense interest in the design and implementation of novel transport protocols.
In 2013, Google introduced QUIC (Quick UDP Internet Connections), a transport protocol that aims at improving throughput performance and reducing page load times for web applications [2]. Since 2016, QUIC is undergoing a stan-  [9]. In this paper, we utilize an implementation of QUIC for the ns-3 [10] in an attempt to explore the performance of QUIC over LTE networks and compare it to that of TCP.
QUIC is a novel transport protocol proposed by Google that aims at replacing TCP in the near future. IETF has undertaken the task of standardizing QUIC and regularly publishes Internet Drafts in order to document the required func-tionalities and other relevant information for QUIC implementations [11].
The protocol stack configuration of a secure HTTP connection over QUIC or TCP is shown in Figure 1. Though QUIC shares common features with TCP, the two protocols exhibit significant differences in various functionalities. QUIC is developed as an application-layer transport protocol that utilizes UDP as a sub-layer. The reasoning behind this approach is that the deployment or the upgrade of transport protocols in the kernel space of the OS is a very difficult and time-consuming task. In contrast, any required changes at the application code of an application-layer protocol, even substantial ones, can quickly be tested and distributed. As such, QUIC will allow for rapid and wide-scale protocol deployment, updates and optimizations in the future.
As stated earlier, QUIC runs on top of UDP. However, as a "best effort" protocol, UDP does not offer reliable data transmission neither guarantees in-order data delivery to the receiver. Therefore, QUIC must also implement a reliable and in-order data delivery mechanism. In a way similar to TCP, QUIC uses acknowledgments (ACK) in order to detect losses of QUIC packets and perform the necessary retransmissions. A key difference, though, is that QUIC uses unique packet numbers to identify each packet that is sent over a QUIC connection. These QUIC packet numbers are monotonically increasing and do not repeat for the lifetime of a QUIC connection. In comparison, TCP uses sequence numbers to identify TCP segments, but the same sequence number may appear more than once over the entire duration of the connection, i.e. in case of packet retransmissions. This feature of TCP causes a problem known as ACK ambiguity, that is TCP is unable to differentiate lost packets from packets that are delivered out-of-order. The introduction of unique QUIC packet numbering enables QUIC to differentiate between lost and out-of-order packets, thus allowing for a more accurate estimation of the connection Round Trip Time (RTT). As a result, QUIC offers 1) improved congestion control by reducing the number of unnecessary retransmissions and 2) more efficient detection of and recovery from packet losses.
Another long-standing problem of TCP is that any internet connection is identified by a unique 4-tuple value that contains the client's and server's IP addresses and port numbers. However, any change in the underlying IP address or port number of a TCP connection will cause the termination of this connection. Therefore, TCP connections do not exhibit resilience to any updates of the above parameters that can be initiated by internet middleboxes and/or client mobility in the case of mobile or Wi-Fi networks. QUIC deals with this issue by introducing a Connection ID which is an 8-byte number that uniquely identifies a QUIC connection and is independent from the IP address and UDP port parameters of underlying protocol layers. As such, even if the IP address or UDP port number of a client changes during the communication with a server, the connection ID provides robustness to the QUIC connection which is not terminated.
Moreover, QUIC is designed to offer end-to-end data encryption, which means that both application data and QUIC headers are authenticated and encrypted over a QUIC connection. As illustrated in Figure 1, in the case of TCP the TLS protocol runs on top of the TCP layer. In the case of QUIC, however, the cryptographic functionalities of the TLS protocol are directly integrated into the QUIC layer of the protocol stack. This feature allows QUIC to minimize the latency introduced during the initial connection establishment phase. In the case of TCP, two handshakes must be completed before the transmission of application data: first the TCP protocol handshake and then the TLS cryptographic handshake. The added latency is equal to either 3-RTT or 2-RTT, depending on which version of the TLS protocol is used, TLS 1.2 or TLS 1.3, respectively. Instead, QUIC embeds the necessary cryptographic parameters into the first QUIC packet that is sent from the client to the server, thus reducing the connection establishment latency to 1-RTT. In the special case where the client and the server have communicated again in the past, stored cryptographic information can be used to perform a 0-RTT handshake, where encrypted application data are encapsulated into the first QUIC packet that is sent. The 0-RTT handshake represents the optimal scenario in terms of connection establishment latency.
Finally, QUIC implements a stream multiplexing mechanism at the transport layer, which is similar to that of HTTP/2 at the application layer. HTTP/2 maps each different web object to an independent application-layer data stream, meaning that if one stream loses data due to packet loss, the other streams can continue to deliver data to the application. However, TCP is not able to differentiate application data streams at the transport layer, an issue that is identified as the Head-of-Line (HOL) blocking problem. When a TCP connection experiences a packet loss, TCP blocks all application-layer streams from delivering their data until the lost packet is recovered, even if the lost packet contains data from a single application stream which is independent from other streams. Therefore, the HOL blocking problem represents a significant adverse factor for HTTP performance. QUIC solves this issue by extending the application stream mechanism to the transport layer, by mapping one-to-one application data streams to independent QUIC data streams that can be transmitted concurrently over the same QUIC connection. Consequently, even if a packet is lost, QUIC can identify the QUIC streams whose data were encapsulated in the lost packet and, thus, block only the affected application-layer streams. All the other streams that are unaffected from the packet loss can continue to deliver data in-order to the application at the receiver. Consequently, the performance of HTTP is significantly improved. Based on simulations employing the ns3 network simulator, this paper aims at comparing QUIC with TCP with respect to basic performance criteria such as the steady state throughput, the fairness in distributing the available bandwidth to competing connection flows and the file download time. To the best of the authors' knowledge, such a simulation based quantitative attempt has not been performed so far.
The rest of the paper is organized as follows. Sec. 2 presents the simulation setup and the scenarios considered for the performance evaluation of QUIC and TCP. The analysis of the simulation results is provided in Sec. 3. Finally, Sec. 4 concludes the paper.

Simulation Scenarios
The simulations aim at investigating the performance differences of the TCP and QUIC protocols over LTE networks. In this course, the Network Simulator-3.29 was utilized [12], which allows for the implementation of various communication scenarios between nodes under various network architectures and topologies. For the LTE part of the networks examined, the LENA (LTE-EPC Network simulAtor) module of ns-3 has been employed, which provides an accurate representation of the LTE protocol stack and Evolved Packet Core (EPC) services, with the sole limitation that the simulated network has a single Serving & Packet Gateway (S/P-GW) node.
The architecture of the simulation model is depicted in Figure 2. Across each link involved, the maximum transmission rate (link capacity) and link delay is given. All queues are DropTail queues with a maximum length of 100 packets. The S/P-GW node is connected with point-to-point links (P2P) to two remote QUIC and TCP servers. Each User Equipment (UE) device uses either the TCP or QUIC protocol and the generated traffic is routed towards the corresponding  [13].
The main target of this paper is to investigate the possible performance gains offered by the new features of QUIC compared to TCP, when data are transmitted over a wireless channel.

Steady State Throughput
The first simulation scenario studies the steady state throughput achieved by

Flow Fairness
The second simulation scenario examines the fairness principle with which any transport protocol must comply as to the fair distribution of the available bandwidth to competing connection flows. If fair bandwidth allocation is not ensured, a connection flow that uses an "unfair" protocol will be able to acquire a disproportional amount of the available resources and, thus, achieve better performance compared to other competing flows. It should be noted that in wired networks it has been shown that QUIC is unfair towards TCP, even when the two protocols employ the same congestion control algorithms [8]. To investigate the fairness issue arising between QUIC and TCP in LTE networks, the Bulk-

File Downloads
The third simulation scenario investigates the file download performance of QUIC and TCP. The distance between the UEs and the eNB is again set at 250 m. Five file sizes in the range 64 kB -2 MB have been considered. The file download time (DLT) has been selected as the relevant performance metric. The results obtained are presented and discussed in Sec. 4.

Effect of QUIC Streams on Throughput
As analyzed in Sec. 2, one of the novelties of QUIC is the extension of the HTTP stream mechanism to the transport layer by introducing the QUIC stream mechanism towards solving the HOL blocking problem of TCP. To quantify the effect of QUIC streams on throughput over LTE networks, an ns-3 QUIC application is used. The QUIC application sends 512 bytes of data every 300 μs from the QUIC server to one UE. The simulation is repeated varying the number of QUIC streams and distance between the UE and the eNB. The results obtained are presented and discussed in Sec. 4.

Simulation Results and Discussion
In this section, the results obtained under the ns-3 simulation scenarios described in Sec. 3 are presented and discussed. Table 1 presents the simulation results for the steady state throughput of the TCP and QUIC protocols when the distance between the UEs and the eNB varies. In order to exclude from the measurements the slow-start phase of the connection establishment, the steady state is defined as the simulation duration from 5 s to 40 s.

Steady State Throughput
From Table 1 it is deduced that the performance gains offered by QUIC in wired networks are also observed in LTE networks, despite the adverse characteristics of wireless channels that deteriorate the operation of a transport protocol. However, as the distance from the eNB increases that causes reduction in the received signal quality at the UE, the throughput of QUIC becomes comparable to the throughput of TCP. A possible explanation behind this phenomenon is given at [14], where it was identified that, in wired networks, QUIC exhibits worse performance than TCP when packet re-ordering is necessary. When the received signal quality over a wireless channel is reduced, severe packet re-ordering is necessary to deal with the adverse effect of fast fading losses. Therefore, our simulation results indicate that packet re-ordering decreases QUIC performance also in the case of wireless networks. Overall, under good wireless channel conditions, QUIC outperforms TCP, but, as the channel conditions become worse, the two protocols exhibit a similar throughput performance.

Flow Fairness
The simulation results examining the fairness between competing QUIC and TCP flows are summarized in Table 2.
In the case of two competing QUIC flows, they exhibit similar throughput and therefore are fair to each other. Next, the co-existence of one QUIC flow with two and five TCP flows is examined. The two and five TCP flows in each case examined achieve approximately the same throughput as denoted by the low relevant standard deviation. This means that the TCP flows are fair with regard to each other. But, the single QUIC flow outperforms the TCP flows though both protocols use congestion control algorithms of similar features. Finally, the case of two QUIC coexisting with two TCP flows is examined. As denoted by the relevant low standard deviation values, fairness is observed between the two QUIC flows as well as between the two TCP flows examined. It is also observed that each QUIC flow acquires almost double the bandwidth compared to that acquired by each TCP flow. It is evident that, though the two protocols exhibit an individual internal fairness, in LTE networks QUIC connections outperform TCP ones with regard to the throughput acquired, thus causing a severe lack of fairness against TCP. Table 3 summarizes the simulation results concerning the file download performance of the QUIC and TCP protocols.

File Downloads
The simulation results show that QUIC achieves quite lower download times for all the files sizes considered. This performance gain of QUIC over TCP is reduced as the file size increases. Additionally, it should be taken into consideration that QUIC allows endpoints that have communicated before in the past, to establish a connection with a 0-RTT handshake. This mechanism speeds up the transmission of application data by at least 1-RTT compared to TCP. When the QUIC 0-RTT handshake was enabled in the examined simulation scenario, the connection establishment time was reduced from 72 ms to 22 ms for all file sizes examined. Given that approximately 40% of all HTTPS connections are reused connections [15], it is clear that QUIC will significantly enhance the user experience in Web Browsing applications.

Effect of QUIC Streams on Throughput
The simulation results concerning the effect of the QUIC streams mechanism on throughput are presented in Table 4.
Overall, the introduction of stream multiplexing at the transport layer improves the throughput performance of QUIC. As intuitively expected, the steady state throughput is higher when the number of QUIC streams served by the connection is increased. However, the gain in throughput is reduced as the number of QUIC streams increases. Moreover, as the distance between the UE and the eNB increases, the throughput gain resulting from the use of QUIC streams gets lower. In conclusion, the use of independent QUIC streams for the transmission of application data deals effectively with the Head of Line Blocking problem caused by TCP, especially in LTE networks where the HOL problem occurs frequently due to random bandwidth fluctuations in the wireless channel.

Conclusions
The key findings of the simulation experiments can be summarized as follows. 1) Under good or average received signal quality at the UE, the QUIC protocol has a better steady state throughput performance compared to TCP. However, when the received signal quality at the UE is poor, the two protocols have a similar performance. 2) QUIC connections acquire a larger portion of the available bandwidth compared to TCP ones. This indicates that fairness between TCP and QUIC does not exist, even when the radio resources at the eNB are equally distributed to all the UEs. 3) QUIC demonstrates quite lower file download times compared to TCP, for all file sizes examined. 4) The 0-RTT handshake feature of QUIC greatly reduces the connection establishment times. 5) The biggest benefit of QUIC lies in the stream multiplexing mechanism at the transport layer, which increases the protocol's throughput performance and effectively counters the HOL blocking problem of TCP.
In future research, it is worth investigating potential optimization schemes for QUIC over wireless networks. As QUIC encrypts all the information contained in the QUIC packet headers, all the TCP accelerator mechanisms that are currently deployed in modern wireless networks will have to be properly modified towards serving and optimizing mobile QUIC traffic. Moreover, it is important for network providers to exploit specific characteristics of QUIC (such as the Spin Bit [16]) towards monitoring the status of QUIC connections within their networks. Furthermore, an interesting further research could be to investigate how QUIC affects the performance of scheduling algorithms in LTE networks [17]. Finally, an important field of future research could be the development of novel congestion control algorithms that can optimally utilize the new characteristics of QUIC.