First Impressions in a Mobile World: How Hotel Sites Compare with OTAs, Aggregators and Peer to Peer Accommodations on Website Performance

This study compares the load time for hotel and hotel company websites with the load times for online travel agencies, aggregators and peer to peer accommodation websites. A traveler’s first impression of a hotel company is likely to be their first visit to the website on either their computer or mobile device. Prior research has shown that a slow loading website leads to user frustration and can cause consumers to abandon a website in favor of faster loading sites. Sites with a slower load time are at a competitive disadvantage. This study compares load times for 261 hotel and hotel company websites, 91 online travel agencies, 26 aggregators, and 40 peer to peer accommodation websites on both mobile and desktop platforms. The study found significant differences between website types for desktop load times. The management implications are discussed.

brings to light that web design needs to not only focus on aesthetics of a site, but also must load quickly or risk losing visitors to the site.
The travel and tourism industry is the world's largest service industry [3]. The World Travel and Tourism Council reports that for the year 2017 travel and tourism accounted for 10.4 percent of global GDP, 313 million jobs, and 9.9 percent of the world's total employment [4]. Growth in this industry has outpaced world economic growth, and accounts for one in five jobs created in the last decade [4]. The accommodation services industry is the largest of the travel and tourism industries, and accounts for nineteen percent of total travel and tourism spending in the US [5]. The accommodation services industry is comprised of hotels, and other lodging facilities. Hotel and lodging services were once booked primarily through travel agents using a Global Distribution system [3]. Travel agents were the primary interface with hotel and lodging businesses during the reservation process. Very often a consumer's first encounter with a hotel was upon arrival at the hotel. This has changed in the last decade during which there has been a mass movement of hotel reservation search and booking to the internet. The services that were traditionally provided by travel agents are now frequently done by consumers directly. They search for and evaluate travel alternatives, and subsequently book their own reservations via a website.
Many factors go into the success of a company, not least of which is how well one stacks up against the competition. Looking at the lodging industry, we investigate how well individual hotel website load performance compares against the performance of third party websites, such as OTAs, Aggregators, and Peer to Peer Accommodation sites.
Mobile technology is increasingly used to access hospitality sites. This shift away from desktop to mobile access impacts the performance of a company's website, and thus it is important to consider the impact of mobile access when designing the site. Because the impact can be significant, we want to know how these websites perform in both desktop as well as mobile environments. How do the hotels sites compare against the third party sites and Peer to Peer Accommodations? These issues are critically important because they impact the user's initial experience and have been shown to impact their behavior. Consumers that experience long load times have abandoned sites in favor of sites with faster load times.

Hotel Distribution Channels
Consumers have many options and channels for booking hotel rooms and other lodging accommodations. Most of those channels are now online. Consumers can make reservations online directly through the hotel's or hotel company's website, or make their reservations through a variety of third party internet travel intermediaries [6]. These third party travel websites can be divided into three categories: online travel agencies (OTAs), meta-search sites, and review sites.

OTAs
Online Travel Agencies are a digital equivalent of the traditional travel agency.
An OTA functions by acting as an agent to sell rooms on a hotel's behalf [6].

Aggregators
The travel industry use the term aggregators to describe both meta-search sites and review sites. Meta-search sites search and display hotel reservation information from across a variety of hotel and OTA websites, combining all the information in one place [7]. Examples of meta-search sites include Kayak, Hotels Combined, Hostelz, and Trivago. Hotel review sites, such as Trip Advisor, combine meta-search information and user generated reviews. Note that the distinction between these Aggregator sub classifications has been blurred, with many hotel aggregators now providing meta-search, user generated reviews and an OTA.

Peer to Peer Accommodations
A recent disruptor to the hotel industry is the growth of the peer to peer ac-Journal of Service Science and Management

Hotels Battle Third Party Internet Travel Sites
Hotels have taken differing stances on the value of the third party internet travel sites in comparison to the cost of the commissions and fees. Hotels and hotel companies overall prefer a consumer to book directly on their own website rather than through a third party so that they do not have to pay commissions or fees [9]. However third party internet travel sites have a large market share, and provide access to a broader consumer base. Some hotel companies have expanded their partnerships with these sites, considering the commission a trade for marketing dollars. In contrast, some of the larger hotel companies have developed marketing campaigns to encourage consumers not to book through third party sites but instead to book directly through their own hotel company website [9]. Hilton's "Stop Clicking Around" is an example. Hotel companies are now offering discounts to loyalty club members who book direct. Many offer additional rewards for booking direct such as free wifi, or online check-in. In addition, most hotel companies will only award frequent stay points when guests book direct rather than through a third party site. The competition between third party internet travel sites and hotels for reservation traffic escalated with an initiative by the American Hotel & Lodging Association in the summer of 2017 [10]. AH & LA's marketing campaign urged consumers to book directly with hotels with slogans such as "Look Before You Book", and "Search Smarter." At first glance the campaign seems geared to protect consumers from fraudulent booking scams and sites. But the campaign also pits hotels against the third party sites, going as far as to claim that third party "sites can present consumers with a series of false choices and misleading marketing messages that can lead to incredibly aggravating experiences and ruined vacations."

Service Impressions
Consumers often make decisions about the quality of a business based on their first service interactions with the business. In today's increasingly mobile world, these first service experiences are often online, and increasingly on a mobile device. Online impressions are powerful. When that first impression or first interaction with the website is positive, consumers have a higher perception of the business, its services and products [11]. In travel planning specifically, when the experience is positive, consumers are more likely to utilize the website for trip planning [12]. When the first impression is negative, consumers are likely to abandon the website, and in the case of hotel reservations, a loss of reservation

First Impressions-Mobile Load Time
The first encounter consumers have with a website happens even before you see the website. If the site loads quickly, the consumers are able to get down to the intended task of finding an appropriate accommodation. If however the site is sluggish and loads slowly, the consumers can get frustrated and annoyed, and it the site loads too slowly, they may even abandon the site. Website load time is defined as the time from the start of the initial navigation request from the user until the browser signals that the website is fully loaded. When a site fails to load quickly, consumers often abandon the site in favor of other sites with faster load times [13] [14] [15]. Research has shown "if the mobile website of the hotel does not display the desired information within a few seconds, then the brand is highly likely to lose customers to its competitors" [3].

Mobile Technology
Consumers access a business using websites in a variety of ways and through a variety of devices [16]. Since the introduction of the iPhone in 2007 how consumers access websites has evolved. Mobile devices are changing how consumers interact with businesses. The number of mobile devices now exceeds the number of people on the earth [17]. As the functions and capabilities of mobile devices and internet capabilities on mobile devices have increased, consumers are moving to mobile devices for hotel reservation processes [ [22]. In 2018 mobile bookings accounted for more than half of all online travel bookings (56.7%) [23].
Mobile devices have many advantages in hotel booking. Mobile devices have brought both speed and convenience in travel booking [2]. Mobile devices can utilize GPS to search for nearby hotels, and provide directions to a hotel from the consumer's current location [2]. However, mobile devices have smaller screen sizes, and have lower computing power than personal computers [2]. Providing a good first impression also requires customer engagement. "Clearly, the multimedia elements (i.e. image, audio, animation, and video) of homepages influence first impressions of websites, as they provide rich cues that facilitate information retention, can attract attention, and encourage website exploration" [24].
However providing a rich multimedia experience increases the amount of content that must be downloaded which in-turn increases the site load time which negatively impacts satisfaction. This can be a concern for potential hotel guests using mobile devices.

Online Tools to Measure Site Performance
There are many on-line tools that can be used to assess website performance Journal of Service Science and Management [25]. These tools allow the user to specify a specific website, pick a platform (i.e., desktop or mobile device) and a worldwide testing location to run the test. After the selected test is run, performance metrics are presented. For this study we used GTMetrix, which provides performance testing on both mobile and desktop platforms.

Service Quality Online
As we become a more digitally connected society, there is a need to examine service quality online. Researchers have applied traditional service quality measures to websites developing e-service models and measurements [26] [27]. Hospitality researchers have adapted these models for use on tourism and hospitality [28] [29] [30] [31]. Hospitality research indicates that system quality has a positive impact on consumer satisfaction [ [39]. Online satisfaction was found to have a direct and positive effect on purchase intentions [34].
A recent study developed an e-service quality measurement scale for hotel websites comprised of several factors: functionality, atmospheric quality, reliable information, locality information, customer reviews [29]. Semi-structured interviews were used to develop a survey to assess customers' needs for hotel websites' e-Service Quality. The resulting survey included 24 e-Service Quality factors focused on the content, features and ease of use of the website, but did not address site performance. They found that from customers' perspectives, usability, reliability and responsiveness are not separated independent factors but can be considered as one integrated core factor for evaluating the e-SQ of a hotel website.
While there is limited study of the impact of load time on user perceptions in the hospitality industry, studies have found that consumers transfer the positive or negative first impressions from a website to other elements of the website and to other services and products [24] [40] [41]. In travel and hotel reservations, research has shown that when consumers have a positive experience with the webpage, they are more likely to stay on the webpage and to make a reservation using that website [11]. However when the experience is a negative one, they are likely to abandon the website in favor of a competitors site [3]. Four out of five travelers abandon a search without booking, with about 10% of that abandonment due to technical issues [42].

Comparing Hotel Websites and OTAs
A key metric in reservation process is conversion, or the conversion of a guest searching for hotel reservation into an actual hotel reservation. When conversion rates are compared across sites and devices, accessing a hotel site through a mobile device has some of the lowest conversion rates as compared to accessing Journal of Service Science and Management an OTA through either a mobile device or desktop computer, or accessing a hotel site through a desktop computer. Comparing desktop access, OTAs and hotels have performed similarly in conversions, trading one another for top performance [43].

Load Time
Research on load time has established several thresholds of user tolerance. When a website loads in 0.1 seconds or less, users perceive the response as immediate.
However, when the load time was 10 seconds a significant number of users perceive the delay to be unacceptable [44]. The study also found that tolerance for load time decreased as the number of pages increased. Load times which were acceptable on the first pages, were not tolerated as the number of webpages visited increased. Consumer tolerance or acceptance of load time is evolving as technology evolves, with consumers becoming more demanding and expecting faster load times. Research shows that "47% of consumers expect a web page to load in two seconds or less, and 40% of people abandon a website that takes more than three seconds to load" [45].

OTAs
Online travel agencies (OTAs) and other third party websites have become popular search and reservation tools during the last decade. Recent research has shown that more consumers search for hotel reservations on OTA websites than on hotel websites [43]. Almost twice as many consumers utilize OTAs than direct booking on hotel websites, and the gap is growing with continued growth in market share by OTAs [55] [56]. Millennial travelers are even more likely to utilize an OTA than a hotel website [55] [57]. The preference for OTAs over hotel direct booking is even greater with reservations of seven nights or less [55].
Travelers who utilize mobile devices to book hotel rooms prefer OTAs to hotel brand sites by a three to one margin [58]. Research shows that even when con-Journal of Service Science and Management sumers book directly with a hotel, they often visit OTA sites to research travel and hotel options [59]. OTAs were often preferred over hotel direct booking sites, due to price transparency [60] [61].
In addition to providing price transparency, OTAs generally provide user generated reviews of the hotels listed [62]. User generated reviews allow consumers to provide advice and recommendations to other travelers [62]. The offering of user generated reviews by OTAs has been shown to impact travelers channel choice in favor of OTAs over hotel websites [61] [63] [64]. Other research on OTAs has focused on pricing, hotel listing search and positioning, and consumer attitudes. Research has examined the effect of position of a hotel listing on an OTA website, finding that hotels listed near the top of the page have an advantage [65] and that hotels listed at the bottom of the page [66] have an advantage over those listed in the middle. Anderson [59] found that when hotels also list their inventory on OTAs a billboard effect occurs, they increase reservation volume through the hotel's direct channels as well. Similar to research on hotel website quality, research for OTA website quality has considered load time and its related factors to be an important dimension [67] [68] [69] [70], but little to no research has measured actual website load time for OTAs.

Website Tools
Web performance tools have been used to study hotel website performance. Lee and Morrison [71] utilized Net Mechanic.com and Linkpopularity.com to compare websites of Korean and US hotels. Zhu [72] utilized Google's Yslow to examine performance of tourism websites. And [73] used Webpagetest.org to examine performance of Airbnb. Gerdes and Stringam [74] used both GTMetrix and WebPagetest to investigate load time differences between desktop and mobile platforms.

Methodology
To evaluate the load performance, we used GTMetrix to evaluate the load performance of the default landing page for each website. This would be the typical starting page for a customer's first visit to the site. Performance information was obtained using GTMetrix.com, which measures and analyzes the load performance of webpages. While there are similar tools that provide this type of analysis, GTMetrix was selected because it allowed the testing of both on a desktop and mobile platform at the same test site.
GTMetrix is an online tool which measures the load performance of a user specified website. The user selects a test location among seven worldwide locations, testing platform (i.e., desktop, or specific mobile device), a browser (i.e., Chrome, Firefox) and network bandwidth (i.e., ranging from an unthrottled connection through dialup speeds). When the test is initiated, the target website is downloaded on the specified test platform. Load performance metrics are captured to indicate how well the site performs. These include PageSpeed Score B. Stringam, J. Gerdes Journal of Service Science and Management and YSlow score, which are metrics developed by Google and Yahoo respectively that provide a high level assessment of the site. These are presented as a letter grade as well as a numeric score for the site. They also provide the ability to drill down to see what issues were identified, and how the score was determined.
Thus it provides a readily accessible score for management, along with the details needed by the technical staff to improve the site's performance. GTMetrix reports 27 elements for the PageSpeed metric, (19 for YSlow) that impact website load performance, each with a letter grade, numeric score, and indication how this affects performance (i.e., server, content, CSS/JS, cookies, and images).
This tool is available as a free service, or through a paid subscription.
The GTMetrix tool provides multiple ways of visualizing the data. Besides the metrics reports, there is a waterfall chart which shows the load sequence of web site elements. This can be helpful to identify sequencing issues that might be slowing the load time. The Page Load Timing chart identifies significant milestones in the load process. The Page Load Video and Filmstrip provides a way to see how the site is loading. This is used to determine the Speed Index, which is the time to load the content "above the fold", (the portion of the site visible without scrolling).
We standardized on a single testing location to minimize external variables that could influence the results. Two series of tests were run. The first on a desktop machine, and the second on a mobile phone, with all test run at GTmetrix's Vancouver, Canada site. Both tests used Chrome as the browser. The mobile tests used a Galaxy Nexus Android Device which at the time was the only mobile testing platform available through this service. Data collection was done in July 2017.
GTMetrix reports extensive performance data. The study focused on the First Visit Load time, which is measured as the time from the start of the initial navigation until the on load event is triggered, signaling that the webpage has been fully loaded. As its name suggests, this metric measures the time to load a page that has never been visited before, and therefore all required support files must be downloaded from the web host. The load time of subsequent visits to the site can be reduced accessing files that have already been downloaded and saved locally.
The set of web sites used in this study was compiled from several sources. The study included a set of 261 websites of international hotel companies and sub-brands. Website URLs used in the study were compiled from industry and financial reports, academic literature, and supplemented through search engine searches [75] [76] [77]. The data set was limited to hotel companies with six or more hotels. A list of 91 OTAs, 26 Aggregators, and 40 Peer to Peer Accommodation sites was compiled from published lists of on-line travel agencies [78]. The study sought to be geographically diverse, including international hotel companies and third party travel sites.

Results
Load time performance varied by device and website type. Mobile load times Journal of Service Science and Management exceeded those of desktop load times (see Table 1). Several sites were considerably slower than the other sites in their respective category. Averages and comparisons for statistical significance were computed with outliers removed.
Aggregator websites had the fastest average load time for desktop access (4.86 seconds). Peer to Peer Accommodation websites were the second fastest loading sites for desktop access (5.53 seconds) followed by OTA websites (7.21 seconds) with Hotel and Hotel Company websites loading the slowest of all the categories on average (11.38 seconds). Mobile load times were slower than desktop overall with Peer to Peer Accommodation websites loading the fastest from a mobile platform (13.45 seconds). Aggregators were the second fastest (13.7 seconds) followed by OTA websites (14.28 seconds), with Hotel and Hotel Company websites loading the slowest for mobile access (14.91 seconds).
The differences in average load times between website categories was greater for desktop access than for mobile access (see Table 1 and Figure 1 and Figure  2). Desktop load times for Hotel and Hotel Company websites was more than double the load times of the two fastest categories, Aggregators and Peer to Peer Accommodations. Kruskal Wallis testing was used to determine if the observed

Across Quartiles
To examine load time performance more closely, the data was examined across quartiles based on load times. Load times were calculated for when 25%, 50% and 75% of the sites were fully loaded.

Desktop Load Times
Examining the quartile performance, Aggregators were fastest with 25% of the sites loading within 3.4 seconds (see Table 2). Hotel company websites were the slowest with the fastest 25% of websites loading within 5.6 seconds. Unfortunately for hotel websites, the differences increase substantially at the third quartile, with Hotel sites loading in 17.2 seconds, while the third party and peer to peer sites loaded in 7.6 seconds or less (see Table 2).

Mobile Load Times
When mobile load times were compared across quartile performance by website Journal of Service Science and Management categories, the differences were considerably smaller than those seen on the desktop platform. For mobile load times, hotel company websites and Aggregator websites performed the fastest with 25% of hotel websites loading at of 9.1 seconds and for Aggregators: 9.5 seconds (see Table 3). OTA's and Peer to Peer Accommodations performed just slightly slower with 25% of the sites loading at 9.6 seconds for OTA's and at 10.6 seconds for Peer to Peer Accommodation websites. Comparisons of load time performance across the remaining quartiles was similar with Aggregator websites the fastest for the 50% and 75% marks, followed closely by hotel websites. Peer to Peer Accommodation websites were the slowest at the second quartile but were faster than OTA at the third quartile (see Table 3).

Conclusions
Examining load times for hotel websites across distribution channels showed differences in performance. Desktop load times for hotel and hotel company websites were the slowest, with Aggregator sites loading fastest. Overall, mobile load times were slower than desktop load times, but performed similarly across distribution channel.
Research has shown that system performance plays an important role in consumer satisfaction which in turn impacts the purchase decision and the intent to return. This study found there is ample room to improve website load times for hotel and hotel websites as compared to third party and peer to peer internet travel sites. Desktop load times varied significantly by category with hotel and hotel company websites performing the slowest. Hotels, unlike third party and peer to peer internet travel sites, are engaged in more than just reservation distribution. As such it is not surprising that load times for hotel websites were the slowest on both desktop and mobile platforms.
Hotels and hotel companies have engaged in substantial advertising campaigns to encourage consumers to book direct rather than through third party internet travel websites. While these advertising campaigns have been successful in getting some consumers to book direct, consumer use of OTA's, aggregators and peer to peer accommodations continues to grow [79]. Given the intensive marketing efforts by hotel companies to get consumers to book direct rather than through third party internet travel sites, the significant difference in desktop load times is concerning. When load times exceed consumer expectations, B. Stringam, J. Gerdes Journal of Service Science and Management consumers will abandon the search and seek to make reservations on other websites [58]. Desktop load times for hotel and hotel company websites were more than double the load times of two of the other website categories. Research has shown that users will leave a site if page load times become too long [13]. The substantive delay in load time has the potential to cause consumers to abandon an attempt to make a reservation on a hotel or hotel company website in favor of using a third part internet travel site which loads faster. Previous research has shown that relatively small increases in load time can have a profound impact on how users react to Web sites [80]. Hotels and hotel companies should pay attention to webpage load times in order to capture reservation inquiries.
Mobile load times were slower than desktop load times, but were more consistent across the four categories of websites studied. Consider the first quartile. Reducing load time is important not only due to the impact has on visitor satisfaction, but also on search engine rankings. Google uses load time as a factor when ranking search engine results [82]. This does not impact customers that directly typed in the corporate web page address, but could impact where the company is placed in Google search results. Consumers use on-line search engines to learn about potential hotels, particularly in the early phase of the search [83].
The good news is that there is an array of on-line tools that can provide a performance assessment of your site. They provide a multi-tiered analysis, including a management dashboard that reports key performance metrics such as Site Load time and Speed Index (Above the Fold load time). These dashboard metrics provide management a quick way to assess the performance of the site. For web managers the tools also provide suggestions as to how the site can be improved. It is important to realize that these tools only address load performance, and not site design. Implementing the suggested improvements will not impact the look or design of website, just make the loading faster. For example, images should be sized appropriately when displayed on a website. If the image is larger than necessary, it takes longer to download and the browser needs to take some time to scale the image to the proper size, which delays load time. An even better approach is to cache static information locally on the user's browser, in which case it only needs to be downloaded on the first visit. Enabling this local caching can result in a significant reduction in site load time.

Directions for Further Research
While the study sought to be broad and representative in scope, there are many additional factors that could be studied. Independents and smaller hotel properties were not included. It would be interesting for load time be studied for these hotel types. Also, as technology evolves, so too does website design trends, which would impact download time. Mobile technology is particularly dynamic with the introduction of newer and faster phones and networks such as 5G. It would be interesting to see if companies adapt their websites to take advantage of this new technology, and also adapt their sites to meet consumer expectations. Given the rapid changes in web technology, it is recommended that load time be reevaluated on a regular basis.
Another area that could be addressed is a content analysis to determine what type of content tends to impact site performance the most, and also what are the most common issues that impact performance. This current study looked only at the landing page of a site. It would be interesting to determine if this is representative of the site as a whole, or is there an alternate page that should be tested.
It would also be interesting to measure user behavior in response to differing load times. The tolerance for longer delays may be different for new users vs. return customers, or for customers with different objectives (i.e., immediate purchase, or browsing for a future trip). As mobile technology for both devices and connection improves, some consumers are using more apps in lieu of mobile websites. Studies on the use of apps over websites should be included in future studies of load time. Finally, technology and consumer demands are ever changing.

B. Stringam, J. Gerdes Journal of Service Science and Management
It is recommended that similar studies be conducted at future intervals to capture the effects of changing technologies and investments in hotel reservation technologies.

Conflicts of Interest
The authors declare no conflicts of interest regarding the publication of this paper.