Mapping High-Level Application Requirements onto Low-Level Cloud Resources ()
1. Introduction
Cloud computing provides many potential benefits to business organizations. It provides access to scalable ondemand computing resources based on a pay-as-you-use pricing model. Many organizations currently use the cloud as an alternative platform for executing business applications. However, increasing number of cloud providers supply the cloud marketplace with a wide range of cloud resources. Different providers offer cloud resources in different formats using different pricing structures. Given the wide range of products available and the complexity of the cost structure for consuming cloud resources, it can be difficult to estimate the cost of running an application in the cloud. Searching for a suitable set of cloud resources that meet an application’s requirement and budget is a challenging task. For many applications, a high degree of business continuity and service availability are of paramount importance. Many business organizations demand high availability of resources and require a solution that can deliver a highly resilient business service. In the event of service interruptions, caused by a provider’s resource malfunctioning, such as the AWS incidents [1], organizations should have the option to migrate their applications elsewhere.
From the application developer’s point of view, it is desirable to have a programming model for constructing cloud-based infrastructures on which to deploy business applications. This model should provide an applicationcentric mechanism for specifying high-level application requirements and a means of translating them onto the low-level cloud resources offered by different cloud providers in the marketplace. Many frameworks have been proposed for managing cloud resources [2,3]. However, these frameworks do not provide a mechanism for comparing and selecting cloud resources according to application’s constraints. Papers [4-6] propose different approaches for describing cloud resources, but requirements are typically analyzed from the provider’s perspective which is usually based on the resource capabilities offered by the providers. Currently there are no suitable mechanisms for describing high-level requirements from the application’s point of view The pricing structure for consuming cloud resources varies across cloud providers. Some providers, such as Amazon Web Services, offer compute resources in multiple geographical areas and charge different prices for each area. Certain provider uses a rate-tier structure for data transfer usage (or bandwidth) whereas others use a flat rate structure. For example, HP Cloud charges $0.12 per GB per month for usage up to the first 10 TB, $0.09 for the next 40 TB, $0.07 for the next 100 TB and so on; In contrast, Rackspace charges a flat rate of $0.18 per GB. It is difficult to compare the costs of using cloud resources for running applications in a multi-provider cloud environment.
This paper proposes an application-centric, multi-layer ontological approach for specifying requirements. The ontologies enable application developers to formulate highlevel domain-specific requirements and subsequently apply these descriptions to search for the most suitable set of resources in a multi-provider cloud environment. Cost ontologies are proposed to model the costs of cloud resources; these can be used for estimating and comparing the costs of running applications in the cloud.
This paper is organized as follow. Section 2 describes an ontology-based resource mapping model. Section 3 defines ontologies for specifying cloud requirements and resources. A use case example is given in Section 4. Section 5 discusses related research and conclusions are drawn in Section 6.
2. A Resource Mapping Model: An Ontological Approach
An ontology provides a means of modelling a domain of knowledge. It provides formal descriptions of a set of entities and the relationships between them. In this paper, we propose a mechanism for mapping high-level application’s requirements onto low-level cloud resources using an ontological approach (see Figure 1).
In the proposed model, high-level application requirements are expressed using domain-specific ontologies. These ontologies provide a semantic mechanism for capturing high-level requirements in a language or terminology familiar from a user’s application domain. Domain-specific ontologies focus on user’s needs. Once the high-level domain-specific ontologies are constructed, it is up to developers to translate these domain-specific ontologies onto infrastructure deployment ontologies. Knowledge databases or historical databases can be used to facilitate the translation process by providing application-specific performance and stability data. For example, in [7] several tasks are run simultaneously on a multicore and high memory resource in order to meet a delivery deadline.
The infrastructure deployment ontologies provide constraints for resource selection. For example, in [8], a high availability and high resilience constraint results in an application being deployed on mirror infrastructures that are located at different locations. This ontological layer is resource-agnostic and gives an abstract view of infrastructure requirements from the application’s perspective.
Figure 1. An ontology-based resource mapping model.
In the proposed model, infrastructure deployment ontologies are then used to search for the most appropriate set of resources from the resource layer.
The resource layer contains a pool of cloud resources offered by various providers. Resource ontologies, such as [5,6], are used to describe the features or capabilities of resources offered by cloud providers. Cost ontologies are proposed for specifying the costs of resources. The choices of cloud resources (or provider) depend on the user’s preferences. Multiple cloud providers can be used as long as appropriate financial constraints are satisfied.
A two-phase resources discovery approach is used for selecting the most appropriate set of resources for a given application [9]. In the first phase, possible resources which meet an application’s requirements are identified. In the second phase, suitable heuristics, such as cost or performance, are used for filtering the initial sets of resources. Once the best set of resources is identified, the resources can be instantiated and managed using a cloud management API, such as [10,11].
3. Cloud Ontologies
In this section, we describe an ontological approach for specifying cloud application requirements. OWL 2 [12] is used to specify ontologies.
3.1. Domain-Specific Ontology
A domain-specific ontology defines the high level requirements that are needed (or desired) for an application. These ontologies are application-specific and expressed using the terminology familiar in the user’s application domain. An example of a domain-specific ontology is given below.
Media Transcoding and Distribution Services
Media transcoding is the process of converting media files from one format to another; media distribution is the process of distributing media contents (using streaming media technology) to many online users (viewers or listeners) simultaneously over the internet. Media transcoding and distribution services require high CPU, high memory, large storage space and fast bandwidth. Consider a company that is offering an online training service. It needs to broadcast a series of training videos simultaneously to 500 users in UK and to 500 users in Singapore. These videos use avi format and are made available 5 hours before the broadcast schedule. In order to provide a good viewing experience and achieve high user satisfaction, these videos need to be transcoded into high quality formats in different screen sizes for different devices (such as PC, iPad, iPhone). In addition, there is a budget constraint of £5 per user. These requirements may be specified in high-level notations as follows:
Video sources: avii;
Video quality: high definition;
Playback devices: PC, iPhone, iPad;
Broadcast schedule: 9 am next morning;
Simultaneous users: 500 (UK) and 500 (SG);
Budget: £5 per user.
We have used this example and other examples of transcoding to devise a generic ontology for specifying media transcoding and distribution requirements. The ontology includes:
• Budget requirements which specify monetary constraints on leasing the infrastructure. These can be expressed as the maximum cost per day, per hour, or per user.
• Format requirements which specify the container format of the media. For example, transcoding a video from avi format to high quality windows media format.
• Device requirements which specify the destination devices (i.e. viewer’s devices) that the media will be played on; for example, iPad or iPhone.
• Delivery time requirements specify the time when the transcoded media will be ready for broadcast.
• Capacity requirements estimate the number of users that will use the service simultaneously.
• Location requirements specify the geographical location of users that will use the service.
The ontology adopts media terminology. Many media users would be familiar with such high-level specifications rather than the details of low-level infrastructure resources (hardware or software) which support the media service. By using the proposed model, media users can specify media services’ requirements using appropriate terminology. Application developers could utilize knowledge databases, such as historical records, CPU benchmarks of transcoding tasks or performance benchmarks of media servers, to map high-level domainspecific ontologies onto infrastructure deployment ontologies.
3.2. Infrastructure Deployment Ontology
The next layer in our programming model provides a means of specifying a generic infrastructure which can support the high-level user requirements. The generic infrastructure model should have the potential to be instantiated using a wide range of cloud products, supplied from the multi-provider marketplace.
In the proposed ontology, infrastructure requirements define the capabilities, features or qualities that are necessary (or desired) for an infrastructure on which to execute the application. In general, infrastructure requirements are divided into the following categories (see Figure 2):
• Cost requirements specify the budget for deploying cloud infrastructure.
• Performance requirements refer to quality and performance of the infrastructure. These can be further categorized as:
1) Network latency performance, which indicates the delay incurred in the processing of data across the network;
2) Bandwidth performance, which indicates the speed of the network bandwidths including incoming and outgoing bandwidths.
• Resource requirements describe the specifications of resources, such as hardware, software and operating system. Three categories are identified:
1) Hosting environment defines the operating system requirements of the host, such as Windows 7 or Ubuntu Maverick;
2) Hardware capability defines the hardware components, such as CPU core, CPU architecture, RAM, storage space;
3) Software stack indicates the list of software or services that need to be installed on a resource.
• Geographical requirements refer to the location of resources, including data. For example, data must be processed and stored within UK.
• Compliance code requirements refer to the name or code of regulatory, industry or security standard that the infrastructure must comply with, such as HIPAA or ISO27002.
An infrastructure requirement can be either hard or soft. A hard requirement is compulsory and remains invariant over the application’s lifecycle; for example, legislation regulations are compulsory; a soft requirement is desirable and can change or be re-prioritized; for example, budget or performance characteristics may have a degree of flexibility. Each requirement has a priority level, which indicates how importance the
Figure 2. Categories of infrastructure requirements.
requirement is. This can be used during the requirements prioritization and resources filtering phases. Requirements may be inter-dependent. For example, the UK Data Protection Act (a compliance requirement) indicates that no data can be processed or stored outside the UK. This translates to a dependency relationship on geographical requirement. Figure 3 provides an overview of the ontology for infrastructure requirements.
A class restriction is defined to identify the relevant conditions or constraints associated with a requirement (see Figure 3). Each requirement is constrained by at least one restriction. The detailed ontology relationships between infrastructure requirements and restrictions are given below:
• A cost requirement is constrained by cost restrictions. A cost restriction can be a total cost or it can be subdivided into compute costs, software costs, storage costs, or bandwidth costs. Each cost restriction is associated with cost frequency (per hour, per day) and financial cost (amount and currency).
• A performance requirement is constrained by performance related restrictions. Network latency performance is constrained by latency restrictions. Incoming and outgoing bandwidth performances are constrained by bandwidth restrictions. Bandwidth restriction indicates the minimum amount of bandwidth required.
• A resource requirement is constrained by resourcerelated restrictions.
1) The hosting environment is constrained by operating system restriction which specifies the operating system types.
2) The hardware capability is constrained by various hardware restrictions, such as minimum number of CPU cores, CPU speed, CPU architecture type, RAM, and storage space restriction.
3) The software stack is constrained by software restrictions which specify the list of software or services that need to be installed on the resource.
• Geographical requirement is constrained by location restrictions. Location restriction indicates the location of resource or data processing.
• Compliance code requirement is constrained by compliance restrictions, which can be industry’s standard restriction or regulatory restriction.
Deployment specifications are expressed using layers: domain, site, group, and node (see Figure 4). A domain represents the top-layer of the infrastructure deployment layout and has at least one site. A site is composed of one or more groups. A group contains a set of nodes which provide same functionality, such as web servers or databases. A node is a specific type of resource such as a computational unit or storage. The ontologies include requirements which apply to many different layers of the deployment structure. For example, if a location requirement is applied at the domain layer, all sites, groups and nodes within the domain must fulfil the same location constraint; hardware requirements, such as CPU and memory, can be applied at group level or at individual node level.
Once the infrastructure deployment specification is defined, it is then used to search for resources in the lowlevel resource pool.
3.3. Resource Ontology
The resource ontology defines the properties of the resources offered by cloud providers. This layer has been widely investigated elsewhere [5,6]. In the proposed model, we adopt a similar approach as [6] (using the concept of resource capabilities) for describing cloud resources.
A cloud resource is associated with different resource capabilities, which can be storage capability, compute capability, memory capability, software capability and host capability. Storage capability consists of amount of storage space and various storage types, such as local
Figure 3. Overview of infrastructure requirements ontology.
Figure 4. Ontology for infrastructure deployment.
storage or SAN storage; compute capability consists of CPU core, CPU architecture type, CPU speed; memory capability consist of memory size; software capability consist of software name, version, maker and features; host capability consist of operating system type and version. Cloud resource also has other properties such as location and vendor name. An overview of cloud resource ontology is illustrated in Figure 5.
Every cloud resource has a price structure which represents the resource consumption cost. Estimating costs of using cloud resources is a challenging task. Different providers have different means of charging cloud resources. In this paper, we consider normal pay-as-youuse pricing model. Discounted prices, such as Amazon Web Services’ reserved instances or free-tier offers are not considered. We analyse the pricing structures of various providers and propose a simplified ontological pricing model. The cost ontology is shown in Figure 6.
Figure 5. Overview of resource ontology.
In general, prices of consuming cloud resources can be categorized as follows:
• Resource prices refer to the cost of allocating the resources and keeping the resources switched on (i.e. keep-alive). These include machine cost and storage cost.
• Bandwidth prices refer to the cost of data transfer in to and out from the resource. These include data transfer within local network, regional network or internet network.
• Disk prices refer to the cost of disk operations. For example, disk read request and disk write request.
• Miscellaneous prices refer to other costs not included in the above categories, such as the price of allocating additional IP addresses.
The proposed resource and cost ontologies provides a semantic mechanism for annotating cloud resources offered by different providers.
4. A Media Transcoding and Distribution Example
In this section, a media example is used to illustrate how a domain-specific ontology can be translated to a set of infrastructure deployment constraints which are in turn mapped onto actual cloud resources. Media developers need to provision an infrastructure which fulfils transcoding and distribution requirements as well as satisfying cost and delivery time constraints.
4.1. Specifying High-Level Requirements Using Domain-Specific Ontology
Based on the example given in previous section, highlevel media requirements are represented using the proposed domain-specific ontology:
• Converting avi to high definition Windows Media format is a type of Format requirement.
• Supporting playback devices, such as PC, iPad and iPhone, is a type of Device requirement.
• Serving 500 users in UK and 500 users in Singapore are types of Capacity and Location requirement.
• Making the media available before 9 am next morning is a type of Delivery Time requirement.
• Having a budget constraint of £5 per user is a Budget requirement.
Table 1 summarizes the representation of high-level media requirements using domain-specific ontology.