Received 22 February 2015; accepted 1 April 2015; published 2 April 2015

ABSTRACT

To convert invisible, unstructured and time-sensitive machine data into information for decision making is a challenge. Tools available today handle only structured data. All the transaction data are getting captured without understanding its future relevance and usage. It leads to other big data analytics related issue in storing, archiving, processing, not bringing in relevant business insights to the business user. In this paper, we are proposing a context aware pattern methodology to filter relevant transaction data based on the preference of business.

Keywords:

Context Aware, Pattern Recognizer, Big Data

1. Introduction

There are varieties of diversified portfolio of applications getting deployed in the Enterprise Infrastructure space, and each application has a different trend of arrival patterns which generates machine data that need to be captured and processed to gain business insights. The problem starts from collection and filtering and processing of the data becomes difficult due to the rate in which the data getting generated is huge [1] . This requires an efficient way to interpret and bring relevance to the particular context the business deals with. The preferred way to look at this issue is to bring in relevance when the data are getting generated real time so that only the relevance and needed machine data are getting captured by leaving the unwanted machine data.

The purpose of this Context Aware Transactional framework is to categorize the patterns and filter the machine data based on the relevance of each transaction getting performed. Based on the filtered data, the enterprise can concentrate on how to effectively bring out the business insights within the application by not spending too much on the cost aspects with respect to the data storage and processing [2] .

The objective of this paper is to propose a methodological approach to implement a non-intrusive component which can be plugged into the existing enterprise infrastructure layer to bring out all the insights business wants by capturing only the relevant business oriented machine data [3] . This paper is organized as follows. Section 1 gives an overview of context aware filtering. Section 2 gives the proposed architecture and the multiple phases involved. In Section 3 approaches and the API for context aware filtering are described [4] . Section 4 and Section 5 consist of experimental analysis and the amount of data saved during the data collection gain with respect to storing, archiving, and processing in the context of the component proposed [5] .

2. Non-Intrusive Context Aware Transactional Framework

Context aware filtering is the process of recognizing the machine data based on the pattern. The patterns are application specific based on specifics like business rules, database access related, and external interfaces. The methodology is built in such a way that this component can be deployed as non-intrusive into any of the enterprise layer to generate data insights. The high level logical steps involved in context aware filtering are shown in Figure 1.

2.1. Pattern Builder

In this phase preserving of existing business and technical knowledge are captured will be utilized and key characteristics of the existing application are captured and stored in the master Meta data in the repository.

2.2. Pattern Recognizer

In this phase pattern matching will be applied on the machine data, and the recognized data are retrieved and stored in the desired repositories. The generation of machine data is in multiple phases, so the pattern recognizer will be a logical independent component which can be made as non-intrusive deployment whenever any transaction happens.

2.3. Pattern Filter

In this phase the pattern filter gets applied on to the pattern recognized machine data which properly filters the relevant and store it in the database for further processing.

2.4. Pattern Extractor & Visualizer

In this phase the pattern extractor and visualizer helps the enterprise to devise the strategy based on the business rule to extract data.

3. Proposed Architecture Frameworks

The high level proposed architecture is explained in Figure 2 and the components involved in creating the Context aware filtering are explained in the following sections.

3.1. Channel Listener

This component will act as a listener component to the channels. The channel sends the request based on the request the listener component intelligently forms the triggering point for the Pattern builder to trigger its operations. It acts as a signal sender for the next component to act upon.

3.2. Pattern Composer

This component will also act as intelligent interpreter and filters out the rules present across applications.

3.3. Pattern Builder

This component retrieves the composed pattern from pattern composer and builds a searchable pattern format

which can be directly applied on to the enterprise contextual data getting captured by the Channel Listener component. This component also deals with the intelligent interpretation of the contextual data from the Enterprise with multiple dimensions and variety. The smartness is built into the component itself and different scoring algorithms based implementation is leveraged to achieve the same.

3.4. Pattern Filter

This component applies the filter rules and it has to interact much with the infrastructure component. The filtered data after the appliance of rules will be streamed to Pattern processor component

3.5. Pattern Processor

This component retrieves the filtered contextual data and parses efficiently to aggregate and assemble the data as per the requirements

3.6. Infrastructure Component

This component provides Authorization, Authentication, Logging, Security etc. and it’s visible to all the other components in this framework. This leverages most of the open source libraries for its operation.

3.7. Transaction Log Parser

A portion of transaction log file used for the experimentation has been shown in Figure 3.

3.8. API Details

The API and the corresponding functions are explained in Table 1. These APIs are used in the process of data filtering.

3.9. High Level Logical Details

Properties details and context aware details to filter the relevant data to connect to twitter and also the extraction based on filtering option given in context Data Pattern parameter is defined in Figure 4.

4. Report Analysis

The report analysis for visualization of contextual relevant data and also the percentage of savings before and after context aware filtering is shown in Figure 5 and Figure 6 respectively.

5. Conclusion

Most of the analytics application captures all the data for future analytics, but the key aspect is to bring in the context aware filtering on the data getting generated from multiple sources of applications. This eases out the analytics complexity on the enterprise and brings in better prospects towards visualization of data. The complexity and cost factor involved in data management like storing, archiving, backup, recovery, etc. can be reduced by this framework.

References