An Efficient Agglomerative Clustering Algorithm for Web Navigation Pattern Identification

Web log mining is analysis of web log files with web page sequences. Discovering user access patterns from web access are necessary for building adaptive web servers, to improve e-commerce, to carry out cross-marketing, for web personalization, to predict web access sequence etc. In this paper, a new agglomerative clustering technique is proposed to identify users with similar interest, and to determine the motivation for visiting a website. Using this approach, web usage mining is done through different stages namely data cleaning, preprocessing, pattern discovery and pattern analysis. Results are given to explain how this approach produces tight usage clusters than the existing web usage mining techniques. Rather than traditional distance based clustering, the similarity measure is considered during clustering process in order to reduce computational complexity. This paper also deals with the problem of assessing the quality of user session clusters and cluster validity is measured by using statistical test, which measures the distances of clusters distributions to infer their dissimilarity and distinguish level. Using such statistical measures, it is proved that cluster accuracy is improved to the extent of 0.83, over existing k-means clustering with validity measure 0.26, FCM (Fuzzy C Means) clustering with validity measure 0.56. Rough set based clustering with validity measure 0.54 Generation of dense clusters is essential for finding interesting patterns needed for further mining and analysis.


Introduction
Data mining is the process of extraction of implicit, novel, useful, interesting and understandable patterns of knowledge in large databases.Application of data mining techniques to World Wide Web is referred as web data mining.Web mining is defined as the task of extracting interesting information from World Wide Web.The data available in the web is classified into different categories namely: • Content of pages of websites.
• HTML or XML code for the page.
• Inter page structure is the actual linkage structure between pages of a website.
• The data captured about the user and usage of website.It includes IP addresses, date and time of visit, user Id, user registration data and customer profile information etc.
Based on the class of web data used for mining process, web mining is classified into Web content mining, Web Structure mining, usage mining.
Web content mining is a technique of discovering useful information from web content, data and documents.The web content mining is further divided into agent based content mining and database based content mining.Web structure mining focuses on using the analysis of link structure of web in order to identify relevant document.Data is represented as a tree structure where html tag becomes the root of the tree ad edges represent hyperlink.Web usage mining is the type of web mining activity involving automatic discovery of user access patterns from multiple web servers.Many organizations depend on internet and World Wide Web to conduct business.Organizations often accumulate huge amount of data in day to day activities.Most of the information is usually generated by web servers and collected their access logs.Other sources of information include, referrer logs showing details of pages from which a user comes from, and user registration or survey data [1] [2].Further processing on these data can help these organizations to determine the valuable customers, cross-marketing strategies of multiple products and to promote trade among other things [3].For organizations that sell advertising on world wide web, analysis of user access patterns helps in targeting advertisements to specific group of users.

Background of Study
Hierarchical algorithms group the objects in hierarchical manner.There are two main techniques namely, the bottom-up approach and the top-down approach [4] [5].In case of bottom-up approach, at the beginning are all singleton clusters, but it ends up with single cluster.In top-down method at the beginning of the algorithm there is a single cluster which is partitioned subsequently until singleton clusters are formed.A dendrogram can depict the algorithm.The resulting clusters are determined by breaking up of the dendrogram in a certain level.Many definitions can be used to measure distance between data objects, like Eucledian distance [5], City-Block, Minkowski and so on.The distance between any two clusters can be calculated as the distance between two nearest objects, or as the two farthest clusters centres.The limitation of hierarchical algorithm is that the clustered object cannot be reassigned to another cluster later.
The iterative splitting algorithm [5] starts with a codebook of size one where the only code vector is the centroid of whole training set.This is iteratively enlarged by a splitting procedure until it reaches the desired cardinality.The pairwise nearest neighbor uses an opposite, bottom-up approach to the code book generation.In the initialization step, each training vector is assumed to be its own code vector.At each step two code vectors are merged and the process is repeated until the desired size of code book is reached.The greatest deficiency of PNN is its slow speed.A straight forward implementation of this requires an O(N 3 ) time which is rather slow for large training set.
A lot of previous work has focused on web data clustering.Web data clustering is a technique of grouping web data objects into clusters so that intra cluster object similarity is greater and inter cluster object similarity is lesser.Its goal is to organize data circulated over the web into groups.Vakali [6] categorize web data clustering into two classes 1) users session based and 2) link based.The former uses the web access log and attempts to group users navigation sessions having similar characteristics.Web access log provide information about browsing behavior performed by a user while the user is navigating inside a website [7].
An agglomerative clustering approach in combination with upper similarity approximation is used for mining click stream data.Zdzislaw Pawlak introduced Rough set theory to deal with fuzziness, that is with uncertain and vague data [8].The building block of rough set theory is an assumption that with every set of universe of discourse, some information is associated in the form of data and knowledge.Lower approximation A(X) is union of all elementary sets are included in X.On contrary.the upper approximation A(X) is union of all elemen-tary sets that have non-empty intersection with X.This approach directly or indirectly addresses the issues involved patter N identification.Based on rough set approach, the lower approximation of a concept consists of all objects that are surely belonging to the concept.The upper approximation involves the one that may belong to the concept.This possibility concept degrades the performance of this approach as it reduces the tightness of clusters [9].
In rough set based approach, each click stream transaction is initially a singleton cluster.The similarity upper approximation for each click stream transaction is calculated for a given set of transactions [10] [11].At every step of agglomerative clustering, the clusters are agglomerated based on the similarity upper approximation.However, many number of objects are clustered in same iteration, this speed up factor cannot be compromised with cluster validity [9].
In k means clustering, the number of clusters and cluster seeds are provided initially.Distance is calculated and the objects closer to the seed are clustered on every iteration.At each iteration, new cluster centre is calculated and reassignment of objects is performed.This process is continued until no more merging is possible.Advantage of this algorithm is simplicity and easy implementation.It has limitation of being affected by outliers and noise [12] [13].
Fuzzy C-Means clustering is a technique of finding clusters in data set with fuzziness.In this approach, membership values are assigned for each data object.It indicates the degree of belongingness of an object with a cluster.The objective function in FCM states that, the sum of membership values for each object to all clusters is 1.The problem with this approach is that, noisy objects are also assigned with high membership values if they occur closer to the centre.These unnecessary assignment affects cluster tightness [14] [15].

Summary of Limitations in Existing System
Various limitations identified in the survey of existing systems are summarized in Table 1.

Contributions of Proposed System
The proposed model overcomes all the above drawbacks and provide many advantages as depicted in Table 2.
On summary, the proposed model identified dense clusters with cluster validity index of 0.83.Identifying dense patterns will provide good accuracy during pattern analysis phase, e.g.providing web navigation recommendation for web surfers, e-learners, e-governance etc.

Proposed Web Usage Mining Model
The proposed web mining process involves six major phases namely Data Cleaning, Finding Transaction, Integration, Transformation, Pattern Discovery and Pattern Analysis as given in Figure 1.
At the initial step, the user access information is collected from server log as shown in Figure 2. The access log is used in this algorithm as a major data source [16].The major fields in the access description are: Table 1.Summary of limitations identified in existing systems.

S.No
Clustering Method (Existing) Drawbacks Identified Root Cause
• Time complexity-O(N 3 ) • Pairwise merging of data object lead to slowness

Upper Approximation Based Rough Set Clustering
• Poor cluster validity index value • Possibility factor lead to reductionin cluster tightness
• Formation of loose clusters.
• Need to specify the number of clusters.
• Reassignment of cluster objects lead to slowness • Existence of outliers affect the cluster • Mean value and therefore tightness.

4.
Fuzzy C Means Clustering   • IP address-IP address of machine, that contacted our site.
• Time Stamp-Time stamp of visit as seen by web server.
• Access Request-The type of access requested like GET or POST.
• Result status code-The resulting status code like 200 for success, 404 for not found.
• Bytes Transferred-The number of bytes transferred.This is equal to file size on download.
• Referrer URL-It means to a referring page.This is the page, the visitor was on, when the clicked to come to this site.User Agent-The user agent identifier.The user agent is the tool the user used to refer this site.It is usually a browser.

Data Cleaning
It is the process of extracting only the required portion of data from server log (raw data) by filtering out information irrelevant to the data mining task.
Preprocessing is done in order to decrease mining time and facilitate effectiveness.Web log files contain large amount of erroneous, misleading and not complete [7] [17].This step removes irrelevant data and noisy log entries.Elimination of items deemed irrelevant by checking the suffix of URL name such as gif, jpeg and jpg.On each document download, several log entries such as graphics and CGI scripts are together downloaded.A user will not explicitly request all of the graphics that are in web page that are automatically downloaded due to HTML tags.It does not make sense to include file request that a user does not explicitly request.The HTTP status code can be returned as unsuccessful requests because there may be wrong, missing or unreachable pages or unauthorized requests etc. Enough meta data to map these unauthorized requests into semantically meaningful actions are also eliminated since these are changing from time to time make sense to decision makers.Only the user IP address and web page clicks are extracted as the goal of this paper is to identify users of similar interest based on their click sequence.By reducing the dimension of data the time complexity is also reduced.

Transaction Identification, Integration and Transformation
The log data are partitioned into user sessions, based on IP and duration.Same user can access a web site for several times.The objective is to divide the page access of every user into individual sessions.A user session is defined as relatively independent sequence of web requests accessed by same user.Very short and very long click stream are eliminated as their contribution is less for clustering task [18].Very short click stream will not match with many of clusters.Their similarity contribution is very less.For very long transaction, as they match with most of other click streams they can be placed in any cluster.So such kind of click stream transaction can be filtered out.
Most researchers use heuristic methods to identify web access sessions.Whenever a new IP address comes, it is considered as next session.The goal of transaction identification is to bring out individual set of clicks for every user.It involves combining log information from multiple data sources.The server log can be either a single transaction of many page references or a set of many transactions each consisting of single page reference.
A user session is a click stream of page views for a single user in a website.Each user session is considered as a click stream transaction, which contains the sequence of URLs of a visitor visiting a website.Click stream is a set of URLs followed by a user in a particular website in one session.The preprocessed sequence of transactions are transformed into the form suitable for mining task so as to prepare a formatted data.All the web page clicks by single user are combined to form a click stream transaction sequence.In this way, click stream transaction sequences are formed for different user sessions.Therefore, the formatted data contains only IP and click stream transaction in the required format.

Pattern Discovery and Pattern Analysis
In this step, a variant of PNN based agglomerative clustering technique is used to identify useful patterns like motivation for visiting a particular website.
Structure of Agglomerative Clustering Algorithm: Place each transaction in individual cluster Construct similarity matrix Repeat Find two nearest clusters to be merged Merge the selected clusters Update the data structures Until no more clusters Given any two transactions t and s, the similarity measure is given by ( ) In general, users access are related to their common areas of interest.Actually, the navigation or browsing behavior of users may not be exactly identical, but they have some common interesting characteristics.
Similarity Matrix A similarity Matrix is calculated for all valid transactions.This is a symmetric matrix.This is scanned row by row to find nearest neighbor for each user transaction.Then based on similarity value the pair with highest similarity value is merged.The similarity value is updated for every merge operation as the universe of discourse is changed during merging process.

Experimental Results
The proposed method is implemented using Java and experiments are performed on a click through data set from WAMP server's log.Preprocessing is done to restrict transaction length as small as five click and as large as fifteen clicks.The average weighted length of click is 10.06.Very small and very large click stream may not provide any useful information about users navigation behavior.Thus transaction length having < 5 clicks is considered as short transaction and transaction length with > 15 clicks are considered as long transaction.In preprocessing step, very short and very long transactions are removed from data set.
Tagota suggests a measure that emphasizes cluster tightness or homogeneity [19].
Inter cluster validity of entire data set is referred as, where, N is the cluster count.q is the intra cluster quality.Intra cluster validity of every cluster is defined as: where, |C i | is the size of i th cluster.

( )
, i S X U is the similarity value between element x to the universe of discourse to cluster C i .The input data set has 4 click stream transactions as follows: Register/regport/regform1/regform2 Register/regport/regform1/addcart Register/regform2/p_info Register/addcart/regform2/catalog By rough set based upper approximation method [20], only cluster {t0, t1, t2, t3} is produced with inter cluster tightness measure of about 0.5 i.e. 50% only.
In this way, by comparing similarity values dynamically during clustering process, dense clusters can be formed.Moreover, our method does not suffer from the problem of fixing threshold values and approximations concerned with possibility of being a cluster member.Accurate membership values are considered in this method and dynamically changing values at every merge step, plays a vital role in determining cluster membership.These two techniques are mainly responsible for improving the accuracy of clusters in the proposed approach.
Table 3 and Figure 3 depict the performance evaluation data for the sample data above.

Conclusion and Future Enhancement
Improvement in cluster validity is achieved by implementing exact agglomeration rather than approximate agglomeration.This improvement is only because of dynamically updating similarity values.The neighborhood weightage is updated after merging every cluster pair.In rough set based method, only approximation is done and static neighbor weightage is used.The dynamic updation leads to additional computation time.The overall time complexity of this approach is O(N 2 ).Good cluster validity index value of 0.83 is achieved whereas existing k-means, FCM and Rough set approaches attained only <0.6 index values.The proposed method eliminates tradeoff between Cluster Accuracy and Speed.
It is proposed to do future research on this work to reduce this time complexity by considering small neighborhood.Also, it is proposed to extend this method in future for providing navigation recommendations for e-learning, e-commerce, e-governance etc. Pattern discovery by proposed method can be easily integrated with  rule extraction or sequential patter extraction to develop Navigation Prediction Engines and to find improvement in mining results which can be studied in future.

Figure 1 .
Figure 1.Stages in web usage mining.

Figure 2 .
Figure 2. A sample web log.

Table 2 .
Contributions of proposed system.