Motion Classification Using Proposed Principle Component Analysis Hybrid K-Means Clustering *

This study investigates and acts as a trial clinical outcome for human motion and behaviour analysis in consensus of health related quality of life in Malaysia. The proposed technique was developed to analyze and access the quality of human motion that can be used in hospitals, clinics and human motion researches. It aims to establish how to widespread the quality of life effects of human motion. Reliability and validity are needed to facilitate subject outcomes. An experiment was set up in a laboratory environment with conjunction of analyzing human motion and its behaviour. Five classifiers and algorithms were used to recognize and classify the motion patterns. The proposed PCA-K-Means clustering took 0.058 seconds for classification process. Resubstitution error for the proposed technique was 0.002 and achieved 94.67% of true positive for total confusion matrix of the classification accuracy. The proposed clustering algorithm achieved higher speed of processing, higher accuracy of performance and reliable cross validation error.


Introduction
This study focuses on investigating the human motion and movement behavior through analyzing their jogging, walking and throwing patterns, to come out with a better solution for movement classification and nature behavior analysis.The methodology of this research is to get the motion pattern through few sensors attachment on skin for processing and analysis.The reviews from previous research on the requirement of experiment design and the current trend of analysis act as guidance to develop a good research framework.
The objective of this study is to investigate the human motion and movement behavior in order to establish how widespread the quality of life effects of motion are by quantifying them.The expected results in terms of the stability, design, efficient control for mobility will help researchers to consider the outcomes of a human motion and movement.This paper presents a novel motion signal processing technique, and presents ideas for further development and recognition, to give researchers ideas of how they can use human movement in related field for product development.
This paper is divided into six sections.The first section mainly introduces the whole study.It provides the general overview of the human motion analysis system.The second section includes the objectives of this study, which describes the aims that needed to be achieved.It also discusses the background studies, literature review and the basic concept in this study.Section 3 discusses about the study implementation and a specification of the experiment environment, thorough discussion on the developmental technique or algorithm and analysis on human motion.Finally, the last two sections provide the conclusions, future developments and possible enhancement and improvement.

Research Review
Accelerometer, gyroscope and compass sensors are the most common devices used in movement detection and analysis system [1].Introduction of human actions into digital domain is a primary driver for innovation of motion functionality.Human motion signal processing technique, which combines inertial measurement units with digital signal processing, enables people readily incorporate motion [2,3].Description in the next subsection provides readers with understanding of the sensors combinations used in motion detection and analysis field [4,5].

Accelerometer
The primary usage of accelerometer is measuring linear acceleration and tilt while velocity can be obtained by a single integration and relative distance by a double integration.The benefit of a accelerometer is that it able di-rectly measure tilt angle and linear distance based on acceleration of gravity [6].The main drawback of the sensor is it unable to distinguish between acceleration due to linear movement and acceleration due to gravity.Problem can be solved by combining accelerometer with gyroscope sensor.

Gyroscope
Gyroscope is mainly used to measure absolute rate of rotation and relative angle by a single mathematical integration.The performance is fast and accurate without corrupted by linear acceleration or magnetic fields.However, sometimes the integration may lead to errors over time but it can be solved by combining with accelerometer.
Data are collected for training and training classifiers.In order to achieve accurate result, data size should be large enough to cover all relevant parameters.Training and testing data should be different.
Pre-processing of signals aims to reduce noise and normalize interests in a data set.
Feature extraction is a process to extract features that characterize the region of interests.There are many types of features such as histogram, shape information, texture information, scale invariant feature transform and many others.Principle Component Analysis (PCA) was used to reduce the number of variables, from six to three; to reduce the complexity of large data set.
Classifiers are trained by training data set.There are two types of classifier: supervised and unsupervised learning.Classifiers used in this study are Fuzzy, proposed PCA-K-Means, C-Means, Naive Bayes and decision tree.Classifier should not just memorize class labels of testing data.This gives 100% accuracy on training data, but may not work on other unseen data.Hence, novel testing data is used to test the performance of trained classifier that able to generalize to novel data.
Cross validation aims to provide more thorough and accurate evaluation for the classification process.The large samples data are divided into a few subsets.The first subset is left out for testing and the rest are used for training.The process is repeated for each of the subset in turn.Average accuracies is achieved from all the runs and this is called N-fold cross validation.

Study Sample
Five healthy volunteers were selected inside university campus for taking part in this study.Their ages are around 20 years old with normal limbs movement and significant mobility in everyday routine independent of any walking aid.

Experimental Setup
For this study, experimental setup was done using a wireless 3-axis accelerometer.This device employs a YEI 3-Space Sensor breakout board for the tri-axial gyroscope, accelerometer, and compass sensors in conjunction with advanced processing and on-board quaternionbased Kalman filtering algorithms to determine orientation relative to an absolute reference in real-time in an enclosure measuring 60 mm × 35 mm × 15 mm.The device was connected to a laptop using a standard USB 2.0 host system wireless asynchronous serial transmission.
The subjects wore a wearable sensor on above right arm which employed of 3 sensors (gyroscope, accelerometer and compass) inside the package.These sensors were attached firmly on subjects' skin with a special designed holder.

Data Collection and Management
In the initial phase of the trial study, experiment was conducted for three activities, there are jogging, walking and throwing.Subjects were asked to perform a normal walking with speed 3.7 ft/s and jogging with speed 6.5 ft/s on a treadmill with regular motor.Throwing was performed by throwing a paper roll 1.50 m apart.These activities were performed in a supervised and comfortable environment with presence of researcher for timestamping the start and end time of activities period.
Subjects were encouraged to perform the jogging, walking and throwing activities at their own pace and convenience.The whole experiment setup place was ensuring a relaxing and natural mood for the sake of subjects for reflective of real world conditions.
The data were transmitted from sensors to the laptop for further processing.

Data Analysis
Data were collected through transmission of a mini wireless dongle from the sensors.They were pre-processing for noise elimination and then extracted for classification as below: 1) Fuzzy C-Means Clustering: This clustering is an iterative process.The parameters required for this clustering are numbers of cluster/class, exponent for the matrix partition, maximum number of iteration and minimum of improvement.First, an initial fuzzy partition matrix was generated and the initial fuzzy cluster centers were calculated.The cluster centers and the membership function Copyright © 2013 SciRes.ENG were updated during each step of iteration to minimize the objective function for the best location for the clusters.The process stopped either once the maximum number of iterations was reached or the minimum amount of improvement specified between two consecutive objective functions was achieved [11][12][13][14].
2) PCA-K-Means Clustering: K-Means Clustering is a function partitions the N X P data matrix into K clusters/classes through a fully vectorized algorithm.N is the number of data points while P is the number of variables.In this project, the sum of all points to cluster centroid was calculated using proposed Euclidean distances for all clusters.The Euclidean distance d between two vectors x and y is: (1) The function was returned the centroids locations for all the clusters until the minimum sums were achieved.PCA was hybrid with K-Means clustering to reduce the variables dimensions.
3) Naïve Bayes: Naive Bayes performs classification works based on diagonal covariance matrix estimations.This classifier assumes the variables are conditionally independent given the class label.It has been found to work well in practice for many large or small data sets.First the classifier was modelled using Gaussian distribution and assumed that multivariate data has normal Gaussian distribution.Then, the classifier was enhanced using kernel density estimation, which is a more flexible nonparametric technique [15].4) Decision Tree: Decision Tree is an algorithm that following simple rules, such as "if the Y-axis gyroscope reading is less than 0.5898, then classify the data as Walking motion."It is a nonparametric technique since it does not require any assumptions about the distribution of the variables in each class.A set of rules was generated by training data.Decision Tree used this set of rules to divide the plane and assign each data to each specific class [16][17][18].
5) LDA and QDA: LDA is Linear Discriminant Analysis and QDA is Quadratic Discriminant Analysis.Data were classified using default LDA.Some data were misclassified by the LDA function with drawing X through the points.The function has separated the plane into region divided by lines, and assigned different regions to different classes.A grid was created for region visualization.For some data as for this project, the variables were not separated well into the correct classes.Hence, QDA was proposed for the data.

Instrument Revision
The preliminary set of outcome measures was shown in this paper.There are 2 sensors used in this experimental setup: gyroscope and accelerometer.The ability of the classifiers in differentiating jogging, walking and throwing patterns were distinguished in discussion part.
Accelerations due to jolting of the sensors if loosely attached may add noise to the signal.The special designed of sensor holder capable attached firmly to the subject's skin to avoid any disturbance.

Result and Discussion
Results of the project were displayed in tabular and graphical form as Figure 1.
Table 1 shows the time consumption for every classifyer used.Proposed PCA-K-Means clustering used 0.058 seconds for the whole pattern recognition process.It is the fastest classification process compared with Fuzzy C-Means (2.32 s), Naive Bayes (3.72 s), decision tree (3.16 s) and normal K-Means statistical toolbox (0.56 s).
Table 2 shows the resubstitution and cross validation errors.PCA-K-Means achieved 0.002 and 0.328 for resubstitution and cross validation errors respectively.
Table 3 shows the confusion matrix and total true positive for every algorithm used.PCA-K-Means and decision tree achieved the highest true positive (accuracy) percentage, 94.67% followed by Naive Bayes kernel density 86%, QDA 77%, Naive Bayes Gaussian kernel 75% and LDA 64.33%.

Fuzzy C-Means Clustering
Figure 2 shows the initial and final fuzzy cluster centers.The bold number represents the final centers after updated from each iteration iteratively.Throwing always has the higher peak then followed by jogging and walking for gyroscope values above 5g.

PCA-K-Means Clustering (Proposed)
Figure 3 shows the final clusters centroids for all of the clusters.The bold "+" marks are the final updated centroids locations for every cluster.

Naïve Bayes
Gaussian distribution and kernel density were hybrid with Naive Bayes for classification.As shown in Table 2, errors of resubstitution and cross validation were reduced for kernel density due to its flexibility characteristic.

Decision Tree
Initially, a full 12 level of pruning with 30 terminal nodes cluttered-looking tree was generated after a series of rules was applied to each data.The data were then performed resubstitution and cross validation error to create a simplest and smallest tree as in Figure 4.

LDA and QDA
For data from this study, LDA was not an appropriate algorithm for classification.QDA perform a quadratic analysis to the data and the resubstitution error and cross validation error were reduced.In order to obtain a clearer and cleaner approximation plot, data received from the sensors were pre-processing using averaging filter to eliminate DC noise and disturbance.

Discussion
Experimentally, K-Means statistical toolbox took shortest time for classification process compared among other classifiers, however, the proposed PCA-K-Means clustering even shorten the process period by 10 times with lowest resubstitution error and highest true positive for confusion matrix by hybrid with PCA .
Decision tree classifier achieved 94.67% true positive as accurate as proposed method, however, decision tree took 3.165 s for 300 data sample size.The proposed classifier is able to process data with 54 times faster than decision tree.Furthermore, the proposed method achieved 31 times accuracy for resubstitution error.In short, decision tree is not appropriate for analyzing large sample data.
LDA, QDA, Fuzzy C-Means, Naive Bayes Gaussian and Naive Bayes kernel density classifiers took longer time (more than one second) for processing, higher resubstitution and cross validation error and lower true positive percentage for confusion matrix.These rates are very important for a testing in large dataset.
There is a lot to do with this study depending on the imagination.One but not the only one straight forward application for this research is motion recognition.It also can be applied on the incredible thing likes gesture recognition, behavioural analysis and gait analysis.
There is also a possibility of incorporating an EEG and ECG into this study.ECG could involve the condition of human body while wearing sensors and EEG could involve condition of human brain activity while performing task.This data would be collected simultaneously using the accelerometer.

Conclusions and Future Work
The sensor is capable to filter and normalized data using Kalman filter.Results presenting in scatter plot successful reveal information needed.The attachment of sensors on subject's skin was firm without significant disturbance.Overall this study completed the objectives from attachment, detection, orientation, transmission, receiving, filtering, and analyzing.
The proposed PCA-K-Means classifier is successfully recognized and classified all three motions data with shortest period, higher accuracy and lower errors.The classifier also appropriate for processing large data set within period.
In order to fully realize this study, there are few things that could be considered, the main feature of interest is the data processing unit.All data are process under the same platform without bias.Further approach need to be taken in order to achieve a higher aim in this research.
As the initial, the study took place in a laboratory environment, it was considered appropriate for the initial phase of the quantitative study to be conducted in a similar environment.Further work is planned to widen the sample and to encompass different environments in both the dynamic and transition activities.

Figure 1 .
Figure 1.Scatter plot for accelerometer and gyroscope data.

Figure 2 .Figure 3 .
Figure 2. 2D initial and final fuzzy cluster centers for two characteristics (AX-AZ, AX-GX, AX-GZ, AZ-GX, AZ-GZ, GX-GZ) of the three types of motions (Walking in red, Jogging in green and Throwing in blue).