Clustering-Inverse : A Generalized Model for Pattern-Based Time Series Segmentation *

Patterned-based time series segmentation (PTSS) is an important task for many time series data mining applications. In this paper, according to the characteristics of PTSS, a generalized model is proposed for PTSS. First, a new interpretation for PTSS is given by comparing this problem with the prototype-based clustering (PC). Then, a novel model, called clustering-inverse model (CI-model), is presented. Finally, two algorithms are presented to implement this model. Our experimental results on artificial and real-world time series demonstrate that the proposed algorithms are quite effective.


Introduction
With the large amounts of time series data arising from various fields in recent years, the time series data mining has emerged as an important research topic in the field of data mining [1][2][3].Especially, as one of the most fundamental tasks in time series data mining, the time series segmentation has attracted extensive attention [4,5].
The conventional time series segmentation algorithms can be classified into the following two main categories [6].The aim of the first category is to create a high level representation of the time series for indexing, clustering, and classification [7,8] and the aim of the second category is to identify switching dynamics in time series [9,10].
In addition to these two main categories of algorithms mentioned above, a kind of novel pattern-based time series segmentation algorithm (PTSS) in [11] and its improved versions [7,8] were developed to segment financial time series.The distinctive characteristic of PTSS is that a pattern time series set (i.e. a pattern template set) is given to control the segmentation.In the pattern time series set, a pattern time series also represents a technique template, which indicates a representative time series segment in a time series.For different practical applications, the corresponding technique templates usually represent the special meaning.The aim of PTSS is to obtain the time series segments which are similar to a certain pattern template in the given pattern time series set.In this paper, we mainly focus on the study of PTSS.
In this study, by comparing PTSS with the prototype-based clustering (PC), we find that PTSS may be interpreted as an inverse problem of PC.According to this interpretation, a generalized model, called Clustering-Inverse model (CI-model), is proposed for PTSS.Then, the main components and processing operations of this model are discussed in detail.Furthermore, a detailed algorithm is presented to put this model into practice.As the matching measure of time series is of fundamental importance in segmentation, we also propose a new Perceptually Important Point (PIP) based Dynamic Time Warping (DTW) measure by integrating PIP identification mechanism [12] with DTW measure [13].The advantage of the proposed measure is that it combines the merits of both PIP mechanism and DTW simultaneously.To investigate the performance of the proposed CI-model, we have applied the proposed algorithms to the real-world time series.
The contributions of this study can be summarized into the following four aspects: The paper is organized as follows.In section 2, a formal description and a new interpretation of PTSS are given.The proposed CI-model is introduced and its main components and processing operations are discussed in section 3. Section 4 presents a detailed algorithm to realize the proposed model.Experimental results are reported in Section 5.The conclusions and some prospects are given in section 6.

Description about PTTS
First, we give a formal description of PTSS (Pattern based Time Series Segmentation).Given a time series     sim  denotes a similarity measure between two time series.

New Interpretation about PTSS
In this subsection, we will discuss the relationship between PTSS and the prototype-based clustering (PC), and then give a new interpretation for PTSS.PC can be expressed as the following problem: Given a dataset D and the number c of its clusters, the aim of PC is to get c optimal prototypes i V (e.g., cluster centers), 1, , i c   .The sample points associated with the same prototype are as compact as possible while the sample points associated with different prototypes are as scattered through the data space as possible.Figure 1(a) shows a simple illustration for PC.Correspondingly, we also give an illustration for PTSS in Figure 1(b).
By comparing Figure 1(a) with Figure 1(b), it is easy to find that the pattern template i P in Figure 1(b) can be interpreted as a prototype i V in Figure 1(a).Meanwhile, the set of time series segments s D in Figure 1(b) can be interpreted as the dataset D in Figure 1(a).The essential difference between PC and PTSS lies in the fact that given the dataset D , the aim of PC is to get the optimal prototypes i V in PC by optimization learning, while given the prototypes i P , the aim of PTSS is to obtain the optimal dataset s D .Hence, we can give the following interpretation: PTSS can be viewed as an inverse problem of PC and here we call it the clustering-inverse problem.segment.Meanwhile, S D and P D can be taken as the dataset and the prototype set for computing the objective function J in Figure 2. The objective function J of the prototype-based clustering is adopted as the optimization objective and the segmentation point set of time series T , as the solution variable, will be updated by an optimization method.In the optimization learning procedure, P D is always unchangeable and the aim of optimization is to obtain the optimal time series segment set S D of T .

The Main Components and Operations in CI-Model
In this subsection, we discuss the main components and operations in the proposed CI-model, including 1) the objective function, 2) the optimization method, 3) the data processing operation, 4) the matching measure, and 5) the initialization operation.

Objective Function
Many objective functions based on different principles have been proposed for PC.In practice, according to the analysis in section 2.2, almost all these objective functions in PC are available for the proposed CI model.Here, we only introduce four representative ones: the objective function of K-means clustering, the objective function of FCM clustering [14], the objective function of fuzzy clustering neural network (FCNN) [15] and the objective function of maximum entropy clustering (MEC) [16].Now, we give a brief description of the four objective functions.K-means clustering is the most classical clustering algorithm in PC.Its objective function can be expressed as where j x is the data vector; i v is the cluster center vector (i.e. the cluster prototype); c is the number of clusters;   , j i D x v denotes the dissimilarity measure between j x and i v .The most commonly used dissimilarity measure in (1) is the Euclidean distance.
The famous FCM clustering is the fuzzy version of K-means clusterig.Compared with K-means, it is more robust to noisy environments in real applications.The objective function of FCM clustering can be expressed as where ij u denotes the fuzzy membership; 1 m  denotes the fuzzy index;   , j i D x v denotes the dissimilarity measure between j x and i v .The update rule of the fuzzy membership ij u in (2) can be formulated as The FCNN clustering also utilizes the fuzzy conception, but the most important basis of FCNN clustering is the neural-network learning.Its objective function can be written as where ij u denotes the corresponding fuzzy membership;   , j i D x v denotes the dissimilarity measure between j x and i v ;  is a constant parameter.The update rule of the fuzzy membership ij u in (4) can be formulated as The MEC clustering is a typical algorithm of the probability-theory based PC algorithms.Its objective function can be expressed as where ij u denotes the corresponding joint distribution probability;  is a constant parameter;   , j i D x v denotes the dissimilarity measure.The update rule of ij u in (6) can be formulated as x v (7) The above four objective functions provide several possible schemes to implement the proposed CI-model.Note here that the objective functions in (1), ( 2) and ( 6) are expected to achieve the minimum values, while the objective functions in ( 4) is expected to achieve the maximum value.

Optimization Method
In the proposed CI-model, the objective function is just the clustering objective function and the solution is the segmentation point set , , , seg po s D q q q   , which can be equivalently written as a vector and n is the length of time series T .
Since the objective function usually can not be written as an analytic expression of the solution , , , s q q q  q  directly, the traditional gradient optimization methods and some other commonly used derivative-based optimization methods are not suitable for this problem.Under these conditions, the following optimization methods can be considered.
1) Dynamic programming.It is the earliest technique utilized to solve the traditional time series segmentation problem [7].A crucial step for dynamic programming methods is to transform the optimization problem into its corresponding dynamic programming equation.The weakness of this kind of method for time series segmentation is the high computational complexity.Meanwhile, it is not a trivial problem to transform the optimization process into its corresponding dynamic programming.
2) Random optimization.It is a simple method for the time series segmentation [17].This method first gives an initial segmentation.Then, in the whole learning procedure, the following operation is repeated: a segmentation point is selected randomly to be replaced with an optimal point obtained by trying all other potential segmentation points.So the computational complexity is also quite high like the dynamic programming method.
3 ) Evolutionary optimization.Evolutionary optimization methods have been extensively studied and utilized in various fields in recent years and their applications in time series segmentation has been demonstrated in [11,12].Compared with the above two optimization methods, the advantages of evolutionary optimization methods are their easy use and high intelligence.Due to the advantages of evolutionary optimization methods, in the following algorithms presented to implement the proposed CI-model, this kind of optimization methods is adopted.

Data Processing
Data processing is an important operation: 1) to transform the time series segments and pattern time series into easily processed data vectors; 2) to facilitate computing the matching measures between the obtained time series segments and the pattern time series.For example, the transformation of the obtained time series segments and the pattern time series into the corresponding vectors with the same dimensional number will facilitate computing their matching measure using Euclidean distances; 3) to effectively reduce the computational complexity of matching measures.For example, the commonly used DTW measure for time series matching needs to solve a dynamic programming problem.With the increase of time series length, the computation time of DTW measure will become very burdensome.Thereby, the data processing is needed to improve the efficiency of DTW measure.
In this study, to realize the above functions we introduce the PIP (Perceptually Important Point) identification mechanism [11,12] as a data processing operation in CI.The PIP mechanism is to identify some perceptually important points in time series to represent the time series.Thus the computational complexity is greatly decreased because the number of PIPs is far smaller than the length of time series.Now we give a brief introduction to the PIP mechanism.
The idea of PIP identification is motivated by the fact that a time series can be usually characterized by a few important points.For example, the head-and-shoulder pattern time series consists of a head point, two shoulder points, and a pair of neck points.These points are perceptually important in the human visual identification process.Therefore, they can similarly be taken into account in the pattern matching process.Given a time series   , its m PIPs can be obtained by the following process.The first two PIPs that are found will be the first and last points of X .The next PIP will be the point in X with the maximum distance to the first two PIPs.The fourth PIP will then be the point in X with maximum distance to its two adjacent PIPs, either between the first and second PIPs or between the second and the last PIPs.The process of locating the PIPs continues until the number is equal to m .Figure 3 shows the process of identifying 7 PIPs from the head-and-shoulder pattern time series with length 60 n  .

Matching Measure
The matching measure of times series is of fundamental importance in time series data mining.The most commonly used measures are Euclidean distance and DTW measure [13].In [11,12] the PIP-based Euclidean distance was proposed, which revealed much better performance in reducing the running time.Here, we first briefly introduce the three matching measures, and then propose a PIP-based DTW measure for time series matching.Our purpose is to present a measure which can simultaneously take advantage of these merits.
Euclidean distance: Given two time series   , n Y y y y   , which have the same length, the Euclidean distance of these two timeseries can be expressed as , The Euclidean distance measure has been extensively used in time series data mining.However, there is an increasing awareness that it is a much brittle measure in time series data mining [18].
DTW measure: To overcome the weakness of Euclidean distance, Berndt and Clifford introduced the Dynamic Time Warping (DTW) measure in the database community [13].DTW measure enables matching similar time series with different lengths.
PIP-based Euclidean distance: A PIP-based Euclidean distance is presented in [11].For two time series   , assuming Y is a pattern time series, the PIP-based Euclidean distance between X and Y , pip D , can be formulated as , , where denotes the new time series containing m PIPs of time series X .
PIP-based DTW measure: Compared with Euclidean distance, the DTW measure has its distinctive advantages, such as robustness, elasticity.However, the DTW meas-ure has an obvious weakness of high computational complexity.To reduce this weakness, a new modified DTW measure, named as PIP-based DTW measure, pdt D , is introduced.This measure can be formulated as where P X and P Y are two new time series which are composed of the obtained PIPs from X and Y , respectively.In this new measure, two adjustable parameters, r and 2 r , are introduced to control the number of desired PIPs, where . Here, , m n are the lengths of X and Y , respectively; and , m n   are the lengths of P X and P Y respectively.The ranges of 1 r and 2 r are dependent on the time series to be segmented.For the time series that changes slowly, we can set much smaller values for 1 r and 2 r .When the time series changes quickly, the 1 r and 2 r should be much larger in order to maintain the segmentation performance.
In our experiments, we find that to set is appropriate in many cases.

Initialization
With different initializations, the obtained clustering prototypes in PC may be different.Likewise, the solutions of PTSS will be influenced by the initialization.
Here two heuristic initialization strategies are proposed for the proposed CI-model.1) Initialization for a desired solution with prior knowledge.With his knowledge, the user can specify the approximate length dlen of the desired times series segments.For example, given a stock index time series with length 3000 l  taken from the daily closing price, if the user wants to get the segments about a month period or three-month period, he sets 30 dlen  and 90 dlen  , respectively.Correspondingly, 1 s l dlen       segmentation points can be initialized with the interval dlen r  , where x     denotes the minimal integer which is bigger than x and r is a random number to control the initial interval between the adjacent segmentation points.To set is appropriate for most cases.
2) Initialization evaluated by a specific performance index.In some cases, little information can be used to set dlen for initializing the segmentation points.In this situation, we suggest to segment the time series by setting different values of dlen , such as setting 100 dlen  , 200 dlen  , and so on.Then, the appropriate initialization can be determined by a specific performance index as the criterion.

General CI-Model Based PTSS Algorithm
In this subsection, the general CI-model based algorithm for PTSS is presented in Table 1.Evolutionary optimization methods have their distinctive advantages, such as much better global search ability.Among various evolutionary optimization methods, Particle Swarm Optimization (PSO) [20] is one of the most extensively studied methods.In the following subsection, we will present such a CI-model based PTSS algorithm using PSO.

PTSS Algorithm Based on CI-Model + SPSO
In this subsection, we introduce the PSO (Particle Swarm Optimization) for CI-model to implement the PTSS.The PSO is a population-based optimization method.The PSO is motivated by the behavior of organisms such as fishing schooling and bird flock.In a PSO system, each individual is taken as a particle and a particle is taken as a candidate solution to the problem at hand.Particles of the population fly around in a multi-dimensional search space, to find out an optimal or sub-optimal solution by competition as well as by cooperation among them.
For simplicity, here, we directly use the well-known Stand Particle Swarm Optimization (SPSO) as the optimization method for CI model.First, we describe the related important conceptions and search strategies of SPSO.Then, a PTSS algorithm based on CI-model + SPSO is presented.

Conceptions about SPSO
In SPSO, the population contains M particles and the th i particle can be described by two state variables: the position vector , , , and the velocity vector   , , , For the PTSS, the segmentation vector , , , of the time series T can be taken as the position variable of the th i particle.The velocity variable i v of the th i particle is usually initialized to be zero vector.In where t denotes the number of generations and   J  denotes objective function.
In SPSO, another very important variable is the global best position variable g p , which describes the best position among all the positions that all particles ever reached and represents the social shared information.The global best position g p is updated in every generation by the following rule.

Evolutionary Rules of SPSO.
With the above concepts, the evolutionary rules of SPSO can be described as (13) where i denotes the th i particle; j denotes the th j dimension; t denotes the th t generation; w denotes the inertia weight, which often is set to be linear decrease in a interval (e.g.[ 0.4, 0.9 ]); and 1 2 , c c are two acceleration constants (often set in the interval (0, 2] ); 1  and 2  are two random numbers in the interval   0 , 1 .When a segmentation point vector i q is taken as a position vector i x in the SPSO, the constraint conditions x n  must be satisfied, where n is the length of the time series to be segmented and s is the number of segmentation points of a segmentation point vector, i.e. the dimensional number of i x .For the constraint conditions in PTSS, we need to make further processing on   , meanwhile, for simplicity, we set 0 Based on the above conceptions and evolutionary rules of the SPSO, a detailed PTSS algorithm based on the CI-model +SPSO is presented in Table 2.

Some Discussions
In the above subsection, we present a PTSS algorithm based on the CI-model.In fact, these two algorithms are only two feasible schemes and can be furthermore improved by different tactics.
In [11], an evolutionary PTSS algorithm is proposed for financial time series segmentation and the corresponding improved versions are presented in [12].Here, we give a brief discussion about the relationship between these PTSS algorithms in [11,12] and the proposed CI-model based PTSS algorithms in this study.In essence, the algorithm in [11,12] optimizes the fitness objective function given in ( 16) by a GA optimizationmethod with the segmentation point set of the time series as the solution.

Table 1. General CI-model based PTSS algorithm.
Step1 1) Given the time series to be segmented and the pattern time series set.2) Select a specific optimization objective function (e.g. the objective function of FCM clustering) and the corresponding update rules of the related variables (e.g.fuzzy membership variables in the FCM objective function).3) Select a matching measure of time series and the corresponding data processing operations.
4) Set the desired approximate segment length dlen and initialize the segmentation point set with dlen .
5) Select the optimization method.

Step2
Use the selected optimization method to optimize the objective function and gain the optimal segmentation point set.

Step3
Use the obtained optimal segmentation point set to get the set of time series segments.

Table 2. PTSS algorithm based on CI-model + SPSO.
Step1 1) Given the time series to be segmented and the pattern time series set.
2) Set the approximately desired segment length dlen .
3) Initialize the state variables of the population.4) Select the objective function and the corresponding update rules of the variables in this objective function.
Step2 1) Select the matching measure and the corresponding data processing operation.2) Transform the pattern time series into the corresponding prototype vectors by a data processing operation.3) Set the termination conditions of the SPSO.

Step3
Obtain the time series segments sets associated with different individuals, and then transform them into the corresponding data sets.

Step4
Update the corresponding variables (e.g. the FCM membership variables).

Step5 Compute the objective function values of all individuals
Step6 If the termination conditions of the SPSO are satisfied, then terminate the learning of the SPSO and go to Step 8; Otherwise, go to Step 7 Step7 Generate the next generation population with the evolutionary rules of the SPSO in ( 11)-( 15); Then, go to Step 3.

Step8
Obtain the optimal individual, i.e. the optimal segmentation point vector.Furthermore, get the set of time series segments associated with the optimal segmentation point vector.
where j S ( 1, 2, , ) are the obtained time series segments and the pattern time series, respectively,

 
Dis  denotes the dissimilarity distance measure.Substituting ( 17) into ( 16), ( 16) can be reformulated as By comparing ( 18) with (1) (i.e. the objective function of K-means clustering), we find that the two objective functions are very similar if the same dissimilarity measure is adopted.The only difference is the coefficient 1 k in (18).In fact, the variable k only fluctuates in a narrow range by some parameter controls [11,12].So the presented algorithm in [11] can be approximately regarded as a special case of the proposed CI-model based PTSS algorithms when the objective function of Kmeans clustering and GA optimization method are adopted for CI-model.
In the exiting time series segmentation algorithms, the Gath-Geva fuzzy clustering algorithm is a representative one that has introduced the clustering technique for time series segmentation [4].In our work, the clustering techniques are also introduced for this purpose.However, the two methods are very different.For the Gath-Geva clustering based method, the representative time series patterns are unknown.By the clustering procedure the representative time series patterns and the corresponding segmentation results are obtained.However, in our work, the proposed time segmentation algorithms are specially designed for the pattern based time series segmentation (PTSS), where the representative patterns have been known and the aim of time series segmentation is to get the similar time series segments which are similar to a certain pattern time series in the given pattern template set.Especially, the Gath-Geva clustering based method is to segment the time series by clustering procedure and the clustering centers can be obtained as the representative patterns simultaneously, while the proposed method can be taken as the inverse procedure of clustering and the clustering centers, i.e. the representative patterns, are given ahead.

Experimental Studies
In this section, the experimental study of the proposed CI-model is carried out.To effectively investigate the performance of the CI-model, we present a performance index in (20) to evaluate the segmentation results.
  At present, there is no performance index that can be proved to be absolutely fair in all cases for evaluating the segmentation results.For example, given two segmented results of a time series in a practical application, the result with the worse performance index may be considered much better by the expert.Therefore, a more elastic and appropriate evaluation index deserves furthermore studying in future.In this study, all the experiments were carried out with the MATLAB code in the computer with 1 G ROM and 1.66 GHz CPU.

Segmentation Results Analysis
In this subsection, to demonstrate the effectiveness of the proposed CI-model, we report segmentation results of the CI-model based PTSS algorithm on a real-world stock time series.First, thea real-world stock time series are briefly described, and then the segmentation results are reported and discussed.

The Real-World Stock Time Series
The adopted real-world stock time series R01 is shown in Figure 4(a) and a pattern template set is given in Figure 4(b).Here the real-world stock time series is adopted from [12].For the real-world time series the pattern template set is used due to the fact that these pattern templates are the important technique templates to segment the stock/index time series for the trend analysis of stock market.

Analyses of Segmentation Results
Here, the segmentation results obtained by the CI-model based PTSS algorithm with the FCM objective function, the pip D measure and the SPSO optimization are adopted for analysis purpose.In Figure 5, the obtained time series segments are labeled with the corresponding patterns.Each series segment is associated with one pattern in Figure 4(b).With the segmentation result, the   obtained time series mainly belong to the three classes associated with the pattern template A, D and F, respectively.From Table 3 we can see that although some time series are classified into the same class, their membership values are different.More higher the membership value, more possible the time segment belongs to the corresponding class.The obtained segmentation result is also quite valuable for further analysis.For example, the expert may analyze the trend of stock market by these time series segments.

Image Segmentation Application
In this subsection, we give an application of the CImodel based PTSS algorithm to gray image segmentation.Image thresholding is one of the important image segmentation methods, which can find some thresholds in the histogram of a gray image.In fact, image thresholding can be taken as a PTSS problem when the histogram of a gray image is viewed as a time series.Especially, a histogram time series usually contains the following representative technique patterns: peak and valley.The peaks and valleys usually indicate the existence of the sooth areas and edges.Therefore, a relative independent region of a gray image should correspond to a histogram time series segment which appears convex and is similar to one of pattern templates in In our experiment, the pattern template set in Figure 6 is adopted for PTSS.In fact, some other similar pattern template set also can be adopted for the histogram time series segmentation.By large amounts of experiments, we find that with the similar pattern template sets, the segmentation results are usually approximately equivalent.In this experiment, the SPSO, the objective function     are adopted for the proposed image segmentation algorithm.Using Snum to denote the desired number of different image sections, the adopted images can be segmented into two sections with 2 Snum  .The right column of Figure 7 shows segmentation results of the four images.We can see that the obtained segmentation results are encouraging.As a novel method for image threshoding, the proposed algorithm is very promising.

Conclusions and Future Work
In this study, a new interpretation for PTSS is presented and then a generalized CI-model is proposed.A detailed algorithm is proposed to implement this model.The proposed CI-model deserves furthermore studying on many aspects in future, such as the objective functions, the optimization methods, and so on.Moreover, it is also very attractive to apply the proposed CI-model based PTSS algorithms to other research fields.For example, by integrating CI-model with the biomedicine knowledge, the proposed algorithms can be adopted to analyze the biomedicine signal.
Although the proposed CI-model for PTSS has shown a promising performance, it still has the following disadvantage.The proposed CI-model based PTSS algorithms usually need more parameters to implement time series segmentation.For example, when the pdt D measure is adopted two parameters for control the number of the selected PISs are required.In order to make the proposed CI-model based PTSS algorithms more efficient, the further study for the choices of these parameters is very valuable.


Give a new interpretation for PTSS;  Propose a generalized CI-model for PTSS;  Propose a PIP-based DTW measure;  Present an algorithm based on the CI-model.

b
is the beginning position of i S in T , 1 1 b  and i e is the ending position of i S in T , k e n  .The obtained time series segments i S from time series T by PTSS may have different lengths and can be classified into different subgroups, which are associated with the corresponding pattern templates.Therefore,

1 .Figure 1 .
Figure 1.The illustrations of (a) the prototype-based clustering PC and (b) the pattern-based time series segmentation PTSS.

Figure 2 .
Figure 2. The framework of the proposed CI model for PTSS.

Figure 3 .
Identification of seven PIPs from the head-and-should pattern time series.
is the length of the warping path in the DTW measure; c and k are the number of the pattern time series and the obtained time series segments, respectively.The purpose of settingeva dtw l D D p is to effectively reduce the influence of the difference in the length of different time series segments.

Figure 4 .
Figure 4.The real-world stock times series and the pattern template set.(a) Real-world stock time series R01; (b) the pattern template set.

Figure 5 .
Figure 5. Segmentation results of the time series R01 with dlen = 180.

Figure 6 .
By taking the image thresholding as a PTSS problem, we propose a PTSS based image thresholding algorithm with the following steps.1) Obtain the histogram of a gray image; 2) Take the gray histogram as a time series and present some representative template patterns; 3) Segment the time series with the CI-model based PTSS algorithm; 4) Take the obtained time series segmentation points as the thresholds for image thresholding.Two gray images, as shown in the left column of Fig- ure 7 are adopted to test the proposed image segmentation algorithm.All the images have the gray level from 0 to 255.The length of histogram time series of each image is 256, as shown in the middle column of Figure 7.

Figure 6 .
Figure 6.Proposed pattern template set for the histogram time series segmentation.

Figure 7 .
Figure 7. Segmentation results of the four gray images by the proposed CI-model based PTSS algorithm.