Unsupervised Functional Data Clustering Based on Adaptive Weights

Yutong Gao¹, Shuang Chen^2*

¹School of Sciences, Hebei University of Technology, Tianjin, China.
²School of Sciences, Beijing Institute of Petrochemical Technology, Beijing, China.

DOI: 10.4236/ojs.2023.132011                

Abstract

In recent years, functional data has been widely used in finance, medicine, biology and other fields. The current clustering analysis can solve the problems in finite-dimensional space, but it is difficult to be directly used for the clustering of functional data. In this paper, we propose a new unsupervised clustering algorithm based on adaptive weights. In the absence of initialization parameter, we use entropy-type penalty terms and fuzzy partition matrix to find the optimal number of clusters. At the same time, we introduce a measure based on adaptive weights to reflect the difference in information content between different clustering metrics. Simulation experiments show that the proposed algorithm has higher purity than some algorithms.

Keywords

Functional Data, Unsupervised Learning Clustering, Functional Principal Component Analysis, Adaptive Weight

1. Introduction

Functional data analysis (FDA), proposed by J.O. Ramsay [1] , is a high-dimensional data analysis method based on the functionalization of discrete data to describe statistical relationships from a continuous perspective. It is characterized by treating discrete data as a functional whole with an inherently uniform structure, rather than just an arrangement of individual observations. In recent years, the extension of classical multivariate statistical methods to functional data is an important part of data mining research. In the scalar environment, there are many mature multivariate clustering algorithms, such as Affinity Propagation clustering, hierarchical clustering, K-means clustering, DBSCAN clustering, Chameleon clustering, etc. Most of the multivariate cluster analysis methods cannot be directly applied to functional data. The intuitive analysis idea is to discretize functional data based on sampling, and then use multivariate clustering model to cluster discrete data points [2] [3] . Although this can solve the problem that functional data is difficult to apply multivariate clustering algorithms, it also violates the main idea of “taking the whole function as the analysis object”. Tarpey T et al. [4] proposed a clustering algorithm based on function principal points. The clustering results of this algorithm are affected by the number and location of extreme points, therefore it lacks robustness. Tokushige [5] extended the K-means clustering algorithm to functional data, where the cluster center is determined by the minimum value of the objective function. Antoniadis et al. [6] used wavelet analysis techniques in functional data clustering. Wang et al. [7] proposed a weighted clustering algorithm based on functional principal components, which can effectively make up for the limitations of ordinary functional principal component clustering analysis. Delaigle et al. [8] proposed a weighted K-means algorithm by using the Haar base function proposed by Hardle [9] and the unbalanced Haar base function proposed by Fryzlewicz [10] . This algorithm selects the projection function by weighting the K-means clustering criteria and proves that nearly perfect clustering can be achieved by projecting the data onto the chosen finite-dimensional space. Sharp and Browne [11] imposed jointly generalized hyperbolic distributions on projections of basis expansion coefficients into group specific subspaces. Parameter estimation is done through a multicycle ECM algorithm. Wu et al. [12] assumed that the distribution of the cluster members around their projections onto the cluster’s principal curve is an isotropic Gaussian, and that the projections of cluster members onto the cluster’s principal curve are uniformly distributed. The proposed method employe Bayesian Information Criterion to automatically and simultaneously find the appropriate number of features, the optimal degree of smoothing and the corresponding cluster members. It can be noticed that cluster analysis based on continuous perspective and cluster analysis of discrete data still have a lot in common. If infinite-dimensional functional clustering can be transformed into finite-dimensional clustering, then many excellent multivariate clustering analysis can be used in functional data clustering.

2. Preliminaries

2.1. K-Means

The K-means is one of the most popular unsupervised learning algorithms that solve the well-known clustering problem. Let $X=\left\{x_1,\cdots ,x_n\right\}$ be a data set in a d-dimensional Euclidean space ${\text{R}}^d$ . Let $V=\left\{v_1,\cdots ,v_c\right\}$ be the cluster centers. Let $Z={\left[z_{ik}\right]}_{{}_{n\times c}}$ , where $z_{ik}\in \left\{\text{0,1}\right\}$ is binary variable, indicating if the data point $x_i$ belongs to k-th cluster, $k=1,2,\cdots ,c$ . The K-means objective function is

$J\left(z,V\right)={\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{z}_{ik}{\Vert x_i-v_k\Vert }^2$ (1)

The K-means algorithm is iterated through necessary conditions for minimizing the K-means objective function $J\left(z,V\right)$ with updating equations for cluster centers and memberships, respectively, as

$v_k=\frac{{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{z}_{ik}{x}_i}{{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{z}_{ik}}$ (2)

$z_{ik}=\{\begin{array}{l}1\text{if}{\Vert x_i-v_k\Vert }^2=\underset{1\le k\le c}{\min }{\Vert x_i-v_k\Vert }^2\\ 0\text{otherwise}\end{array}$ (3)

where $\Vert x_i-v_k\Vert$ is the Euclidean distance between the data point $x_i$ and the cluster center $v_k$ .

2.2. Functional Principal Component Analysis

Functional principal component analysis (FPCA) is a generalization of principal component analysis. Suppose $x_i\left(t\right)$ is a random process defined on a compact interval T. Let $\mu \left(t\right)$ be the mean function. Let $R\left(s,t\right)=\mathrm{cov}\left\{x\left(s\right),x\left(t\right)\right\}$ be the covariance function. By using the spectral decomposition of the covariance function, we can obtain:

$R\left(s,t\right)={\displaystyle \underset{p=1}{\overset{\infty }{\sum }}}{\lambda }_p{\phi }_p\left(s\right){\phi }_p(\; t\; )$

where ${\lambda }_p$ is a non-negative eigenvalue of $R\left(s,t\right)$ and satisfies ${\lambda }_1\ge {\lambda }_2\ge \cdots \ge 0$ . ${\phi }_p\left(s\right)$ is the corresponding eigenfunction. The eigenfunctions are regarded as a set of basis functions, called principal component basis functions. The principal component basis function combined with the Karhunen-Loeve expansion can represent the infinite-dimensional function data in a limited manner. Project X into the principal component basis function, and the principal component score of the function ${\xi }_p$ is the corresponding coefficient.

${\xi }_p={\displaystyle \underset{\tau }{\int }}X\left(t\right){\phi }_p\left(t\right)\text{d}t$ (4)

Applying the Karhunen-Loeve expansion, we can get

$X\left(t\right)=\mu \left(t\right)+{\displaystyle \underset{p=1}{\overset{\infty }{\sum }}}{\xi }_p{\phi }_p\left(t\right)$ (5)

3. Unsupervised Functional Data Clustering Based on Adaptive Weights

Multivariate clustering analysis is difficult to directly extend to functional data clustering. But from the knowledge of principal component analysis in the previous section, functional data $X\left(t\right)$ is equivalent to its principal component score ${\xi }_i={\left({\xi }_{i1}\mathrm{,}\cdots \mathrm{,}{\xi }_{id}\right)}^{\text{T}}$ . Since the variance contribution rates of the principal component scores are different, the importance of different principal components is obviously different. It is difficult to achieve an ideal classification effect, if we ignore the objective differences in the efficiency of different principal component scores, and use the principal component score to represent the sample function for clustering [13] . K-means cluster analysis is not completely unsupervised learning, because it requires the number of clusters to be given first. Therefore, we use functional principal component analysis to map functional data from infinite-dimensional space to finite-dimensional space, and then use unsupervised functional data clustering based on adaptive weights (UFDCAW) to cluster ${\xi }_i={\left({\xi }_{i1}\mathrm{,}\cdots \mathrm{,}{\xi }_{id}\right)}^{\text{T}}$ on the finite-dimensional space. Finally, the clustering result of the functional data $X\left(t\right)$ is obtained.

Let $\xi =\left\{{\xi }_1,\cdots ,{\xi }_n\right\}$ which is equivalent to $X\left(t\right)=\left\{X_1\left(t\right),\cdots ,X_n\left(t\right)\right\}$ be the data on the d-dimensional Euclidean space ${\text{R}}^d$ . Let $V=\left\{v_1,\cdots ,v_c\right\}$ be the set of c class centers. Let $u={\left\{u_{ik}\right\}}_{n\times c}$ be the degree of membership function matrix, where $u_{ik}$ is the membership degree of ${\xi }_i$ belonging to the k class. For any

$i\mathrm{,}k$ , there is $0\le u_{ik}\le 1$ and ${\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}=1$ . Let $a=\left({\alpha }_1\mathrm{,}\cdots \mathrm{,}{\alpha }_c\right)$ be a set of mixed proportions, where ${\alpha }_k$ represents the probability that data point belongs to the k class, and satisfies ${\displaystyle \underset{k=1}{\overset{c}{\sum }}}{\alpha }_k=1$ . $-{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{\alpha }_k\ln {\alpha }_k$ is the entropy. The objective function of unsupervised functional data clustering based on adaptive weights is

$J\left(u,V,\alpha \right)={\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}^m{d}_{ik}^2-n\beta {\displaystyle \underset{k=1}{\overset{c}{\sum }}}{\alpha }_k\ln {\alpha }_k-\gamma {\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}\ln {\alpha }_k$ (6)

where $d_{ik}$ is the distance from the data point ${\xi }_i$ to the class center $v_k$ . Considering the difference in the scoring efficiency of different principal components, let

$d_{ik}=\sqrt{{\displaystyle \underset{p=1}{\overset{d}{\sum }}}\left({\omega }_p{\left({\xi }_{ip}-v_{kp}\right)}^2\right)}$

where ${\omega }_p={\lambda }_p/{\displaystyle \underset{p=1}{\overset{d}{\sum }}}{\lambda }_p$ is the adaptive weight of the p item of the principal component score.

The Lagrangian function of formula (3.1) is

$\begin{array}{c}\tilde{J}\left(u,V,\alpha ,r_1,r_2\right)={\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}^m{d}_{ik}^2-n\beta {\displaystyle \underset{k=1}{\overset{c}{\sum }}}{\alpha }_k\ln {\alpha }_k-\gamma {\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}\ln {\alpha }_k\\ -r_1\left({\displaystyle \underset{k=1}{\overset{c}{\sum }}}{\alpha }_k-1\right)-r_2\left({\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}-1\right)\end{array}$ (7)

First, we take the partial derivative of Largrangian (3.2) with respect to $u_{ik}$ , we obtain

$\frac{\partial \tilde{J}\left(u,V,\alpha ,r_1,r_2\right)}{\partial u_{ik}}=m{u}_{ik}^{m-1}{d}_{ik}^2-\gamma \ln {\alpha }_k-r_2=0$ (8)

Thus, we have

$u_{ik}={\left(\frac{\gamma \ln {\alpha }_k+r_2}{m}\right)}^{\frac{1}{m-1}}{d}_{ik}^{-\frac{2}{m-1}}$

From ${\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}=1$ , we can get

$u_{ik}=\frac{1}{{\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\left(d_{ik}/d_{is}\right)}^{\frac{2}{m-1}}}$ (9)

Then we get the update equation for the center point $v_k$ as follows:

$v_k=\frac{{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{u}_{ik}^m{\xi }_i}{{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{u}_{ik}^m}$ (10)

We next take the partial derivative of the Lagrangian (3.2) with respect to ${\alpha }_k$

$\frac{\partial \tilde{J}\left(u,V,\alpha ,r_1,r_2\right)}{\partial {\alpha }_k}=-n\beta \left(\ln {\alpha }_k+1\right)-\gamma {\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{{\alpha }_k}-r_1=0$ (11)

then we get the updating equation for ${\alpha }_k$ as follows:

${\alpha }_k^{\left(t+1\right)}={\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{n}+\frac{\beta }{\gamma }{\alpha }_k^{\left(t\right)}\left(\ln {\alpha }_k^{\left(t\right)}-{\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}\right)$ (12)

where t is the number of iterations of the algorithm.

${\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s\ln {\alpha }_s$ is the weighted mean of $\ln {\alpha }_k$ in equation (3.7). For the k-th

mixing proportion ${\alpha }_k^{\left(t\right)}$ , if $\ln {\alpha }_k^{\left(t\right)}$ is less than the weighted mean, then the new ${\alpha }_k^{\left(t+1\right)}$ will be less than the ${\alpha }_k^{\left(t\right)}$ . That is, the smaller proportion will decrease,

and then competition will occur. If ${\alpha }_k\le 0$ or ${\alpha }_k<\frac{1}{n}$ for some k, they are

considered to be illegitimate proportions. In this situation, we discard those clusters and then update the cluster number $c^{\left(t\right)}$ to be

$c^{\left(t+1\right)}=c^{\left(t\right)}-\left|\left\{{\alpha }_k^{\left(t+1\right)}|{\alpha }_k^{\left(t+1\right)}<\frac{1}{n},k=1,\cdots ,c^{\left(t\right)}\right\}\right|$ (13)

where $\left|\left\{\right\}\right|$ denotes the cardinality of the set $\left\{\right\}$ . After updating the number of clusters $c^{\left(t+1\right)}$ , the remaining mixing proportion ${\alpha }_k^{\left(t+1\right)}$ and corresponding $u_{ik}^{\left(t+1\right)}$ need to be re-normalized by

${\alpha }_k^{\ast }=\frac{{\alpha }_k^{\ast }}{{\displaystyle \underset{s=1}{\overset{c^{\left(t+1\right)}}{\sum }}}{\alpha }_s^{*}}$ (14)

$u_{ik}^{\ast }=\frac{u_{ik}^{\ast }}{{\displaystyle \underset{s=1}{\overset{c^{\left(t+1\right)}}{\sum }}}{u}_{is}^{\ast }}$ (15)

We next concern about the parameter learning of $\gamma$ and $\beta$ for the two terms of ${\displaystyle {\sum }_{i=1}^n}{\displaystyle {\sum }_{k=1}^c}{z}_{ik}\ln {\alpha }_k$ and ${\displaystyle {\sum }_{i=1}^n}{\displaystyle {\sum }_{k=1}^c}{z}_{ik}\ln {\alpha }_k$ . Based on some increasingly learning rates of cluster number with ${\text{e}}^{-c^{\left(t\right)}/100},{\text{e}}^{-c^{\left(t\right)}/250},{\text{e}}^{-c^{\left(t\right)}/500},{\text{e}}^{-c^{\left(t\right)}/750}$ , and ${\text{e}}^{-c^{\left(t\right)}/1000}$ , it is seen that ${\text{e}}^{-c^{\left(t\right)}/100}$ decreases faster, but ${\text{e}}^{-c^{\left(t\right)}/500}\mathrm{,}{\text{e}}^{-c^{\left(t\right)}/750}\mathrm{,}{\text{e}}^{-c^{\left(t\right)}/1000}$ decreases slower [14] . We suppose that the parameter $\gamma$ should not decrease too slow or too fast, and so we set the parameter $\gamma$ as

${\gamma }^{\left(t\right)}={\text{e}}^{-c^{\left(t\right)}/250}$ (16)

To guarantee condition $\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t+1\right)}\le 1$ , the parameters $\beta$ are defined as

${\beta }^{\left(t+1\right)}=\min \left(\frac{{\displaystyle \underset{k=1}{\overset{c}{\sum }}}\exp \left(-n\eta \left|{\alpha }_k^{\left(t+1\right)}-{\alpha }_k^{\left(t\right)}\right|\right)}{c},\frac{\gamma \left(1-{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{n}\right)}{-\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t\right)}\left({\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}\right)}\right)$ (17)

where $\eta =2/d^{\lfloor d/2-1\rfloor }$ and $\lfloor a\rfloor$ represents the largest integer that is no more than a and t denotes the iteration number in the algorithm. In Equation (3.12), if $\left|{\alpha }_k^{\left(t+1\right)}-{\alpha }_k^t\right|$ is large, then $\beta$ is smalll to maintain stability. Although functional principal component analysis expands the function data from infinite dimension to finite dimension, the dimension of the sample is still relatively high, so we add $\eta =2/d^{\lfloor d/2-1\rfloor }$ adjust parameter $\beta$ to Equation (3.12).

In our setting, we use all data points as initial means with $v_k={\xi }_k$ , i.e. $c^{\left(0\right)}=n$ , and we use ${\alpha }_k^{\left(0\right)}=1/n$ as initial mixing proportions. Thus, the proposed unsupervised Learning functional data clustering based on adaptive weights can be summarized as Algorithm 1.

4. Theoretical Properties

Proposition 1. Given the iterative formula ${\alpha }_k^{\left(t\right)}$ that minimizes the objective

function $J\left(u,V,\alpha \right)={\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}^m{d}_{ik}^2-n\beta {\displaystyle \underset{k=1}{\overset{c}{\sum }}}{\alpha }_k\ln {\alpha }_k-\gamma {\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}\ln {\alpha }_k$ , if there exists k such that $\ln {\alpha }_k^{\left(t\right)}$ is less than its weighted mean, then ${\alpha }_k^{\left(t+1\right)}<{\alpha }_k^{\left(t\right)}$ .

proof.

$\begin{array}{c}{\alpha }_k^{\left(t+1\right)}={\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{n}+\frac{\beta }{\gamma }{\alpha }_k^{\left(t\right)}\left(\ln {\alpha }_k^{\left(t\right)}-{\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}\right)<{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{n}+\frac{\beta }{\gamma }{\alpha }_k^{\left(t\right)}\left(\ln {\alpha }_k^{\left(t\right)}-\ln {\alpha }_k^{\left(t\right)}\right)\\ ={\alpha }_k^{(\; t\; )}\end{array}$

That is, there exists k such that $\ln {\alpha }_k^{\left(t\right)}<{\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}$ is satisfied, we can get ${\alpha }_k^{\left(t+1\right)}<{\alpha }_k^{\left(t\right)}$ . □

Proposition 2. Given the iterative formula ${\alpha }_k^{\left(t\right)}$ that minimizes the objective

function $J\left(u,V,\alpha \right)={\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}^m{d}_{ik}^2-n\beta {\displaystyle \underset{k=1}{\overset{c}{\sum }}}{\alpha }_k\ln {\alpha }_k-\gamma {\displaystyle \underset{i=1}{\overset{n}{\sum }}}{\displaystyle \underset{k=1}{\overset{c}{\sum }}}{u}_{ik}\ln {\alpha }_k$ , if $\beta \le -\gamma \left(1-{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{n}\right)/\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t\right)}\left({\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}\right)$ , then $\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t+1\right)}\le 1$ .

proof.

$\begin{array}{c}\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t+1\right)}=\underset{1\le k\le c}{\max }{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{n}+\frac{\beta }{\gamma }\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t\right)}\left(\ln \underset{1\le k\le c}{\max }{\alpha }_k^{\left(t\right)}-{\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}\right)\\ <\underset{1\le k\le c}{\max }{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{n}+\frac{\beta }{\gamma }\left(-\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t\right)}\left({\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}\right)\right)\\ <\frac{1}{n}\underset{1\le k\le c}{{\displaystyle \max }}{\displaystyle \underset{i=1}{\overset{n}{\sum }}}{u}_{ik}-\frac{\gamma \left(1-{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{n}\right)\left(-\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t\right)}\left({\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}\right)\right)}{\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t\right)}\left({\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}\right)\gamma }\\ =1\end{array}$

Therefore, when

${\beta }^{\left(t+1\right)}=\min \left(\frac{{\displaystyle \underset{k=1}{\overset{c}{\sum }}}\exp \left(-n\eta \left|{\alpha }_k^{\left(t+1\right)}-{\alpha }_k^{\left(t\right)}\right|\right)}{c},\frac{\gamma \left(1-{\displaystyle \underset{i=1}{\overset{n}{\sum }}}\frac{u_{ik}}{n}\right)}{-\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t\right)}\left({\displaystyle \underset{s=1}{\overset{c}{\sum }}}{\alpha }_s^{\left(t\right)}\ln {\alpha }_s^{\left(t\right)}\right)}\right)$

holds, $\underset{1\le k\le c}{\max }{\alpha }_k^{\left(t+1\right)}\le 1$ can be guaranteed. □

5. Simulation Study

In this section, we use simulation experiments to verify the performance of the proposed clustering algorithm and whether it can calculate the optimal number of clusters c. At the same time, we compare with fuzzy C-means (FCM) and adaptive weighting functional clustering (AWFC). The result is evaluated using cluster purity. Purity is calculated using Equation (5.1):

$\text{purity}\left(\Omega \mathrm{,}C\right)=\frac{1}{N}{\displaystyle \underset{K=1}{\overset{K}{\sum }}}\underset{1\le j\le J}{\max }\left|w_k\cap c_j\right|$ (18)

where $\Omega =\left\{w_1\mathrm{,}\cdots \mathrm{,}{w}_K\right\}$ is the cluster division, $C=\left\{c_1\mathrm{,}\cdots \mathrm{,}{c}_J\right\}$ is the real class division. The value range of purity is $\left[\mathrm{0,1}\right]$ , the larger the value, the better the clustering performance.

Assume that the three functional data for the simulation experiment are: $X_1\left(t\right)=t+t^3/\sqrt{5}$ , $X_2\left(t\right)=t+\sin \left(\pi t\right)/\sqrt{5}$ , $X_3\left(t\right)=t+{\left(t-2\right)}^3/\sqrt{5}$ , $t\in \left[\mathrm{0,1}\right]$ . $Y_{ij}=X_i\left(t_{ij}\right)+{\epsilon }_{ij}$ is the observation value of the function data with error, and the error ${\epsilon }_{ij}$ obeys a normal distribution with mean 0 and variance 0.02. In order to ensure that the FCM clustering algorithm is available when simulating experiments, we assume that $t_{ij}$ is 100 equally spaced observation nodes on the interval $\left[\mathrm{0,1}\right]$ . The observation sample size of each function data in the simulation experiment is the same, and the total sample size is 30, 300, 600, in order to compare the effectiveness of the clustering algorithm under different sample sizes. The figure shows the clustering data when $n=300$ . Figure 1(a) is the observation value with noise. Figure 1(b) is the function data generated using the B-spline basis.

We can see that the three types of curves are increasing functions from Figure 1(a) and Figure 1(b), but there are significant differences in their derivatives. Following the steps of UFDCAW, the scatter plot of functional principal component scores of 3rd and 5th iterations for a sample size of 300 is shown in Figure 2. From Figure 2(b), we can see that the number of clusters decreases rapidly from 300 to 165 after 3 iterations. The algorithm ends after five iterations and obtains the correct number of clusters $c=3$ and three class centers. The purity of clustering is 0.993.

Table 1 shows the average of the results of 100 replicate experiments using FCM, AWFC and UFDCAW on simulated data with sample sizes of 30, 300 and 600, judged by the purity of the clustering results.

The data presented in Table 1 shows that UFDCAW has higher cluster purity than the FCM and AWFC for sample sizes of 30, 300 and 600. The results of the comparison show that: 1) UFDCAW has higher cluster purity than FCM and can display sample distribution characteristics in low dimensional space. 2) FCM and AWFC required random initial class centres, therefore the cluster prurity will be low when algorithm selects an unsuitable class center. UFDCAW is more

Table 1. Comparison of clustering algorithm results.

Figure 1. (a) Observation value with noise, (b) function data for a sample size of 30.

Figure 2. Scatter plot of the functional principal component scores of data with a sample size of 30 (a) $t=0$ , (b) $t=3$ , (c) $t=5$ .

stable because there is no need to select the initial clustering center. 3) UFDCAW can be iterated to obtain the number of clusters.

6. Conclusion

At present, the data we collect is becoming more and more complicated. Using multivariate statistical models to deal with complicated data will cause problems such as dimensional disaster, and functional data emerges as the times require. Based on the former studies, we expand the K-means clustering algorithm based on adaptive weights. It not only reduces the influence of noise, but also can find the optimal number of clusters using the algorithm when the initialization parameters are missing. It is verified by simulation that the algorithm proposed in this paper can obtain more accurate clustering results and has a certain degree of feasibility. But, the algorithm is slightly more complicated, and some information will be lost when using basis function fitting and functional principal component analysis, which needs to be further improved.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Ramsay, J.O. (1982) When the Data Are Functions. Psychometrika, 47, 379-396. https://doi.org/10.1007/BF02293704
[2]	Bouveyron, C., Girard, S. and Schmid, C. (2007) High-Dimensional Data Clustering. Computational Statistics & Data Analysis, 52, 502-519. https://doi.org/10.1016/j.csda.2007.02.009
[3]	Zhu, J. and Chen, K. (2007) The Cluster Analysis of Panel Data and Its Application. Statistical Research, 24, 11-14.
[4]	Tarpey, T. and Kinateder, K.K.J. (2003) Clustering Functional Data. Journal of Classification, 20, 93-114. https://doi.org/10.1007/s00357-003-0007-3
[5]	Tokushige, S., Yadohisa, H. and Inada, K. (2007) Crisp and Fuzzy K-Means Clustering Algorithms for Multivariate Functional Data. Computational Statistics, 22, 1-16. https://doi.org/10.1007/s00180-006-0013-0
[6]	Antoniadis, A., Brossat, X., Cugliari, J. and Poggi, J.M. (2013) Clustering Functional Data Using Wavelets. International Journal of Wavelets, Multiresolution and Information Processing, 11, 1350003. https://doi.org/10.1142/S0219691313500033
[7]	Wang, D., Zhu, J. and Wang, D. (2015) Research of Clustering Analysis for Functional Data Based on Adaptive Weighting. Journal of Applied Statistics and Management, 34, 84-92.
[8]	Delaigle, A., Hall, P. and Pham, T. (2019) Clustering Functional Data into Groups by Using Projections. Journal of the Royal Statistical Society: Series B: Statistical Methodology, 81, 271-304. https://doi.org/10.1111/rssb.12310
[9]	Hardle, W., Kerkyacharian, G., Picard, D. and Tsybakov, A. (2012) Wavelets, Approximation, and Statistical Applications. Springer Science & Business Media, Berlin.
[10]	Fryzlewicz, P. (2007) Unbalanced Haar Technique for Nonparametric Function Estimation. Journal of the American Statistical Association, 102, 1318-1327. https://doi.org/10.1198/016214507000000860
[11]	Sharp, A. and Browne, R. (2021) Functional Data Clustering by Projection into Latent Generalized Hyperbolic Subspaces. Advances in Data Analysis and Classification, 15, 735-757. https://doi.org/10.1007/s11634-020-00432-5
[12]	Wu, R., Wang, B. and Xu, A. (2022) Functional Data Clustering Using Principal Curve Methods. Communications in Statistics-Theory and Methods, 51, 7264-7283. https://doi.org/10.1080/03610926.2021.1872636
[13]	Wang, D., Zhu, J. and Xie, B. (2012) Research on Clustering Analysis for Functional Data Based on Adaptive Iteration. Statistical Research, 4, 91-96.
[14]	Sinaga, K.P. and Yang, M.S. (2020) Unsupervised K-Means Clustering Algorithm. IEEE Access, 8, 80716-80727. https://doi.org/10.1109/ACCESS.2020.2988796