_{1}

^{*}

In this paper, we consider a fuzzy c-means (FCM) clustering algorithm combined with the deterministic annealing method and the Tsallis entropy maximization. The Tsallis entropy is a
*q*-parameter extension of the Shannon entropy. By maximizing the Tsallis entropy within the framework of FCM, membership functions similar to statistical mechanical distribution functions can be derived. One of the major considerations when using this method is how to determine appropriate
*q* values and the highest annealing temperature,
*T*
_{high }, for a given data set. Accordingly, in this paper, a method for determining these values simultaneously without introducing any additional parameters is presented. In our approach, the membership function is approximated by a series of expansion methods and the K-means clustering algorithm is utilized as a preprocessing step to estimate a radius of each data distribution. The results of experiments indicate that the proposed method is effective and both
*q* and
*T*
* _{high}* can be determined automatically and algebraically from a given data set.

Techniques from statistical mechanics can be used for the investigation of the macroscopic properties of a physical system consisting of many elements. Recently, research activities utilizing statistical mechanical models or techniques for information processing have become increasingly popular.

Rose et al. [

Tsallis [

One of the major challenges with using Tsallis-DAFCM is the determination of an appropriate value for

Accordingly, we presented a method that can determine both

To overcome this difficulty, in this study, we propose a method that utilizes K-means [

Experiments are performed on numerical data and the Iris Data Set [

Let

be the objective function of FCM, where

On the other hand, the Tsallis entropy is defined as

where

Next, we apply the Tsallis entropy maximization method to FCM [

Then, the objective function in Equation (1) is rewritten as

Under the normalization constraint of

the Tsallis entropy functional becomes

where

where

From Equation (7), the expression for

The performance of Tsallis-DAFCM is superior to those of other entropy-based- FCM methods [

When the temperature is high enough, if the series expansion up to the third order terms is used, Equation (10) becomes

where

Based on the results in Section 3.1, we propose a method for determining both

First, to ensure the convergence of Equation (10), we use the following expression for

where

Then, setting

where

From this equation,

Furthermore, by assuming that the radius

It should be noted that in this equation, for simplicity,

because Equation (7) tends to

By combining the method presented in the previous section with Tsallis- DAFCM, we proposed the following fuzzy c-means clustering algorithm [

In the first algorithm shown in

The second algorithm is the conventional Tsallis-DAFCM algorithm [

1) Set the temperature reduction rate

2) Generate c initial clusters at random locations. Set the current temperature

3) Calculate

4) Calculate the cluster centers using Equation (9).

5) Compare the difference between the current centers and the centers of the previous iteration obtained using the same temperature

turn to Step 2.3.

6) Compare the difference between the current centers and the centers of the previous iteration obtained using a lower temperature

The experimental results in [

To examine the effectiveness of the proposed algorithm, we conducted two experiments.

The first experiment examined whether appropriate

In this experiment, data sets containing (a) three clusters and (b) five clusters were used, as shown in

Dependencies of the maximum, minimum, mean and standard deviation of

In _{max} for example is calculated using Equation (17) as

N | Maximum | Minimum | Mean | Std. deviation |
---|---|---|---|---|

10 | 5.624e−06 | 3.877e−06 | 5.147e−06 | 6.568e−07 |

100 | 6.098e−06 | 3.446e−06 | 5.350e−06 | 6.024e−07 |

1000 | 6.121e−06 | 2.793e−06 | 5.353e−06 | 5.775e−07 |

10,000 | 6.207e−06 | 2.497e−06 | 5.351e−06 | 5.914e−07 |

N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|

5 | 200.4 | 196.8 | 198.8 | 1.7 | |

10 | 204.6 | 196.8 | 200.1 | 2.2 | |

100 | 203.8 | 195.7 | 198.8 | 1.7 | |

1000 | 305.9 | 195.0 | 199.3 | 6.2 | |

10,000 | 313.4 | 194.8 | 199.6 | 8.1 | |

5 | 197.5 | 190.3 | 192.0 | 2.8 | |

10 | 193.3 | 190.7 | 193.8 | 2.7 | |

100 | 198.3 | 189.5 | 193.1 | 3.0 | |

1000 | 292.8 | 188.1 | 194.1 | 8.8 | |

10,000 | 298.0 | 188.2 | 194.0 | 8.6 |

fixing

From

From

Comparing the results in

Accordingly,

Dependencies of the maximum, minimum, mean and standard deviation of

N | Maximum | Minimum | Mean | Std. deviation |
---|---|---|---|---|

10 | 5.878e−06 | 2.686e−06 | 4.030e−06 | 9.264e−07 |

100 | 5.088e−06 | 2.466e−06 | 3.618e−06 | 5.801e−07 |

1000 | 6.738e−06 | 2.316e−06 | 3.608e−06 | 6.117e−07 |

10,000 | 7.060e−06 | 2.118e−06 | 3.608e−06 | 6.320e−07 |

N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|

5 | 343.2 | 116.1 | 236.0 | 99.0 | |

10 | 353.4 | 116.1 | 229.4 | 93.7 | |

100 | 356.4 | 116.0 | 193.8 | 91.0 | |

1000 | 394.2 | 115.7 | 203.8 | 92.3 | |

10,000 | 395.0 | 115.7 | 198.4 | 91.6 | |

5 | 189.6 | 115.3 | 150.4 | 33.2 | |

10 | 151.7 | 115.1 | 121.9 | 13.1 | |

100 | 190.2 | 115.2 | 134.7 | 24.5 | |

1000 | 249.5 | 115.1 | 138.3 | 30.0 | |

10,000 | 279.5 | 115.0 | 134.4 | 29.5 |

Comparing these results with those in

In

Substituting the values of

with

for

The figures show no significant difference between

Compared with the clusters in

From these results, it can be confirmed that

In this experiment, the Iris Data Set [

The maximum, minimum, mean, and standard deviation of

From

N | Maximum | Minimum | Mean | Std. deviation |
---|---|---|---|---|

10 | 1.455e−01 | 8.040e−02 | 1.097e−01 | 1.554e−02 |

100 | 1.455e−01 | 7.409e−02 | 1.081e−01 | 1.765e−02 |

1000 | 1.810e−01 | 5.978e−02 | 1.075e−01 | 1.872e−02 |

10,000 | 1.857e−01 | 5.893e−02 | 1.076e−01 | 1.949e−02 |

N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|

5 | 3.855 | 3.855 | 3.855 | 0.000 | |

10 | 3.855 | 3.855 | 3.855 | 0.000 | |

100 | 4.935 | 3.855 | 3.866 | 0.107 | |

1000 | 4.935 | 3.855 | 3.861 | 0.083 | |

10,000 | 4.935 | 3.855 | 3.862 | 0.085 | |

5 | 2.066 | 2.066 | 2.066 | 0.000 | |

10 | 2.066 | 2.066 | 2.066 | 0.000 | |

100 | 2.066 | 1.849 | 2.064 | 0.022 | |

1000 | 2.066 | 1.849 | 2.063 | 0.026 | |

10,000 | 2.066 | 1.849 | 2.063 | 0.027 |

can be calculated regardless of the value of

It can be found that these tables show that the proposed method gives similar results to those in the Section 5.1, and

It is also found that not only the estimations of the radius are important to improve the accuracy because

The maximum and mean number of data points misclassified by the previous method [

Even though the experiment was repeated 1000 times, the results obtained with the proposed method were almost identical.

By comparing the mean number of misclassified data points of the proposed method with those of the previous method, it can be confirmed the results from both methods are not significantly different when

By comparing the mean number of misclassified data points of the proposed method with those of Tsallis-DAFCM, it can be confirmed the proposed method can get slightly better results. By examining the maximum number of misclassified, we see that Tsallis-DAFCM misclassifies data more often than does the proposed method.

These results confirm that appropriate values of

tering in 1000 trials (Executions were conducted on an Intel(R) Core(TM)2 Duo CPU E6550 @ 2.33 GHz).

From the experimental results in 5.1 and 5.2, the effectiveness of the proposed algorithm using K-means can be evaluated as follows:

1)

2)

3) Much computational time;

4) The numerical data sets and the Iris Data Set can be clustered desirably using

The Tsallis entropy is a q-parameter extension of the Shannon entropy. FCM with the Tsallis entropy maximization has a proper characteristic for clustering, especially when it is combined with DA as Tsallis-DAFCM. The extent of its membership function strongly depends on the parameter

In this study, we proposed a method for approximating the membership function of Tsallis-DAFCM which, by using the K-means method as a preprocessing step, determines

Experiments were performed on the numerical data sets and the Iris Data Set, and showed that the proposed method can more accurately and stably determine

In the future, as described in 5.1, we first intend to explore ways to improve the accuracy of the estimates for

Yasuda, M. (2017) On Utilization of K-Means for Determination of q-Parameter for Tsallis-Entropy-Maximized- FCM. Journal of Software Engineering and Applications, 10, 605-624. https://doi.org/10.4236/jsea.2017.107033

N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|

5 | 1.825 | 1.165 | 1.437 | 0.225 | |

10 | 2.089 | 1.290 | 1.688 | 0.249 | |

100 | 2.823 | 1.029 | 1.667 | 0.320 | |

1000 | 5.699 | 1.006 | 1.631 | 0.374 | |

10,000 | 9.884 | 1.001 | 1.632 | 0.412 | |

5 | 2.256 | 1.820 | 2.043 | 0.160 | |

10 | 2.279 | 1.959 | 2.038 | 0.106 | |

100 | 2.612 | 1.501 | 2.072 | 0.235 | |

1000 | 2.749 | 1.255 | 2.050 | 0.247 | |

10,000 | 2.889 | 1.198 | 2.060 | 0.244 | |

5 | 2.415 | 2.394 | 2.405 | 0.006 | |

10 | 2.415 | 2.372 | 2.397 | 0.014 | |

100 | 2.421 | 2.374 | 2.404 | 0.010 | |

1000 | 2.427 | 1.888 | 2.401 | 0.031 | |

10,000 | 2.430 | 1.827 | 2.400 | 0.040 | |

5 | 2.461 | 2.410 | 2.449 | 0.019 | |

10 | 2.459 | 2.404 | 2.438 | 0.020 | |

100 | 2.461 | 2.404 | 2.440 | 0.021 | |

1000 | 2.476 | 1.909 | 2.435 | 0.046 | |

10,000 | 2.474 | 1.913 | 2.436 | 0.046 |

N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|

5 | 2.266 | 1.042 | 1.616 | 0.498 | |

10 | 2.106 | 1.122 | 1.650 | 0.330 | |

100 | 2.596 | 1.018 | 1.784 | 0.281 | |

1000 | 2.400 | 1.020 | 1.781 | 0.238 | |

10,000 | 8.514 | 1.002 | 1.789 | 0.259 | |

5 | 2.884 | 2.308 | 2.562 | 0.184 | |

10 | 2.816 | 1.218 | 2.514 | 0.181 | |

100 | 3.125 | 1.890 | 2.489 | 0.227 | |

1000 | 3.381 | 1.855 | 2.490 | 0.213 | |

10,000 | 4.355 | 1.730 | 2.482 | 0.220 | |

5 | 3.452 | 2.245 | 2.790 | 0.543 | |

10 | 3.452 | 1.860 | 2.786 | 0.567 | |

100 | 3.453 | 1.841 | 3.005 | 0.525 | |

1000 | 3.454 | 1.765 | 2.946 | 0.535 | |

10,000 | 3.454 | 1.760 | 2.979 | 0.529 | |

5 | 3.456 | 2.966 | 3.240 | 0.223 | |

10 | 3.457 | 3.205 | 3.421 | 0.077 | |

100 | 3.488 | 2.962 | 3.361 | 0.151 | |

1000 | 3.492 | 2.630 | 3.341 | 0.183 | |

10,000 | 3.496 | 2.455 | 3.344 | 0.181 |

N | Maximum | Minimum | Mean | Std. deviation | |
---|---|---|---|---|---|

5 | 1.606 | 1.314 | 1.515 | 0.110 | |

10 | 1.829 | 1.057 | 1.412 | 0.268 | |

100 | 1.952 | 1.011 | 1.382 | 0.218 | |

1000 | 1.993 | 1.011 | 1.408 | 0.209 | |

10,000 | 2.000 | 1.010 | 1.410 | 0.205 | |

5 | 1.920 | 1.649 | 1.821 | 0.093 | |

10 | 1.953 | 1.628 | 1.767 | 0.099 | |

100 | 1.984 | 1.270 | 1.730 | 0.160 | |

1000 | 1.998 | 1.201 | 1.744 | 0.164 | |

10,000 | 2.000 | 1.170 | 1.746 | 0.163 | |

5 | 1.069 | 1.013 | 1.044 | 0.022 | |

10 | 1.105 | 1.013 | 1.050 | 0.030 | |

100 | 1.962 | 1.013 | 1.062 | 0.095 | |

1000 | 1.962 | 1.013 | 1.057 | 0.056 | |

10,000 | 1.962 | 1.013 | 1.056 | 0.055 | |

5 | 1.876 | 1.876 | 1.876 | 0.000 | |

10 | 1.876 | 1.876 | 1.876 | 0.000 | |

100 | 1.993 | 1.876 | 1.877 | 0.012 | |

1000 | 1.993 | 1.876 | 1.878 | 0.012 | |

10,000 | 1.993 | 1.876 | 1.877 | 0.012 |

Method | Maximum | Minimum | Mean | ||
---|---|---|---|---|---|

Previous method ( | 5 | 1.515 | 14 | 14 | 14.00 |

10 | 1.412 | 16 | 15 | 16.00 | |

100 | 1.382 | 16 | 16 | 16.00 | |

1000 | 1.408 | 16 | 14 | 16.00 | |

10,000 | 1.410 | 17 | 16 | 16.00 | |

Previous method ( | 5 | 1.821 | 13 | 13 | 13.00 |

10 | 1.767 | 13 | 13 | 13.00 | |

100 | 1.730 | 14 | 13 | 13.05 | |

1000 | 1.744 | 13 | 13 | 13.00 | |

10,000 | 1.746 | 13 | 13 | 13.00 |

Proposed method ( | 5 | 1.044 | 17 | 17 | 17.00 |
---|---|---|---|---|---|

10 | 1.050 | 17 | 17 | 17.00 | |

100 | 1.062 | 17 | 17 | 17.00 | |

1000 | 1.057 | 17 | 17 | 17.00 | |

10,000 | 1.056 | 17 | 17 | 17.00 | |

Proposed method ( | 5 | 1.876 | 16 | 13 | 13.00 |

10 | 1.876 | 13 | 13 | 13.00 | |

100 | 1.877 | 13 | 13 | 13.00 | |

1000 | 1.877 | 13 | 13 | 13.00 | |

10,000 | 1.877 | 13 | 13 | 13.00 | |

Tsallis-DAFCM | 1.2 | 16 | 16 | 16.00 | |

1.6 | 16 | 14 | 15.91 | ||

2.0 | 14 | 14 | 14.00 | ||

2.4 | 27 | 13 | 13.01 | ||

2.8 | 26 | 13 | 13.01 |

Clustering | ||||
---|---|---|---|---|

5 | 0.000 | 0.000 | 0.049 | |

10 | 0.000 | 0.000 | 0.059 | |

100 | 0.000 | 0.000 | 0.060 | |

1000 | 0.016 | 0.023 | 0.059 | |

10,000 | 0.156 | 0.242 | 0.059 | |

5 | 0.000 | 0.000 | 0.047 | |

10 | 0.000 | 0.000 | 0.047 | |

100 | 0.000 | 0.000 | 0.047 | |

1000 | 0.016 | 0.023 | 0.047 | |

10,000 | 0.156 | 0.219 | 0.047 | |

5 | 0.000 | 0.000 | 0.068 | |

10 | 0.000 | 0.023 | 0.067 | |

100 | 0.000 | 0.188 | 0.068 | |

1000 | 0.016 | 1.695 | 0.068 | |

10,000 | 0.156 | 17.586 | 0.068 | |

5 | 0.000 | 0.000 | 0.047 | |

10 | 0.000 | 0.000 | 0.047 | |

100 | 0.000 | 0.000 | 0.047 | |

1000 | 0.016 | 0.039 | 0.047 | |

10,000 | 0.156 | 0.430 | 0.047 |