Journal of Software Engineering and Applications, 20 13, 6,34-41
doi:10.4236 / j sea.2013.67B007Published Online July 2013 (http://www.scirp.org/journal/jsea)
Copyright © 2013SciR es. JSEA
Using Optimized Distributional Parameters as Inputs in a
Sequential Unsupervised and Supervised Modeling of
Sunspots Data
K.Mwitondi1, J. Bugrien2, K. Wang3
1Sheffield HallamUniversity, Department of Computing; 2Statistics Department, BenghaziUniversity; 3Radio and Space Services,
Australian Bureau of Meteorology.
Received June,2013
ABSTRACT
Detecting natura lly ar isin g structure s in data iscentral to kno wledge extraction from data. In most applications, the main
challenge is in the choice of the appropriate model for exploring the data features. The choice is generally poorly under-
stood and any tentative choice may be too restrictive. Growing volumes of data, disparate data sources and modelling
techniques entail the need for model optimization via adaptability rather than comparability. We propose a novel
two-stage algorithm to modelling continuous data consisting of an unsupervised stage whereby the algorithm searches
thro ugh t he da ta for optimal parameter values and a supervised stage that adapts the parameters for predictive mode lling.
The method is implemented on the sunspots data with inherently Gaussian distributional properties and assumed
bi-modality. Optimal values separating high from lows cycles are obtained via multiple simulations. Early patterns for
each recorded cycle reveal that the first 3 years provide a sufficient basis for predicting the peak. Multiple Support
Vector Machine runs using repeatedly improved data parameters show that the approach yields greater accuracy and
reliability than conventional approaches and provides a good basis for model selectio n. Mo del reliability is establis hed
via multiple simulations of this type.
Keywords:Clustering; Data Mining; Density Estimatio n; EM Algorithm; S unspots; Supervised M odelling; Support
Vector Machines; Unsupervised Modelling
1. Introduction
Many real-life problems are tackled via knowledge ex-
traction from data – a process typically associated with
detecting naturally arising structures in the data. A typi-
cal example is the sunspots dataset [11] – an average
oscillating sequence of the beginning and ending periods
of solar cycles with an approximate periodicity of 11
years [7]. Recorded sunspots span across the first cycle
(Ma rch 17 55 to June 1766 ) to the first fe w mont hs of t he
current (24th) cycle. Clustered in non-random positions
above and below the equator, the spots are generated by
interactions between the sun's surface plasma and its
magnetic field [19 and 22]. Solar magnetic activity
cycles have attracted the attention of scientists for many
years. Solar flares, for instance, affect our planet in dif-
ferent ways - including ejecting plasma and energetic
particles and potentially causing geomagnetic storms and
damaging satellites [16]. The paper is motivated by the
documented effects of sunspots on terrestrial conditio n s.
Correlations between space and terrestrial weather have
been indicated in solar studies dating back many years
[13, 18 and 20]. Climatic variations in Lapland via co m-
plex variations in the atmosphere, lunar gravitation and
solar activity have also been explained [11].
This paper will be subjecting sunspots data to a se-
quential analysis involving unsupervised and supervised
modeling. The two concepts represent the typical data
mining problems – data c lustering and cla ssification. T he
primary goal of the former is to partition a given dataset
with a known or unknown distribution into subgroups in
such a way that data points in each group are as homo-
geneous as possible while those in different groups are as
heterogeneous as possible. The method is typically ap-
plied in pro ble ms in which t her e is no clear mathematica l
formulation for describing the underlying structures.
Various approaches to data clustering have been studied
and are well-documented in the literature [21, 17 and 6].
However, determining the number of naturally arising
structures in data remains a daunting challenge among
the data science community. Many clustering tools in the

Using Optimized Distributional Parameters as Inputs in a Sequential Unsupervised
and Supervised Modeling of Sunspots Data
Copyright © 2013SciR es. JSEA
35
literature are based on the conventional mechanics of
minimization of the distances between data points – a
feature which inherently constitutes the same challenge
the methods are designed to address – that is, determin-
ing the optimal number of clusters. The primary goal of
the latter is to allocate new cases in known classes and
one of its main challenges is balancing model accuracy
and reliability.
Let a dataset of independently identically distributed
random vectors
{ }
12 1
,,, ,
d
nn
xxx x
−
…∈
represent fea-
tures an underlying density function. The main features
of interest may include modes (local maxima), an-
ti-modes (local minima) and bumps - regions where the
second derivative is negative. In an exploratory setting,
the number and locations of these features are not known
a priori.Many real-life data take this form and with large
volumes of data generated from different sources and
inputted into different models, we are constantly faced
with the challenge to d eter mine op timal stationer y points.
The challenge is to address model complexity via adap-
tability rather than comparability. In other words, we
seek to minimise inherent randomness in tra ini ng and test
data via novel ad aptive metho ds of data ana lysis [10 and
1].
This paper proposes a novel approach to detecting na-
turally arising structures in data that searches for genera-
lising parameter levels and adapts them to supervised
modeling. Its main research problem is to develop an
algorithm for predicting future cycles given historical
solar activity d ata. We tr y to addre ss this proble m via the
following objectives.
1) To determine naturally arising s t ruc t ur es i n t he d a ta .
For simplicity, we shall be seeking to identify and sepa-
rate high from low solar activity cycles. This objective
constitutes the unsupervised stage of the algorithm.
2) To predict future cycles based on information in
previous c yc l es. T hi s i s the super vised stage.
3) To search for an optimal solution based on repeated
simulations at the unsupervis ed and sup ervise d stages.
The paper is organised as follows. Section 1 provides
the introduction followed by methods in Section 2. Data
analyses and discussions are in Section 3 and concluding
remarks and p otential new dire c tions in Section 4 .
2. Method s
Choosing a parametric form of the density to explore
features is generally poorly understood and any tentative
choice may be too restr ictive. Often under s uch circ ums-
tances non-parametric density estimation, e.g. Kernel
Density Estimation (KDE) technique [21]allows for
practical solutions to the classical problem of choosing
the level of smoothing (bandwidth), can be efficiently
used. For example, given the data points
{ }
12 1
,,, ,
d
nn
xxx x
−
…∈
, the K DE appro ach to clust ering
defines clusters as regions of high density separated by
regions of no or low density. Its main idea is to first
compute a kernel density estimate,
( )
ˆt
fx
, say, from the
data, with a Gaussian kernel and isotropic bandwidth
0t>
controlling the amount of smoothing. In its sim-
plest form, KDE can be thought of as an alternative to
the histogram as it typically provides a smoother repre-
sentation of the data, and unlike the histogram, its ap-
pearance does not depend on a choice of starting point.
The scenario represents a problem amenable to the mul-
tivariate kernel function in Equation 1 where T is a
symmetric positive d by d bandwidth matrix defined as
the diagonal
[ ]
12 1
,, ,,
nn
Tttt t
−
= …diag
with a direct
effect on model complexity.
( )
( )
1
1
ˆn
T ti
i
f xnKxx
−
=
= −
∑
(1)
Without loss of generality, consider a phenomenon
with a binary structure of, say, “highs” and “lows”. De-
pending on the context, a number of models can be ap-
plied. For instance, if we assume a Gaussian kernel, we
can define a parametric pattern of “lows” and “highs” in
the form of a normal mixture model and use the parame-
ter estimates
{ }
Θμ,Σ=
to track the dynamics of the
cycles. Further, if we assume that the probability of a
“high” followed by another “high” structure is Phh and
that of a “low” followed by a “low” structure is P11, we
can define a Hidden Markov Model as in Tabl e 1.In this
case, an HMM provides a formal foundation for linear
sequence labelingof data. Balancing accuracy and relia-
bility amounts to defining an appropriate way of labeling
data using the probabilities and interpretingthe results
probabilistically. We could also define associations, the
corresponding scores and the underlying confidence.
2.1. Data Description, Research Problem and
Objectives
We adopt the sunspots data [11] – an average oscillating
sequence of the beginning and ending periods of solar
cycles forming the densities in F igure 1.
Table 1.A stat e transition matrix for a binary s tructure.
HIGH LOW
HIGH Phh 1 - Phh
LOW 1 - P11 P11

Using Optimized Distributional Parameters as Inputs in a Sequential Unsupervised
and Supervised Modeling of Sunspots Data
Copyright © 2013SciR es. JSEA
36
Figure 1 .Density representation of the cycles in Fig ure 1.
The densities in Figure 1 exhibit different umber of
modes – a feature typically determined by the adopted
level of s moo t hi ng. By contr olli ng t he l e vel o f smoo t hi ng
via a kernel function of the form in Equation 1 or other-
wise we are able to identify different structures in data.
Figure 2 presents a 2-D plot of the sunspots means and
standard deviations. The numbers in the plot represent
the indices for each of the last 23 cycles and the current
cycle (24th). Using a rule of thumb, we can identify the
high, moderate and low solar activity cycles, say. Fol-
lowing [10 and 1]we can treat each cycle as a separate
density and then use their distributional behavior to ex-
plore the underlying structures of the cycle s .
2.2. ModellingStrategy
Conventional approaches to modeling sunspots include
data assimilation [8] and rotational solar dynamo-based
predictive models for short-term predictions[2 and
14].The densities in Figure 1 exhibit typically bivariate
patterns and so we shall assume that the cycles form a
parametric pattern of “lows” and “highs” and define
( )
( )
11 1
|,| ,
∈ ∈∈
= ==
↔
∏∑ ∏
i ii
NK N
kk ik kSk iSk Sk
ik i
πfSμΣfSμΣ
(2)
whereSi denotes the sunspots numbers, K is the number
of components,
( )
**
f.
is a normal distribution,
k
π
is
the prior probability of class membership and i
Sk
∈
are class allocations.Statistically, the high-peaked (more
than normal) and low-peaked (less than normal) cycles
imply high and low solar activities respectively while
those skewed to the right imply few increases and fre-
quent decreases in solar activity and vice versa. Our
strategy involves two main levels – unsupervised and
supervised. At the former level, we examine the initial
and subsequent patterns of the cycles in order to separate
the “lows” from the “highs”. The maximum likelihood
estimates (MLEs) of the random finite mixture densities
are estimated and passed on to a predictive model at the
supervised level as outlined below.

Using Optimized Distributional Parameters as Inputs in a Sequential Unsupervised
and Supervised Modeling of Sunspots Data
Copyright © 2013SciR es. JSEA
37
Figure 2 . Sunspot means and standard deviations.
The above algorithm adapts the EM converging fea-
tures described in [5 and 9 ] . Its for m suits an y supervi sed
modelling technique. In this paper it is implemented in
Support Vector Machines (SVM).
2.3. Supervised Modeling of Labeled Data
We adopt Support Vector Machines (SVM) - a ker-
nel-based discriminant function the mechanics of which
rely on supervised learning of the underlying discrimi-
nating rules from the training data [5]. To put it in con-
text, let the “high” and “low” cycles in our modified set
{ }
ii
S,y :i1,,N= …
,
{ }
i
y 1,1∈−
and
2
i
S∈
be se-
parable by the hyper-plane H . T hen the poi nts lyi ng on H
satisfy the equation
wS a0+=
wherew is normal to the hyper-plane,
a
w
is the per-
pend icular d istance from H t o the or igin and
w
is the
Euclidean norm of w. The points on the hyper-planes
above and below
{ }
H1, H2H
will satisfy the equations
wS a1+=±
(both with normal w and distance to the
original
1a
w
±−
) which means that the gap
{ }
H1, H2
2
w
=
. We need to find hyper-planes maximising the
gap ( minimizing
2
w
) subject to
( )
ii
y Swa 10+−≥
. T he
numbers in
{ }
H1, H2
are the support vectors (suppor-
ters) of the optimal location of the decision surface and
the hardest to classify. Intuitively, the allocation rule is
( )
ii
ii i
ii
S wa1fory1y Swa10
S wa1fory1
+ ≥+=+
↔+ −≥∀
+ −≤=−

(3)
The SVM kernel [4] is generally defined as
( )()
V
ii
i1
FSαΦS,S a
=
= +
∑
(4)
in whic h
i
α
represents the Lagrange multiplier summed
over the values for which
i
α0.>
The upper index V
denotes the number of support vectors as described
above. SVM solution relies on the Lagrangian formula-
tion of the problem – an optimisation method requiring
VN∈
positive multipliers (
i 1,2,,V
α
= …
) for each of the
inequalities on the RHS of Equation 3. The general for-
mulation of the Lagr angia n is
( )
2VV
ii ii
i1 i1
w
LαySwa 10α
2
= =
=−+ −≥+
∑∑
(5)
SVM solution is obtained by minimising Equation 5
with respect to w and a and si mu lt ane o usly re qui ring t ha t
i
i
dL 0
dα= ∀
or equivalently maximising L and require that
bothw and a disappear. The latter implies that
iii
i
wαyS=
∑
and
ii
i
αy0=
∑
transforming Equation 5
into its dual equivalent
dii ji jij
ii
1
Lαααyy S .S .
2
= −
∑∑
The SVM model weights are calculated as the product of
the support vector coefficients and their values and used
in forming the allocation rule. Other than the support
vectors (i α0>) the remaining data points have i α0=
– these are those lying on the two hyper-planes
{} ()
ii
H1, H2ySwa10→ +−=
or beyond them if
( )
ii
ySwa10 .+−>
3. Analyses and Discussions
We now present the two-level analyses described above
in order to establish whether sunspots follow identifiable
patterns which can be used as inputs in a predictive mod-
el.
3.1. Unsupervised: Initia l Patterns and
Maximisation
Figure 3 exhibits the low and high cycles separation
based on the cut-off points above alongside their corres-
ponding overall bi-modal densities. It is based on the
maximum number of sun spots reached by the full cycles
and the number reached in the first 30 and 40 months.
The cut-off point in the LHS panel is set to the mean of
the averaged maximum early sun spots which, in this
case, is 109 - separating the low cycles 1, 5, 6, 7, 9, 10,
12, 13, 14 and 16 from the highs 2, 3, 4, 8, 11, 15, 17, 18,
19, 20, 21, 22 and 23. The densities in the RHS panel
exhibit the eme rging bi-moda lity as a function of ti me.
Figure 3 suggests that the pattern of each solar activity

Using Optimized Distributional Parameters as Inputs in a Sequential Unsupervised
and Supervised Modeling of Sunspots Data
Copyright © 2013SciR es. JSEA
38
Figure 3 .Omega cut-off (LHS) and the corresponding bi-mo dal density (RHS).
cycle is defined by its early patterns. In particular, the
maximum values reached by each cycle appear to pro-
vide an insight into the overall activity of the cycle be-
fore it starts to subside. The foregoing structural detec-
tion of patterns in the sunspots data amounts to unsuper-
vised modelling. Adopting these patterns as a guide to
data labe lling rule yields the two class p riors as
N
il
i1
z
ˆˆ 0.46 0.54
N
lh
ππ
=
==↔=
∑
with .lh θ computed as
above. As the average early patterns for cycle 24 fall
below the cut-off point, it is reasonable to suggest that it
will be a low activity cycle. Implementation of SVM
modelling follows belo w.
3.2. Supervised Level: SVM Supervised
Modelling
Results from SVM modelling based on the initial class
patterns with prior probabilities
lh
ˆˆand
ππ
gave an aver-
aged accuracy of 58% on a cost range of 0.005 to 5 and a
training sample of 500. Posterior class probabilities con-
ditioned on maximised averages of the early low and
high group means reached an average accuracy of 98%
on the same cost range and training sample size. The
support vectors are shown in Figure 4 with the horizon-
tal and vertical axes corresponding to the support vectors
and indices respectively.
Figure 5 shows the best discriminating SVM decision
values at two different bandwidths. The bandwidths and
hence decision values are chosen from multiple simula-
tions as determined by the binary cut-off point demar-
cating low from high cycles.Notice how each of the
modes also exhibits sub-modes To avoid spurious modes
(over-fitting) or masking effects (under-fitting) it is rec-
ommended to use significance test for changes or, for
clear patterns, graphical visualization.
Typically, SVM model weights for each of the support
vectors are obtained as a cross product of the model
coefficients and support vectors [15]. Weight s fro m mul-
tiple SVM runs can be recorded and their graphical pat-
terns be used to guide model selection. Other SVM out-
puts include the individual probabilities and decision
values as in Figure 6. The difference between the lower
accuracy case in the top panel - highlighting the random
nature of class allocation – and the higher accuracy mod-
el in the bottom panel) showing clear concentrations of
lh
ˆˆand
ππ
on either side of the class boundary.The obser-
vations corresponding to the vectors in Figure 4, the
decision values in Figure 5 and to the corresponding
prob abilities in Figure 6 can be identified by indexing.
4. Concluding Remarksand Potential Fut ure
Directions
Predicting solar activity cycles remains one of the major
challenges the scientific community faces with intricacy
being compared to predicting, say, the severity of next
year’s winter. In this weather analogue, if all that is
available is a long vector of temperature readings over
many years, the only sensible approach is to search for

Using Optimized Distributional Parameters as Inputs in a Sequential Unsupervised
and Supervised Modeling of Sunspots Data
Copyright © 2013SciR es. JSEA
39
Figure 4 . Support vectors for the initial patterns (LHS) and maximised parameters (RHS).
Figure 5 . S VM decision values at two different bandwidths sho wing wel l-separated structures.
Figure6 . Sun cycles class probabilities.

Using Optimized Distributional Parameters as Inputs in a Sequential Unsupervised
and Supervised Modeling of Sunspots Data
Copyright © 2013SciR es. JSEA
40
naturally arising structures in the data with the hope that
if uncovered they may provide potentially useful infor-
mation. This paper adopted the foregoing philosophy and
sought to develop a predictive framework for modelling
sunspots data using inherent distributional properties in
the data. The paper relied on a continuous flow of data
for prediction, but rather than assessing model accuracy
on the NOAA benchmark, an SVM model was trained
and tested on a notionally infinite dataset of cycl es.
By examining multiple sets of observations from the
onset of each cycle via graphical visualisation early pat-
terns of sun cycles and their binary nature were deter-
mined. Comparing multiple early patterns for each rec-
orded cycle extracted at different time periods to the cor-
responding full c ycles revealed that the first 3 years pro-
vide a sufficient basis for predicting the cycle’s peak.
The patterns were then adapted as inputs into an inte-
grate d unsup ervi sed and supervi sed mod elli ng algo rithm.
The novel method’s mechanics are geared towards si-
multaneously tracing anomalies via an adaptive approach.
Repeated SVM runs using repeatedly improved parame-
ters showed that the approach yields greater accuracy and
reliability than conventional approaches. Multiple simu-
lations of this type can be generated based on the algo-
rithm above to assist in selecting the most consistent
model. The paper’s main substance can be described as
an enhancement of algorithmic methods for learning un-
derlying rule s from data.
Finally, it is worth noting that while the study was
confined to the conventional periodicity of 11.11 years
[22] with a binary pattern of cycles, the definition im-
plies that the periodicities can differ according to defini-
tions. Further, while we assumed a binary scenario of the
cycles in Figure 1, different bandwidths are likely to
yield different patterns. To address this limitation, the
paper’s findings highlight potential investigations paths
into such variations. Further, the current study, based on
a single application and a single method, could not con-
firm the algorithm’s robustness. Although we adopted
SVM for implementation, the approach is amenable to
any domain-partitioning method. Thus, for model en-
hancement purposes, it will be useful to provide a com-
parative study using other learning algorithms such as
neural networks and decision trees.
REFERENCES
[1] Bugrie n, J. , Mwit ondi, K. and Shuweihdi, F. (2013).
A Kernel Density Smoothing Method for Deter-
mining an Optimal Number of Clusters in Conti-
nuous Data; The 16th International Conference on
Computational Methods and Experimental Mea-
surement s; 2 – 4 July, 2013, A Coruña, Spain.
[2] Choudhuri , A. R., Chatterjee, P. and Jiang, J. (2007).
Predicting Solar Cycle 24 with a Solar Dynamo
Model; Physical Review Letters, Vol. 98, Issue 13,
American Phys. Society.
[3] Cuevas, A., Febrero, M. and Fraiman, R. (2000).
Estimating the number of clusters; The Canadian
Journal of Statistics, Vol. 28, No. 2, pp 367-382.
[4] Cort es a n d V apn i k , (1995). Su ppor t-vector networks;
Machine Learning, Vol. 20, No. 3, pp. 273-297,
Kluwer Academic Publishers.
[5] Dempster, A. P., Laird, N. M., and Rubin, D. B.
(1977). Ma x imum Lik el ih ood from In complete Data
via the EM Algorithm; Journal of the Royal Statis-
tical Society, Vol. 39, pages 1-38.
[6] Hand, D., Mannila, H. and Smyth, P. (2001). Prin-
ciples of Data Mining (Adaptive Computation and
Machine Learning); A Bradford Book; ISBN-13:
978-0262082907.
[7] Kane, R. P. (2007). Solar Cy cle Predictions Based on
Extrapolation of Spectral Components: An Update;
A Journal for Solar and Solar-Stellar Research and
the Study of Solar Terrestrial Physics; Vol. 246, Is-
sue 2, pp 487-493, ISSN: 0038-0938.
[8] Kitiashvili, I. and Kosovichev, A. (2009). Prediction
of solar magnetic cycles by a data assimilation me-
thod; Cosmic Magnetic Fields: From Planets, to
Stars and Galaxies; Proceedings IAU Symposium,
No. 259, Edited by Strassmeier, K, Kosovichev, A.
and Beckman, J. ( 2009) - International Astronomical
Union.
[9] McLachlan, G. Krishnan, T. (1996). The EM Algo-
rithm and Extensions; John W iley.
[10] Mwitondi, K., Said, R. and Yousif, A.: A sequential
data mining method for modelling solar magnetic
cycles; Neural Information Processing, LNCS, Vol.
7663, pp 296-304, Springer (2012).
[11] NOOA (2012). http://www.ngdc.noaa.gov
[12] Pohtila, E. (1980). Climatic fluctuations and forestry
in Lapland; Ecography, Vol. 3, Issue 2, pp 65–136,
ISSN: 1600-0587.
[13] Pielke, R., Avissar, R., Raupach, M., Dolman, A.,
Zeng, X. and Denning, A. (1998). Interactions be-
tween the atmosphere and terrestrial ecosystems:
Influence on weather and climate; Global Change
Biology, Vol 4, Issue 5, pp 461–475.
[14] Qahwaji, R. and Colak, T. (2007). Automatic
Shor t-Term Solar Flare Prediction Using Machine
Learning and Sunspot Associations; SOLAR
PHYSICS, Vol. 241, No. 1, pp 195-211, ISBN
11207-006-0272-5.
[15] R (2011). R Version 2.13.0 for Windows; R Foun-
dation for Statistical Computing.
[16] Reames, D. (2002). Magnetic topology of impulsive
and gradual solar energetic particle events; The As-
trophysical Journal, Vol. 571, pp 63–66.

Using Optimized Distributional Parameters as Inputs in a Sequential Unsupervised
and Supervised Modeling of Sunspots Data
Copyright © 2013SciR es. JSEA
41
[17] Roberts, S. J.: Parametric and Non-parametric Un-
supervised Cluster An alysis, Pattern R ecognition, 30,
5, pp 261-272 (1997).
[18] Rycroft, M. J, Israelsson, S. and Price, C. (2000).
The global atmospheric electric circui t, solar activity
and climate change; Journal of Atmospheric and
Solar-Terrestrial Physics, Vol. 62, Issues 17–18, pp
1563–1576.
[19] Schwabe, S.H. (1843). AstronomischeNachrichten,
20, No. 495, 234-235
[20] Siscoe, G. L. (1978). Solar–terrestrial influences on
weather and climate; Climatology Supplement, Na-
ture, Vol. 276, pp 348-352.
[21] Silverman, B. W. (1981). Using Kernel Density
Estimates to Investigate Multimodality, Journal of
the Royal Stati stica l Society, B , 43, pp 97-99.
[22] Wolf, J. R. (1852). New studies of the period of
Sunspots and their meanings; Communications of
Natural History; Society in Bern, 255, pp 249-270.