Simplex Optimization and Its Applicability for Solving Analytical Problems

Formulation of the simplex matrix referred to n-D space, is presented in terms of the scalar product of vectors, known from elementary algebra. The principles of a simplex optimization procedure are presented on a simple example, with use of a target function taken as a criterion of optimization, where accuracy and precision are treated equally in searching optimal conditions of a gravimetric analysis.


Introduction
Simplex is a geometric figure, formed on the basis of 1 n + points ( ) in the n-dimensional space, i.e., a number of the points exceeds the dimension of the space by one.These points are referred to as the vertices of the simplex.Distribution of the points i A in this space is represented by a matrix of this simplex.Speci- fying the coordinates for i A is easy if it concerns the 2-D or 3-D space, but it becomes less comprehensi- ble when the issue concerns the n-dimensional ( ) -n D simplex.Although in the literature [1] one can find ready-made forms of the simplex matrix in - n D space, but information on how to receive it in an easy manner is lacking.
The simplex concept is the basis for the Nelder-Mead [2] simplex optimization procedure (SOP) [3], conceived as a modification of the simplex method of Spendley, Hext and Himsworth [4].The SOP is easy to use, and does not require calculation of derivatives [5] [6], as in the gradient (local steepest descent) method [7] [8]).The SOP is widely applicable and popular in different fields of chemistry, chemical engineering, and medicine [9].Furthermore, it works well in practice on a wide variety of problems, where real-valued minimization functions with scalar variables are applied.
The sequential simplex method is considered as one of the most effective and robust methods applied in methodology of Evolutionary Operation (EVOP) experimental design [10]- [15].The SOP method was implemented in MINUIT (a Fortran from CERN [16] package), then in Pascal, Modula-2, Visual Basic, and C + + [17].The iterations are stopped when the function minimized is less than a preset value.
In this article, the problem of obtaining the simplex matrix in - n D space will be presented on the basis of the triangle and the scalar product concept, well-known to the students from earlier stages of education.The evolution of the simplex according to SOP [3] [18] will also be presented in a understandable manner.

Simplexes in 2-D and 3-D Space
In order to introduce the simplex concept, we start from the point ( ) 0-D , (I) segment ( ) ) notations express the dimensionality of the corresponding geometrical entities (Figure 1).
The points: 0 A , 1 A , 2 A are vertices of the triangle 0 1 2 A A A ; the points: 0 A , 1 A , 2 A , 3 A are the vertices of the tetrahedron 0 1 2 3 A A A A .0 1 A A is one of the edges of the triangle; 0 1 A A is one of the edges of the tetrahe- dron.Equilateral triangle 0 1 2 A A A is one of walls of the tetrahedron.Segment, equilateral triangle, and tetrahedron can be presented in the same scale; it means that all distances, equal to unity (chosen arbitrarily), between the (neighbouring) points i A and j A of a polygon are assumed, i.e. the length 1 All angles between edges of the equilateral triangle are equal to 60˚ ( ) π 3rd .All angles between adjacent tetrahedron edges are equal 60˚ too, owing to the fact that all the tetrahedron walls are composed of equilateral triangles ( ) 2-D .By turns, tetrahedrons form the 3-D walls in 4-D polygon, etc.It means that all angles between adjacent edges of a symmetrical - n D polygon are equal 60˚, as well.The equilateral triangle can assume different positions on the plane.A particular case is the triangle with unit edges, "anchored" at the origin of co-ordinate axis, as one presented in Figure 2.
The successive vertices of the equilateral triangle can be presented in matrix form, where the rows are expressed by co-ordinates of the corresponding points ( ) The second and third rows of this matrix are identical with components of transposed vectors: 0 1 A A and 0 2 A A , respectively: The tetrahedron can assume different positions in 3-D space.A particular case is the tetrahedron anchored at the origin of co-ordinate axis as one presented in Figure 3.
The successive vertices of the tetrahedron can be presented in the matrix form: The second, third and fourth rows of this matrix are identical with components of the transposed vectors:

Simplexes in n-D Space
The segment, equilateral triangle and tetrahedron are imaginable concepts.However, application of the concepts involved with geometric figures of higher dimension ( ) is beyond imagination.One should be taken into account that, in the simplex optimization, we are forced, as a rule, to consider higher number of factors influencing the optimizing process.
Although the figures in - n D space (Cartesian co-ordinate system) are beyond imagination, one can present them in projective space.In Figure 4, any pair of points of the simplex is connected, informally, by a segment (line); the segments (sides, edges) are marked there with broken lines.Particularly, two points form the 1-D simplex in 1-D space.Adding a third point (that not lies along the segment or its extension) gives a triangle ( 2-D simplex) in 2-D space.Adding a fourth point in 3-D space (not coplanar with 2-D simplex plane), one obtains a tetrahedron ( 3-D simplex), etc.Generally, adding a successive +1 k -th ( ) simplex, one obtains the - k D simplex.In other words, adding the successive point that not lies on (or in plane of) simplexes of lower dimension and connecting it, every time, with the remaining points, one can imagine the simplexes with larger and larger dimensions, in projective space.Let us assume that the polygon is "anchored" in n-D-space in such a manner that 0 A is in the origin of the co-ordinate axis and any successive point "enters" the new dimension of the n-D-space, see Figure 4.It means that the co-ordinates of the corresponding points of - n D polygon are as follows: 0,0,0, ,0,0 ; ,0,0, ,0,0 ; , ,0, ,0,0 ; , , , ,0,0 ; ; , , , , ,0 ; , , , , , .
This means that 0 ij x = for j i > .As stated above, the number of vertices exceeds, by 1, the dimension of the space; it means that the polygon in n D space involves 1 n + vertices.
We consider a pencil of n unit vectors, , , , , , , , all starting from the point , i.e., origin of co-ordinates of - n D space and ending at the points ( ) should notice that the basic components ij x of the vector i x starting at the origin of co-ordinates, are identical with co-ordinates of the point ( ) , , , , , , . Then for the vectors 0 i A A based on the points specified in (3) we have: ,0,0, ,0 ; , ,0, ,0 ; , , ,0, , , , , , , , .
Let us construct the matrix X (Equation ( 5)) composed of the co-ordinates of the corresponding points in Moreover, we assume non-negative values for the coordinates, i.e. 0 ij x ≥ , and then the elements of the ma- can be specified as follows: ( ) ( ) Let i e be the unit vector along the i-th axis, X i , i.e.
. The j e vectors form a set of mutually orthogo- nal vectors in n-D space.The scalar products: ( ) e e e e (7) where ij δ is the Kronecker symbol.
Any vector i x in - n D space can be also written as x , x , , x . Then the scalar product of two vectors, i x and j x , in - n D space is as follows: ( ) where , i j ∠x x is the angle between vectors i x and j x spanning the simplex in - n D space.The angle between different vectors , for any pair of the vectors i x and j x : in triangle ( ) and then ( ) . If the unit vectors , then from ( 8) and ( 4)-( 6) we have: ( ) 1 where min , 2 1, where 1, , From ( 9), ( 10) and ( 6) we get, by turns, ( ) and so forth.On this basis, one can write: ( ) ( ) ( ) The elements ij x of the matrix 5)) are thus derived with use of principles of elementary algebra and geometry.Then the - n D simplex can be written in the form One can notice that h j (Equation ( 15)) is the height of -j D symmetrical simplex and j r (Equation ( 16)) is the radius of -j D sphere (for 2 j = -circle, for 3 j = -"normal" sphere) inscribed in -j D simplex.Moreover, we have the relations: and where R j is the radius of -j D sphere circumscribed on -j D simplex, see Figure 5.In particular, for 7 n = , the matrix X (Equation ( 13)) has 7 columns and 8 rows.

Translation and Reflection of the Simplex
A simplex can be translated and/or rotated in space.Particularly, the simplex defined by X (Equation ( 13)), with the centre at ( ) , can be parallely shifted to the centre ( ) 0 0, , 0 c P =  of the co-ordinate system.This way, one obtains a new matrix tr X R in 2-D space (Equations ( 11) and ( 12)) for 2 n = .
with j R and ( ) as ones in Equations ( 11), ( 12) and ( 14).Reflection of particular vertices of the matrix tr X in the centre of co-ordinate system ( ) 0, , 0  gives the simplex described by the matrix ( ) 1 2 2 0.5 0.289 0.5 0.289 Note that the number of - k D walls in - n D simplex equals to the number of 1 k + point subsets ( - k D walls) generated from the 1 n + -point set.From the elementary theory of combinations it results that the number of k-D walls equals to the number of possible combinations of 1 n + items taken 1 k + at a time, i.e.

Simplex Optimization Procedure (SOP)
The simplex method, based on the simplex concept, is considered among the most efficient optimization procedures, successfully applied in different areas of optimization techniques and chemical analyses.The simplex concept is applicable, among others, for optimisation of analytical methods [13], e.g. for searching the conditions securing optimal accuracy and precision of an analytical method.

An Objective Function
The simplex concept is applicable, among others, for optimisation of analytical methods [13].The variables, i x (scalars), named as factors forming the vector x , affect the variable y , named as objective ("target") function, i.e., ( ) ( ) The essence of optimisation is inherent in the objective function.Accuracy and precision are among the most important criteria of analytical methods.Another object functions are related to sensitivity of a method, efficiency of a chemical reaction, etc.
Let us assume that an analytical method aims to find the conditions securing true contents ( ) 0 , m g g of an analyte in a sample to be obtained.For this purpose, the following criterion (objective function) has been suggested [19], where: were calculated in the i -th simplex point, i A (Equation ( 3)), on the basis of i N measurements ik m of an intensive variable m (e.g.number of grams of an analyte contained in a unit mass of solution); 0 m is the true value of this variable, obtained according to a reference method.In the object function (20), both terms, i.e. accuracy ( ) 0 -m m and precision, expressed by standard deviation, , are considered nearly equivalently; the difference in "weights": 1 and , of the two characteristics of the method becomes less significant at higher i N values.The function (20) is an example of superposition of different characteristics of a method, i.e. accuracy and precision.The optimisation is terminated when the systematic error is insignificant, i.e., the inequality ( )

Decision Variables and Arrangement of Experimental Conditions
The optimization procedure assumes: 1) Selection of natural decision variables, i Z , as independent (in principle) individual factors affecting the values of the objective function (Equation ( 19)) and 2) Their order, ( ) The number ( ) n of the variables j Z chosen defines the simplex dimension.
The kind ( ) and the number ( ) n of these variables that assume continuous (not discrete) values ( ) , consisting the vector , affects the success (efficiency) of optimisation process aiming to get optimal (maximal or minimal) value for the object function, e.g.minimal value for ( ) y y i = defined by Equation (20).It should be noted that the - n D simplex matrix (Equation ( 5)) refers-in principle-to orthogonal (independent) variables.The independent variables assuming continuous values are, among others: pH, temperature, time, rate of an operation.
One should be noticed that the form of the matrix ( 5) enables to add a new variable, assumed at the start for optimization-provided that the decision of inclusion of variable 1 n Z + into the set of variables has been undertaken after measurements done in the points 0 , , p A A  , at p n ≤ , where 1 n Z + remained unchanged.In this case, the measurements done at the points 0 , , p A A  has not to be repeated.

Design of Experiments and Optimization within the Starting Simplex
The introductory step of the optimization procedure assumes fixing the starting values, 0 j z , and steps, ; these values affect strongly the efficiency of the optimization procedure.Some constraints put on j z and j z ∆ values (resulting from physicochemical, chemical or technological reasons) can also be taken into account.These constraints may result e.g., from limited solubility, possibility of phase change affected by temperature, etc.
The optimization is realized there for values of natural variables calculated from the formula where 0,1, , i n =  enumerates successive experimental points realized within the initial simplex.The advised values for ij x in (5) are defined in the matrix (13) and specified e.g. in the matrix (15) if 7 n = .The values ( ), 0, , of the initial simplex are the basis for its further evolution of the simplex.

Optimization at the Points of Evolving Simplex
At the first stage of the simplex evolution (Figure 7), a point A in the centre, ( ) A x , of the remaining n points of the initial simplex, wall not containing the point A .In other words, c1 x is the middle point between w1 x and n 1 + x , i.e., After adding the term ( ) to both sides of Equation ( 26), we obtain the co-ordinates of the new point, After measurement(s) done at the point A with the worst value, ( ) A + is obtained and further operations are repeated again, in cyclic manner.At q -th reflection one can write ( ) In the example presented above, the initial (symmetrical) shape of the evolving simplex is maintained owing to the fact that 1) mirror reflections are applied and 2) equal statistical weights are assumed to all the simplex points considered.Sometimes we are forced to move the simplex within a constrained component space [20].
The evolving simplex approaches the quasi-optimal region, where ( ) y i values do not change distinctly.In this region, the evolving simplex should be contracted or diminished (Figure 8); such operations enter e.g. the MINUIT program.The contraction/dilatation and weighting are involved in the relation [21] [22] ( ) where the parameter α results from re-parametrization of the a line passing through 2 points: one of them is wq x , the second one is the centre cq x of area of q -th, n 1-D − simplex, defined as follows ( ) ( ) where the operator ( ) q i wq ≠ ∑ means summation within q -th n 1-D − simplex ( wq x excluded).The contraction or expansion and weighting ("weights" i p ) different points i x deforms the evolving simplex gradually.As a rule, it is not an advantageous occurrence during the optimization procedure, however.The simplex deformation is influenced mainly by the weighting factors [23].Moreover, a shape of response function may sometimes cause the optimum searching impossible, even in 2-D space.

Example of SOP
To illustrate the principle of SOP, a simple example of gravimetric analysis is considered below.The matrix expressed by Equation ( 15) was chosen for initial simplex, and Equation ( 20) was applied as criterion of optimization.4 i N = measurements were made at each simplex point.In our case, the relation ( 24) is chosen as the cri- terion of optimization.Analyses were made with CdSO 4 solution of ( ) , standardised according to electrogravimetry, considered as the reference method.

Analytical Prescription
The samples, ca. 5 mL of the solution, weighed on analytical balance (±0.1 mg), were taken for analysis.The samples were treated with tartaric acid (0.220 g/mL) solution and then with 2.51 mol/L NaOH solution.After dilution with water, the solution was treated with 8-hydroxyquinoline (HL) solution (3.00% m/v, in ethanol) added dropwise from burette, with defined rate of addition.The mixture was leaved for defined time on the water-bath for re-crystallisation of the precipitate CdL 2 at defined temperature, then filtered, washed with water, dried to a constant mass at 105˚C and weighed.   1 we have 34 8.00 0.817 1.00 8.82 z = + × ≅ .

Results of Analyses within the Initial Simplex
The simplex optimisation procedure was realized in experiments performed in all points of the initial simplex (rows 0, , 7  in Table 1).In each point ( ) i N = measurements were done.The results m ik [gCD/g] are presented in Table 2.  1).As results from detailed calculations done on the basis of results in Table 2, the inequality (23) has not been fulfilled within the initial simplex and then the evolution of the simplex was needed.The coordinates of the new points: 8 A , 9 A and 10 A for evolving simplexes, calculated with use of the coded variables ij x specified.

Optimal Prescription
On the basis of the optimization procedure, one can formulate the following prescription.The samples, ca. 5 ml of the standardised CdSO 4 solution, weighed on analytical balance (±0.1 mg), were taken for analysis.The solution was treated with 2.80 g of tartaric acid and then with 7.9 ml of 2.5 mol/l NaOH solution.After dilution with water up to ca 90 ml, the solution was treated with 7.7 ml of 3% (m/v) 8-hydroxyquinoline solution (added dropwise from burette, with the rate 1.4 ml/min.The mixture was leaved for ca.90 min on the water-bath for re-crystallisation of the precipitate CdL 2 at temperature 63˚C, then filtered, washed with water, dried to a constant mass at 105˚C and weighed.

Final Comments
In this paper, the simplex method has been presented in terms known from elementary algebra and geometry.On the basis of scalar product and length of vectors in normalized scale, the co-ordinates of simplex vertices in n D space were determined.A criterion of optimization, where accuracy and precision of measurements were considered equivalently, has been suggested.
The simplex method was considered as the most efficient among optimization procedures, applicable in simulating procedures realized in recursive computer programs used for simulation purposes and in optimization of analytical procedures.The simplex method requires only one additional experiment, regardless of the number of factors being varied.This drastically reduces the number of experiments required to reach the optimum.

Figure 4 .
Figure 4. Formation of consecutive dimensions (up to 5-D ) of a simplex in projective space.

Figure 5 .
Figure 5. Graphical presentation of 2 r , 2 h and 2 the simplex presented in Figure6(in 2-D space), obtained from (I) by translation (II) and mirror reflection (III) in the origin of ordinate axis, are presented by matrices: tr X and: *

Figure 6 .
Figure 6.The simplexes obtained after successive (II) translation and (III) reflection of the original simplex (I) in the center of co-ordinate axes in 2-D space; for details-see text.

1 wA 1 wA
with the worst ( ) value, ( ) 1 y w , for the object function (Equation (20)) is indicated, is replaced by a new point, 1 n A + , obtained by reflection of 1 w

1 nA
+ , the values ( ) y i are compared again within the new (evolved) simplex (containing 1 n A + and not containing 1 w A ) and the point 2 w

2 wA
is reflected in the centre of n 1-D − wall of the new simplex (not containing 2 w ).This way, a new point 2 n

Z 1 - 4 j
re-crystallisation time [min] of the precipitated 2 CdL ; Z 2 -mass [g] of tartaric acid added in the solution; Z 3 -volume [ml] of NaOH added; Z 4 -volume [ml] of 8-hydroxyquinoline added; Z 5 -rate [ml/min] of 8-hydroxyquinoline addition; Z 6 -temperature [˚C] of water bath where re-crystallization of CdL occurs; Z 7 -volume [ml] of the solution before precipitation.For example, the value 04 7= , from Table

Table 2 ,
the ( ) y i values were calculated (last column in Table

Table 1 ,
we have, for example,

Table 2 .
Results of measurements done within the initial simplex.