Optimal Adjustment Algorithm for p Coordinates and the Starting Point in Interior Point Methods

Optimal adjustment algorithm for p coordinates is a generalization of the optimal pair adjustment algorithm for linear programming, which in turn is based on von Neumann’s algorithm. Its main advantages are simplicity and quick progress in the early iterations. In this work, to accelerate the convergence of the interior point method, few iterations of this generalized algorithm are applied to the Mehrotra’s heuristic, which determines the starting point for the interior point method in the PCx software. Computational experiments in a set of linear programming problems have shown that this approach reduces the total number of iterations and the running time for many of them, including large-scale ones.


Introduction
In 1948, von Neumann proposed to Dantzig an algorithm for finding a feasible solution to a linear program with a convexity constraint recast in the form ([1,2]): Px = 0, e t x = 1, (1.1) 0 x  where , , is the vector of all ones, and the columns of P have norm one, i.e., R n e  1 j P  , for .1, , j n   This algorithm, which was later studied by Epelman and Freund [3], has interesting properties such as simplicity and fast initial advance.However, it is not very practical for solving linear problems because its convergence rate is slow.
Gonçalves [4] presented four new algorithms based on von Neumann's algorithm and among them the optimal pair adjustment algorithm has the best performance in practice.
The optimal pair adjustment algorithm inherits the good properties of von Neumann's algorithm.Although it is proved in [5] that this algorithm is faster than von Neumann's algorithm, nevertheless, it is impractical to solve linear programming problems, because its convergence rate is also very slow.
In [6], the idea presented by Gonçalves, Storer and Gondzio [5] to develop the optimal pair adjustment algorithm was generalized for p coordinates and then the optimal adjustment algorithm for p coordinates arose.The value of p is bounded by the number of variables of the problem.The new algorithm is not suitable to solve linear programming problems until achieving optimality.Thus, the idea is to exploit its simplicity and quick initial progress during the early iterations and to use it together with an interior point method to accelerate convergence.
Knowing that the starting point greatly influences the performance of the interior point method, in this work the optimal adjustment algorithm for p coordinates is applied within the Mehrotra's heuristic [7], which determines the starting point for the method interior points in the PCx software, so that the starting points can be obtained even better.This approach is different to the classical warm-starting approach, which uses a known solution for some problem (e.g., relaxation in the columngeneration) to define a new starting point for a closely related problem or perturbed problem instance ( [8][9][10]).
The paper is organized as follows.In Section 2, the definition of the problem is given and von Neumann's algorithm is described.In Section 3, the optimal pair adjustment algorithm is recalled.In Section 4, the optimal adjustment algorithm for p coordinates is proposed b k and some theoretical results are described.Section 5 discusses warm-starting in interior point methods.Numerical experiments are shown in Section 6.In Section 7, the conclusions are drawn and future perspectives are suggested.

Von Neumann's Algorithm
Considering the problem of finding a feasible solution of the set of linear constraints (1.1), geometrically, the columns P j can be viewed as points lying on the m-dimensional hypersphere with unit radius and center at the origin.This problem can be described as assigning nonnegative weights x j to columns P j in such a way that after being re-scaled its center of gravity is the origin.
First, the algorithm finds the column P s of P which makes the largest angle with the residue b k-1 = Px k-1 and then the next residual b k is given by projecting the origin on the line segment connecting b k-1 to P s .See Figure 1.
4. Update: where e s is the vector of the canonical basis wit in position s.
Stopping Criterion: here ε is a predetermined percentage.e initial approxim while (E k > w Note that b k = Px k , for all 0 k  .Th ation x 0 is arbitrary, since e 1, then x j = 1/n for 1, , j n t x 0 =   was used.For any given iteration k column lumn which makes the largest angle with the vector b k−1 .Furthermore, if reover, the new residual is smaller than the previous one, i.e., u k < u k−1 , as can easily be seen in Figure 1 minated by matrix-vector multiplication needed when selecting column P s which is O(nz(P)) where nz(P) is the number of the entries of P.This effort can be reduced significantly as the matrix P is sparse.For more details of this algorithm see [11,12].

3
In algorithm and introduced four new algorithms based on it.Among them, emphasis is given to the weight-reduction algorithm and the optimal pair adjustment algorithm.The optimal pair adjustment algorithm was the one that performed better in practice.
The weight-reduction algor tempt to improve the efficiency of von Neumann's algorithm.It is based on the idea that the residual b k−1 can be moved closer to origin 0, increasing the weight x j for a given column P j and decreasing the weight x i for another column P i .
In particular, it is e oser to origin 0 than the residual b k−1 if the weight in column P s+ is increased when P s+ has the largest angle with the residual b k−1 and the weight in column P s− is decreased when P s+ has the smallest angle with the residual b k−1 .This corresp rection P s+ − P s− .The new residual b k is the point that minimizes the distance from origin 0 to this line.
Notice that the minimization of this distance is s the maximum possible decrease of x s− .Since 0 j x  for all j, then x s− can be decreased until it vanishe ure 2 is a geometric illustration of how the algorithm works by iteration.s.Fig-

Optimal Pair Adjustment Algorithm
he optimal pair adjustment algorithm is a generalization algorithm, the optimal pa T of the weight-reduction algorithm designed to give the maximum possible freedom to two of the weights x j (see [5]).In a way, we can say that it prioritizes only two variables by iteration because it finds the optimal value for two coordinates and adjusts the remaining coordinates according to these values.
Similar to the weight-reduction ir adjustment algorithm starts by identifying vectors P s+ and P s− , which make the largest and smallest angles with the vector b k−1 .Afterwards, values k s x  , k s x  ,  are found, where  for all j s   j s   .These va ize the distance between origin while satisfying the convexity and the non-negativity constraints.The solution of this optimization problem is easily computed examining the Karush-Kuhn-Tucher conditions (KKT).
The main difference between th rithm and the optimal pair adjustment algorithm is that only the weights of P s+ and P s− are changed in the first algorithm while in the second algorithm all other weights are also changed.
If , then STOP.The problem (1.1) is infea-3.he problem: where   4. Update: In iteration k, column P s+ is the column which makes th .2. Subproblem Solution order to solve subproblem (3.2), first the subproblem while (E k > e largest angle with the vector b k−1 and column P s− is the column which makes the smallest angle with the vector b k−1 such that x s > 0.
where µ i are Lagrange multipliers.Problem (3.3) is solved by selecting a feasible solution re replaced in th ear system is so inates s e g the ideas used in [5] for the op- among all possibilities that meet the KKT conditions.This is done by analyzing the following cases: a)

the known values a
For each case ab e KKT equations and the resulting lin lved.

Optimal Adjustment Algorithm for p Coord
The optimal adjustment algorithm for p coordinates i veloped generalizin d timal pair adjustment algorithm.Instead of only two columns to be used to formulate the problem, any amount of columns can be used and thus, importance will be given to any desired amount of variables.
The strategy to prioritize the variables is free.In this work, we chose to take p/2 columns making the largest angle with the vector b k and the remaining p/2 columns that make the smallest angle with the vector b k .If p is odd then one more column is added to the set of vectors that make the largest angle with the vector b k , for example.
The structure of the optimal adjustment algorithm for p coordinates is similar to the optimal pair adjustment algorithm.It begins by identifying the s 1 columns that make the largest angle with the vector b k−1 , then s 2 columns that make the smallest angle with the vector b k−1 are determined, where s 1 + s 2 = p and p is the number of columns that are prioritized.Next, the optimization subproblem is solved and the residual and the current point are updated.

OPTIMAL ADJUSTMENT ALGORITHM FOR p
, which make the largest angle with .the vetor b k   , which make the smallest angle with the vector b k−1 and , then STOP.The problem (1.1) is infea-3.Solve the problem: 0, for 1, , , 0, for 1, , .
n the same side of the hyper-plane rough the origin and perpendicular to direction b k−1 .This m resulting in the origin can be found.In this case, the In Step 2 o of matrix P are o th eans that any convex combination of the columns P j or each iteration of the optimal adjustment algorithm m (4. 4 lem is solved finding a solution in the ositive orthant, subject to a linear equation system of problem is infeasible.

F
for p coordinates, it is necessary to solve subproble ).This subprob p order (p + 1).A way to solve it is to verify all possible feasible solutions as in the pair adjustment algorithm.The drawback that comes naturally in solving the subproblem in this way is that the number of possible cases to be verified grows exponentially with the value of p as shown next.
In fact, to solve the subproblem (4.4) first the variable λ 0 , as defined in (4.5), is removed from the problem.
Thus, the subproblem (4.4) was rewritten as: where , , and denoting the objective function by , the KKT conditions o e subproblem are given by: .
Then, the subproblem (4.6) is solved by selecting a feasible solution among all the possibilities that satisfy the KKT conditions (4.7).For this, all possible cases of possible values for the variables , , , , , , and others equal to zero, which gives a combination of 2 p C cases, but again we have to consider the case where 1 The total number of cases when p coordinates are modified is: erefore, this stra Th tegy is inefficient even for values of p which are not too large.
In order to deal with such a difficulty, the subproblem (4.4) is approached differently and solved using interior point methods.This is done as follows: First, the subproblem is rewritten in matrix form: Copyright © 2011 SciRes.AJOR The KKT equations of problem (4.8) are given (4.10) where, γ and µ are Lagrange multipliers of equality and in 1) is Now the interior point method is applied to those equations.by: 0, 0, 1 0, 0, 0.
equality constraints respectively and (p + 1) × (p + the dimension of the matrix W t W. The linear system arising at each iteration of the interior point method applied to (4.10) are: where (4.11) .

Theoretical Properties of the Optimal
bed below and proved in [6], nsure that the optimal adjustment algorithm for p coore converg at, om the theoretical point of view, for larger values of p Note that (W t W + Λ -1 U) is a positive definite matrix of order p + 1 and both systems can be solved using the same factorization.

Adjustment Algorithm for p Coordinates
Theorems 1 and 2, descri e dinates converge in the worst case with the sam ence rate of von Neumann's algorithm and th fr the algorithm is more robust and will perform better.
Theorem 1 The decrease in k b obtained by an iteration of the optimal adjustment algorithm for p coordinates, with 1 p n   , where n is the column number of matrix P, in the worst case, is equal to the decrease obtained by an iteration of von Neumann's algorithm.
Theorem 2 The decrease in k b obtained by an iteration of the optimal adjustment algorithm for p 2 coordinates, in the se is equal to that obtained by an iteration of the optimal adjustment algorithm for p 1 coordinates with 1 2 p p n worst ca   , where n is the P column nu p. Howeve that the starting point can influence the erformance of interior point methods.On the other hand, s a s on, a natural idea is use this algorithm to find a good starting point for in-mber.On the other hand, it is not advisable to choose a very large value of p since there is a cost of building and updating matrix W in (4.13).Such a cost is negligible for small values of r, it becomes noticeable for larger values.

Mehrotra's Heuristic and Starting Point in Interior Point Methods
It is well known p as the optimal adjustment algorithm for p coordinates ha mall computational cost per iterati to terior point methods or improve the current one.In this work, few iterations of the optimal adjustment algorithm for p coordinates are performed to improve the starting point introduced by Mehrotra's heuristic [7].This heuristic consists of the following steps: 1) Use least squares to compute the points: where x, y and z are the primal variables, dual variables and dual slacks respectively.
2) Find values δx and δz such that 3) Determine x   and z   suc 0 h that the points x and z 0 these are improved.However, the centrality, which is important for the interior point methods, may be lost.
are centralized: We have adapted the optimal adjustment algorithm for p coordinates in the PCx code [13] aiming to improve the starting point given by Mehrotra [7].
The starting point for the optimal adjustment algorithm for p coordinates is determined by solving the least squares in Step 1.

Columns Problem Rows Columns
In the following experiments, all testing was performed on an Intel Core 2 Duo T7250, 2 GB RAM, 2 GHz and 250 GB hd and Ubuntu 32-bit operating system.We used 76 test problems to compare the performance of the original PCx and the modified PCx (PCxMod), which uses the optimal adjustment algorithm for p coordinates as a warm-starting approach.Most tested problems have free access such as NETLIB problems, QAPLIB problems and KENNINGTON problems ( [14,15]).Other problems, which are not available publicly, were obtained from Gonçalves [4].The test problems are presented in Table 1.
n see [4].pr or ter duce gorithm described above, unlike our proposal, which consists of performing a few iterations of the optimal adjustment alg g, i.e., before Step 2. One explanation for this is that if the algorithm is then used after to centralize the points,

St g
The e pe or is n impo param the performance of the PC better results when the solu accuracy.However, in some cases, the number of iterations needed for convergence of the algorithm can be very large, making it impractical to use.In these cases, a maximum number of iterations should be adopted.Therefore, the stopping criterion for the optimal adjustment algorithm for p coordinates used is the following: maximum number of iterations (100) or the relative error of the residual norm smaller than a given tolerance (10 −4 ) (the one that occurs first).

Results
The performance of the PCx and PCxMod was compared with respect to the total number of iterations and the run time.The results are presente The total number of i Iterations) and Time) of two versions of the PCx are showed in Table 2.The PCxMod is the version that uses the optimal adjustment algorithm for p coordinates in Mehrotra's heuristic and PCx is the one that does not adopt it.Column p shows the value of p used by the optimal adjustment algorithm.Column ItAux gives the number of iterations performed by the optimal adjustment algorithm for p coordinates.problem) for the PCx.In this case, the run time of the PCxMod was lower because a better starting point was used.
In almost all problems with larger running time, the new approach performed better than the traditional one.This reveals a welcome feature of the proposed approach, since t According to the results, th to n the o pro s and th h .M ove total num of roblem n solved.
Although the total number of iterations did not decrease in some problems, a better starting point was determined as the optimal adjustment algorithm for p coordinates was used after Step 1 of the Mehrotra's heuristic and, consequently, the running time was reduced.
An interesting result is the fact that two problems (co9 and nug08) In another experiment, the same set of test problems were solved using only the PCxMod.First, p = 2 (2-coord) was considered, which represents the optimal pair adjustment algorithm and after the value of p according to the stopping cr ined.The results obtained are shown in Table 3.
The results showed that the PCxMod takes less iterations with values of p greater than 2 in 40.8% of the tests problems.For p = 2 this number is approximately 5.3%.The total running time was reduced by about 60 e problems for p-coordinates and only 22.4% were solved in a lower time as p = 2 was conside Again the problem co9 obtained status optimal only for p = 8.
These tests confirmed that the performance of the algorithm is improved by increasing the value of p.That happens because the residual b k has a greater reduction from one iteration to another and also because the int by the algorithm as a solution is different for each value of p.The obtained points for p > 2 achieved better performance in most cases.
It should be mentioned that the required total time to obtain a solution for the optimal adjustment algorithm for p coordinates is not significant in relation to t tion total time of the problems.This time is almost null in many of oblem, the time spent by the optimal adjustment algorithm was 24 seconds, which represents approximately 0.07% of the total running time.

Conclusions and Future Work
In this work, the optimal adju  total number of iterations and the running time for many problems, mainly large-scale ones.In addition, two problems were solved to optimality only by this approach.That is, using the algorithm for p coordinates to improve the starting point computed by PCx leads to a more robust implementation.By incorporating the optimal adjustment algorithm for p coordinates on the code PCx, more specifically Mehrotra's heuristic, and performing a few iterations, a more robust implementation was obtained.
The computational experiments on a set of linear programming problems showed that this approach can reduce the total number of iterations and total run time for several problems, especially the larger problems and ose with higher running time.Moreover, the optimal since the number of iterations performed by such an algorithm directly influences the quality of the starting point determined.New forms of choosing the p columns w so som lved ms was .e w his heuristic the choi of ber dina sev peri p that reduc the numb of fo ns by about 90% ot the test for the lems we ptimal adill be developed.

Acknowledgements
This research was sponsored by the Brazilian Agencies CAPES and CNPq.
, the triangle 0b k−1 b k has hypotenuse u k−1 = 0b k−1 and side u k = 0b k .The effort per iteration of von Neumann's algorithm is P s is the co us, 0 < λ < 1. Mo do .Optimal Pair Adjustment Algorithm his PhD thesis [4], Gonçalves studied von Neumann's ithm was proposed as an at xpected that the new residual b k is cl onds to making residual b k−1 move in the di ubject to
Any linear programming problem can be reduced to problem (1.1).For details of this transformatio Usually, to im ove the perf mance of in ior point methods, modifications are intro d after Step 4 of the al orithm for p coordinates before centralizin st problems. 1