This paper presents a new heuristic to linearise the convex quadratic programming problem. The usual Karush-Kuhn-Tucker conditions are used but in this case a linear objective function is also formulated from the set of linear equations and complementarity slackness conditions. An unboundedness challenge arises in the proposed formulation and this challenge is alleviated by construction of an additional constraint. The formulated linear programming problem can be solved efficiently by the available simplex or interior point algorithms. There is no restricted base entry in this new formulation. Some computational experiments were carried out and results are provided.
There are so many real life applications for the convex quadratic programming (QP) problem. The applications include portfolio analysis, structural analysis, discrete-time stabilisation, optimal control, economic dispatch and finite impulse design; see [
In this paper we present a new heuristic to linearise the convex quadratic programming problem. The usual Karush-Kuhn-Tucker conditions are still used but in this case a linear objective function is also formulated from the set of linear objective function equations and the complementary slackness conditions. There is an unboundedness challenge that is associated with the proposed linear formulation. To alleviate this challenge, an additional constraint is constructed and added to the linear formulation. The new linear formulation can be solved efficiently by the available simplex and interior point algorithms. There is no restricted base entry in the proposed approach. The time consuming complementarity pivoting is no longer necessary. Some computational experiments have been carried out and the objective of the computational experiments was to determine CPU times of the:
1) Proposed heuristic;
2) Regularised Active Set Method Mae and Saunders [
3) Primal-Dual Interior Point Algorithm.
It may be noted that the proposed method is suitable only if the quadratic programming problem satisfies conditions (1) to (5) mentioned in Section 2.1.
Let a quadratic programming (QP) problem be represented by (1).
Minimize
Subject to:
where
It is assumed that:
1) Matrix
2) Function
3) The conditions
4) Since constraints are linear then the solution space is convex, and
5) Any maximization quadratic problem can be changed into a minimization and vice versa.
When the function
The convex quadratic programming problem has special features that we can capitalize on when solving. All constraints are linear and the only nonlinear expression is the objective function. Let the Lagrangian function for the QP problem be
where
Complementary slackness conditions are given in (5) and are only satisfied at the optimal point. These conditions are:
Note
Suppose
Equations (6) and (7) can be easily verified. These simple results are used to eliminate the complementary slackness conditions.
Pre-multiply (3) by X, we have:
From (6)
By rearranging, we have
Pre-multiply (4) by
Since from (4)
From (7), we have:
Subtracting (14) from (15), we obtain (16):
Note that the expression in relation (13) is nonlinear but it can be rearranged so that the original quadratic programming objective function becomes a linear quantity. This can be achieved as follows:
Divide relation (16) by two, one obtains:
Rearranging (17), we obtain (18):
From (1)
Thus the nonlinear objective function of the QP problem is now linearised but it creates a new challenge. We will discuss this in the next section.
From (1), (3) and (20), we have the following LP problem:
Minimize
Subject to:
The minimisation problem (21) will have an unbounded solution due to negative coefficient of
of unboundedness in the LP (21). Here, we let:
tor of dimension
Minimize
coefficients. Both of these constants do not have to assume the same large values. A large number of experiments were done on a large number of quadratic programming problems and and it was observed that
The optimal solution of a convex quadratic programming model is unique and it satisfies the complementary conditions
Minimize
Subject to:
This example was taken from Jensen and Bard (2012) without any modifications.
Linear formulation of the above QP
In this case we took
Maximize
Subject to
The solution of (23) by the simplex method is given by:
From the original QP objective function, we have the objective value given in (25).
Verification of optimality
The solution is optimal because complementary slackness conditions are satisfied as given in (26).
Two more examples are solved to illustrate how the large constants are selected. Example 2 is taken from [
Example 2 from [
Minimize
Subject to:
The linear formulation of (27) becomes as given in (28).
Maximize
Such that:
The solution of (28) is as given in (29) and once again it is optimal as all complementary slackness conditions are satisfied.
Example 3 from [
Minimize
Subject to:
The linear formulation of the above example is given by (30).
Maximize
Subject to:
The solution is given by:
A set of convex quadratic programming test problems are given in [
1) To determine that the LP optimal solution is also optimal to the given QP.
2) Compare CPU times of the proposed heuristics with Regularized Active Set Method and Primal-Dual Interior Point Method
The results are tabulated in
Exp. No. | Prob. Name | No. of constraints (m) | No. of Variables (n) | CPU secs Proposed Heuristic | CPU secs Active Set | CPU secs Interior Point |
---|---|---|---|---|---|---|
1 | AUG2D | 10,000 | 20,200 | 29.34 | 0.55 | 15.12 |
2 | AUG2DC | 10,000 | 20,200 | 34.39 | 0.57 | 14.25 |
3 | AUG2DCQP | 10,000 | 20,200 | 21.89 | 240.73 | 14.63 |
4 | AUG2DQP | 10,000 | 20,200 | 37.19 | 228.72 | 14.76 |
5 | AUG3D | 1000 | 3873 | 0.29 | 0.07 | 1.65 |
6 | AUG3DC | 1000 | 3873 | 0.45 | 0.06 | 1.69 |
7 | AUG3DCQP | 1000 | 3873 | 0.82 | 3.84 | 1.39 |
8 | AUG3DQP | 1000 | 3873 | 0.53 | 5.02 | 1.56 |
9 | BOYD1 | 18 | 93,261 | 89.67 | 214.24 | 107.10 |
10 | BOYD2 | 0 | 93,263 | * | 4168.93 | 2245.64 |
11 | CONT-050 | 2401 | 2597 | 0.31 | 0.84 | 3.37 |
---|---|---|---|---|---|---|
12 | CONT-100 | 9801 | 10197 | 1.88 | 26.37 | 19.12 |
13 | CONT-101 | 10,098 | 10197 | 112.7 | 35855.97 | 20.66 |
14 | CONT-200 | 39,601 | 40,397 | 76.16 | 277.68 | 136.22 |
15 | CONT-201 | 40,198 | 40,397 | 51.12 | 285.50 | 143.56 |
16 | CONT-300 | 90,298 | 90,597 | 219.47 | 2449.75 | 721.76 |
17 | CVXQP1-L | 5000 | 10,000 | 413.18 | 4516.19 | 2488.97 |
18 | CVXQP1-M | 500 | 1000 | 0.65 | 5.94 | 1.85 |
19 | CVXQP1-S | 50 | 100 | 0.01 | 0.04 | 0.21 |
20 | CVXQP2-L | 2500 | 10,000 | 218.82 | 670.50 | 443.34 |
21 | CVXQP2-M | 250 | 1000 | 0.63 | 4.24 | 1.52 |
22 | CVXQP2-S | 25 | 100 | 0.02 | 0.04 | 0.26 |
23 | CVXQP3-L | 7500 | 10,000 | 76.22 | 14069.08 | 736.74 |
24 | CVXQP3-M | 750 | 1000 | 0.65 | 17.10 | 2.63 |
25 | CVXQP3-S | 75 | 100 | 0.02 | 0.40 | 0.23 |
26 | DPKLO1 | 77 | 133 | 0.02 | 0.01 | 0.17 |
27 | DTOC3 | 9998 | 1499 | 74.1 | 0.32 | 107.10 |
28 | DUAL1 | 1 | 85 | ~0.00 | 0.03 | 0.46 |
29 | DUAL2 | 1 | 96 | ~0.00 | 0.01 | 0.43 |
30 | DUAL3 | 1 | 111 | ~0.00 | 0.03 | 0.58 |
31 | DUAL4 | 1 | 75 | ~0.00 | 0.01 | 0.37 |
32 | DUALC1 | 215 | 9 | ~0.00 | 0.02 | 0.60 |
33 | DUALC2 | 229 | 7 | ~0.00 | 0.01 | 0.44 |
34 | DUALC5 | 278 | 8 | ~0.00 | 0.01 | 0.24 |
35 | DUALC8 | 503 | 8 | ~0.00 | 0.02 | 0.70 |
36 | EXDATA | 3001 | 3000 | 154.08 | ~0.00 | 200.08 |
37 | GENH28 | 8 | 10 | ~0.00 | 7.76 | 0.05 |
38 | GOULDQP2 | 349 | 699 | 0.31 | 0.87 | 0.78 |
39 | GOULDQP3 | 349 | 699 | 0.08 | 0.01 | 0.65 |
40 | HS118 | 17 | 15 | ~0.00 | ~0.00 | 0.14 |
41 | HS21 | 1 | 2 | ~0.00 | ~0.00 | 0.14 |
42 | HS268 | 5 | 5 | ~0.00 | 0.00 | 0.16 |
43 | HS35 | 1 | 3 | ~0.00 | ~0.00 | 0.05 |
44 | HS35MOD | 1 | 3 | ~0.00 | ~0.00 | 0.08 |
45 | HS51 | 3 | 5 | ~0.00 | 0.00 | 0.05 |
46 | HS52 | 3 | 5 | ~0.00 | ~0.00 | 0.04 |
47 | HS53 | 3 | 5 | ~0.00 | ~0.00 | 0.09 | |||||
---|---|---|---|---|---|---|---|---|---|---|---|
48 | HS76 | 3 | 4 | ~0.00 | 5.44 | 0.06 | |||||
49 | HUES-MOD | 2 | 10,000 | 2.44 | 5.32 | 3.78 | |||||
50 | HUESTIC | 2 | 10,000 | 1.15 | 9.64 | 3.42 | |||||
51 | KSIP | 1001 | 20 | 0.64 | 6.58 | 2.55 | |||||
52 | LASER | 1000 | 1002 | 0.56 | 213.02 | 1.99 | |||||
53 | LISWET1 | 10,000 | 10,002 | 26.87 | 215.75 | 14.89 | |||||
54 | LISWET10 | 10,000 | 10,002 | 26.12 | 223.19 | 15.12 | |||||
55 | LISWET11 | 10,000 | 10,002 | 24.97 | 234.96 | 15.03 | |||||
56 | LISWET12 | 10,000 | 10,002 | 28.67 | 223.23 | 15.01 | |||||
57 | LISWET2 | 10,000 | 10,002 | 46.96 | 212.28 | 135.27 | |||||
58 | LISWET3 | 10,000 | 10,002 | 33.16 | 214.23 | 145.28 | |||||
59 | LISWET4 | 10,000 | 10,002 | 39.17 | 212.05 | 150.86 | |||||
60 | LISWET5 | 10,000 | 10,002 | 28.18 | 211.28 | 153.74 | |||||
61 | LISWET6 | 10,000 | 10,002 | 40.06 | 203.76 | 136.04 | |||||
62 | LISWET7 | 10,000 | 10,002 | 23.12 | 217.98 | 14.63 | |||||
63 | LISWET8 | 10,000 | 10,002 | 24.05 | 230.47 | 14.54 | |||||
64 | LISWET9 | 10,000 | 10,002 | 35.08 | ~0.00 | 15.05 | |||||
65 | LOTSCHD | 7 | 12 | ~0.00 | 7.14 | 0.09 | |||||
66 | MOSARQP1 | 700 | 2500 | 1.08 | 3.10 | 0.57 | |||||
67 | MOSARQP2 | 600 | 900 | 2.17 | 190.45 | 0.47 | |||||
68 | POWELL 20 | 10,000 | 10,000 | 23.18 | 0.13 | 10.88 | |||||
69 | PRIMAL 1 | 85 | 325 | 0.02 | 0.25 | 0.46 | |||||
70 | PRIMAL 2 | 96 | 649 | 0.03 | 0.64 | 0.64 | |||||
71 | PRIMAL 3 | 111 | 745 | 0.02 | 0.34 | 1.23 | |||||
72 | PRIMAL 4 | 75 | 1489 | 0.02 | 0.34 | 1.18 | |||||
73 | PRIMALC1 | 9 | 230 | ~0.01 | 0.34 | 0.42 | |||||
74 | PRIMAL C2 | 7 | 231 | ~0.01 | 0.36 | 0.26 | |||||
75 | PRIMALC5 | 8 | 287 | ~0.01 | 0.87 | 0.20 | |||||
76 | PRIMALC8 | 8 | 520 | ~0.02 | 0.08 | 0.31 | |||||
77 | Q25FV47 | 820 | 1571 | 32.16 | 0.01 | 11.85 | |||||
78 | QADLITTL | 56 | 97 | 0.01 | 2.18 | 0.17 | |||||
79 | QAFIRO | 27 | 32 | 0.01 | 0.31 | 0.14 | |||||
80 | QBANDM | 305 | 472 | 0.03 | 0.36 | 0.54 | |||||
81 | QBEACONF | 173 | 262 | 0.04 | 0.82 | 0.35 | |||||
82 | QBORE3D | 233 | 315 | 0.06 | 1.27 | 0.51 | |||||
83 | QBRANDY | 220 | 249 | 0.05 | 1.86 | 0.35 |
---|---|---|---|---|---|---|
84 | QCAPRI | 271 | 353 | 0.06 | 4.22 | 1.18 |
85 | QE226 | 223 | 282 | 0.04 | 5.39 | 0.50 |
86 | QETAMACR | 400 | 688 | 0.06 | 1.26 | 1.86 |
87 | QFFFFF80 | 524 | 854 | 0.07 | 6.24 | 1.54 |
88 | QFORPLAN | 161 | 421 | 0.09 | 3.77 | 1.13 |
89 | QGFRDXPN | 616 | 1092 | 1.06 | 8.42 | 2.04 |
90 | QGROW15 | 300 | 645 | 0.08 | 0.84 | 1.32 |
91 | QGROW22 | 440 | 946 | 0.05 | 0.66 | 2.09 |
92 | QGROW7 | 140 | 301 | 0.04 | 0.10 | 0.81 |
93 | QISRAEL | 174 | 142 | 0.02 | 3.68 | 0.71 |
94 | QPCBLEND | 74 | 83 | 0.01 | 0.78 | 0.22 |
95 | QPCBOE11 | 351 | 384 | 0.02 | 1.61 | 1.24 |
96 | QPCBOE12 | 166 | 143 | 0.05 | 55.42 | 0.69 |
97 | QPCSTAIR | 356 | 467 | 0.08 | ~0.00 | 0.86 |
98 | QPILOTNO | 975 | 2172 | 8.15 | 0.02 | 4.76 |
99 | QPTEST | 2 | 2 | ~0.00 | 0.30 | 0.08 |
100 | QRECIPE | 91 | 180 | 0.08 | 2.58 | 0.41 |
101 | QSC205 | 205 | 203 | 0.09 | 0.13 | 0.30 |
102 | QSCAGR25 | 471 | 500 | 0.05 | 1.61 | 0.63 |
103 | QSCAGR7 | 129 | 140 | 0.08 | 5.82 | 0.35 |
104 | QSEFXM1 | 330 | 457 | 0.08 | 12.18 | 0.85 |
105 | QSEFXM2 | 660 | 914 | 0.13 | 0.99 | 1.55 |
106 | QSEFXM3 | 990 | 1371 | 1.12 | 5.25 | 2.38 |
107 | QSCRPIO | 388 | 358 | 0.05 | 0.95 | 0.35 |
108 | QSCRS8 | 490 | 1169 | 0.08 | 4.64 | 1.14 |
109 | QSCSD1 | 77 | 760 | 0.87 | 29.78 | 6.87 |
110 | QSCSd6 | 147 | 1350 | 0.09 | 2,71 | 0.68 |
111 | QSCSD8 | 397 | 2750 | 0.04 | 22.41 | 1.13 |
112 | QSETAP1 | 300 | 480 | 0.08 | 39.33 | 0.50 |
113 | QSETAP2 | 1090 | 1880 | 0.23 | 1.58 | 1.17 |
114 | QSETAP3 | 1480 | 2480 | 0.07 | 0.40 | 1.51 |
115 | QSEBA | 515 | 1028 | 0.06 | 0.11 | 1.80 |
116 | QSHARE1B | 117 | 225 | 0.04 | 23.37 | 0.44 |
117 | QSHARE2B | 96 | 79 | 0.02 | 6.36 | 0.27 |
118 | QSHELL | 536 | 1775 | 0.03 | 3.55 | 3.03 |
119 | QSHIP04L | 402 | 2118 | 0.08 | 48.37 | 1.05 |
---|---|---|---|---|---|---|
120 | QSHIP04S | 402 | 1458 | 0.11 | 13.18 | 0.72 |
121 | QSHIP08L | 778 | 4283 | 1.03 | 23.35 | 6.10 |
122 | QSHIP08S | 778 | 2387 | 0.79 | 12.19 | 1.75 |
123 | QSHIP12L | 1151 | 5247 | 1.26 | 1.90 | 11.76 |
124 | QSHIP12S | 1151 | 2763 | 0.16 | 2.68 | 2.24 |
125 | SIERRA | 1227 | 2036 | 0.12 | ~0.00 | 3.79 |
126 | QSTAIR | 356 | 467 | 0.07 | 36.94 | 0.87 |
127 | QSTANDAT | 359 | 1075 | 0.05 | 39.34 | 0.98 |
128 | S268 | 5 | 5 | ~0.00 | 95.97 | 0.16 |
129 | STADAT1 | 3999 | 2001 | 0.08 | 12.28 | 6.61 |
130 | STADAT2 | 3999 | 2001 | 0.13 | 1.87 | 8.12 |
130 | STADAT3 | 7999 | 4001 | 0.09 | ~0.00 | 14.16 |
131 | STCQP1 | 2052 | 4097 | 0.15 | 759.14 | 1.87 |
132 | STCQP2 | 2052 | 4097 | 0.06 | 0.61 | 3.89 |
133 | TAME | 1 | 2 | ~0.00 | 9.38 | 0.03 |
134 | UBH1 | 12,000 | 18,009 | 34.54 | ~0.00 | 62.83 |
135 | VALUES | 1 | 202 | ~0.00 | 0.55 | 0.51 |
136 | YAO | 2000 | 2002 | 0.08 | 0.57 | 3.66 |
137 | ZECEVIC2 | 2 | 2 | ~0.00 | 240.73 | 0.83 |
(Dual core G2020 2.9 GHz CPU, 2GB DDR3 1333 RAM) was used in these experiments. There were no advanced processing techniques embedded within the three methods. The set up time was excluded from the CPU times in all three methods. The zero (~0.00) means CPU time is less than 0.01 second. In all the test problems, it was found that the LP optimal solution was optimal to the QP problem. However, in the CPU time challenges were observed with the BODYD2 for the proposed heuristic and as a result we could not accurately obtain the necessary CPU time for these two cases. There was no challenge with the other two methods on the same BODYD2 problem. This experiment was conducted twice, but the same observation. We have no reason to support this behaviour but we believe it may be due to some local computational environment.
The convex QP problem can be solved like a linear programming problem efficiently either by the simplex method or the interior point algorithm. The restricted base entry is not necessary by the proposed approach. Complementary slackness can retard the simplex method, which is roughly eight times slower than the full speed simplex method. Taking complementary slackness conditions away itself is a big reduction in the number of constraints in the proposed linear formulation of the quadratic programming problem. More experiments are likely to give more insight and advantages of the proposed approach. The proposed method is in fact the usual simplex method applied to solving an ordinary LP that was obtained from the given convex QP. Also note that a large number of Maros-Maszaros test problems are giving rise to small to medium size LPs and therefore the proposed method dominates solving a large number of QPs, as is reflected in
The authors are thankful to the referees for their helpful and constructive comments.
EliasMunapo,SantoshKumar, (2015) A New Heuristic for the Convex Quadratic Programming Problem. American Journal of Operations Research,05,373-383. doi: 10.4236/ajor.2015.55031