On Merging Cover Inequalities for Multiple Knapsack Problems

This paper describes methods to merge two cover inequalities and also simultaneously merge multiple cover inequalities in a multiple knapsack instance. Theoretical results provide conditions under which merged cover inequalities are valid. Polynomial time algorithms are created to find merged cover inequalities. A computational study demonstrates that merged inequalities improve the solution times for benchmark multiple knapsack instances by about 9% on average over CPLEX with default settings.

A half space is { } 1 : , and a polyhedron is defined as the intersection of finitely many half spaces.A set n S ⊆  is convex if and only if 1  x and 2 x S ∈ implies ( ) A polyhedron is convex, and the convex hull of S, ( ) conv S , is the intersection of all convex sets that contain S.
Let P be the set of feasible points of an integer program, where , but any cutting plane that separates the linear relaxation from ( ) MK conv P may be computationally useful.A thorough explanation of such results is in Nemhauser and Wolsey [9].For a MK problem, a cover cut may be generated in one or more of the m constraints.A set C N ⊆ is a cover for row . The corresponding cover inequality is valid for ( ) MK conv P and takes the form 1 Cover cuts have been studied extensively by Balas and Zemel [10], De Farias et al. [11], Louveaux and Weismantel [12], Nemhauser and Vance [13], and Park [14].Knowledge of cover cuts is critical to this research.Many such covers may exist and pseudo-costing strategies provide a prioritized variable ordering.Pseudocosting strategies for integer programming problems were studied by Benichou, et al. in [15] and Gauthier and Ribiere in [16].Refalo used pseudo-cost strategies to improve constraint programming in [17], and Achterberg, et al. developed reliability branching rules for IPs as an extension of pseudo-costing in [18].
Theoretical foundations for inequality merging were first introduced by Hickman and Easton in [34].Although merging appears similar to lifting, it yields new cutting planes that are not attainable through straightforward applications of known lifting techniques.Their paper creates a single cutting plane by merging two inequalities.This merged inequality can be theoretically stronger than the original inequalities, and it may induce a facet under certain conditions.This paper extends the idea of inequality merging by focusing on cover inequalities in MK problems.Information from two or more cover inequalities in an MK instance may be merged into a single cutting plane.In some instances, simultaneous merging of cover inequalities may occur across multiple rows at the same time.
The next section describes the process of cover inequality merging for MK instances and provides theoretical results and examples.The third section offers the results of a computational study that highlights the computational benefits of employing merged cover inequalities in test MK problems.The final section offers some directions for future research.

Theory and Examples of Merging Cover Inequalities
If the merged inequality is valid, then this inequality includes more nonzero coefficients than either host C or donor C . The question remains as to whether or not the merged inequality is valid.The following theorem provides conditions for its validity.
Theorem 1.Let host C be a cover from row r and donor C be a cover from some row s in a MK instance such that as the merging index with the restriction that if , then the merged cover inequality, is a cover in some constraint for each and the result follows. Theorem 1 describes which indices can be used to create a donor cover.These candidate indices can be easily found based upon a ψ threshold, which is associated with the host cover inequality and the merging variable.Given a host cover host C in row r and a designated merging variable . Every . These two cases are exhaustive.Therefore every and this merged inequality is valid for ( ) To identify valid merged cover inequalities, the user must identify a host cover, host C and a merged index host p C ∈ .Some selections for host C and a merged variable p x may not allow a candidate donor inequality to exist.The Reducing p ψ Algorithm changes host C to increase the likelihood of the existence of an appropriate donor cover.
The input to the Reducing p ψ Algorithm is a multiple knapsack instance, a valid host cover from row r and a merging variable Observe that the Reducing p ψ Algorithm also requires a careful selection of τ to achieve stronger results in many instances.A small value of τ tends to allow indices with small a coefficients to enter Even if the algorithm succeeds, higher values of τ tend to result in relatively smaller reductions in p ψ , possibly requiring multiple calls to this procedure when a valid merged inequality is not yet attainable.Given this sensitivity to τ , a careful selection of τ is required.For practical purposes, it is recommended to consider values of τ between 0.3 and 0.7.
The Reducing p ψ Algorithm is a linear algorithm for each specified τ value.The initialization requires . The main step could search through all other indices, so it performs in ( ) effort.Thus, the algorithm runs in ( ) O N , which is linear for a fixed τ .

Merging over Multiple Donor Covers Simultaneously
This section presents a method to strengthen the previous results by merging on multiple donor covers at the same time.
is likely to be valid for any 1 is any cover from any constraint of the MK instance with The check of validity must assure that there does not exist a feasible point which violates this new inequality.
Prior to this result, define An immediate result of Theorem 3 is an algorithm to merge over multiple donor covers simultaneously.This algorithm explores all rows to determine the smallest eligible covers of each merging variable in .Thus DCSA's effort is ( ) . Although this is a cubic run time, DCSA performs quickly in practice.

Inequality Merging Example
The τ = , then the Reducing p ψ Algorithm seeks a host cover with a ψ value that is less than or equal to 5. In this case {5} is eliminated from the host cover, and the host cover adds an index with a coefficient between 5 and 9. Indices 10, 11, 12, and 13 are all suitable and index 11 is added to host C′ .However, { } The following arguments demonstrate Theorems 1 and 2 in practice.Verifying the validity of (1) requires that is a cover for some constraint for every Each of these merged inequalities remove linear relaxation points and are thus cutting planes.For instance, the point (1,1,1,1,0,0,0,0,0, 1 4 ,1,0,0,0) is eliminated by each of these merged inequalities.Additionally, it is simple to find points that are satisfied by two of the three merged inequalities, but eliminated by the other inequality.Thus, each merged inequality is eliminating distinct regions of the linear relaxation space.
Returning to the original host cover, it is also possible to generate new families of merged inequalities if merging on 11 p = instead of 12 p = .By changing the index selected for merging, ( ) The idea of ψ guarantees validity, but it is not necessary to merge covers.Consider { } The authors believe that such constraints may be more useful computationally since they are incorporating covers from multiple constraints to obtain validity.For instance, the linear relaxation point (1,1,1, 1 3 ,0,1,0,0, 1 4 , 0,0,0,0, ) is eliminated by this inequality.
To demonstrate Theorem 3, an additional row is added to this example.Now consider the following multiple knapsack instance 20 Observe that this new inequality dominates all of the previous inequalities.Furthermore, to achieve this inequality all rows are necessary.For instance, the smallest cover in row 3 containing index 6 has 6 indices and thus row two is necessary.Similarly, the smallest cover in row 2 containing index 7 has 6 indices and thus row 3 is necessary.To argue validity of ( 6), consider Theorem 3. Since The final benefit of this example demonstrates that merging cover inequalities are not an immediate extension of known methods.There are similarities between inequality merging and some categories of lifting.Any type of sequential lifting has integer coefficients [35], and sequence independent lifting would require all non-cover coefficients in this example to be 0 [30].Thus neither of these methods generate (6).While simultaneous lifting could theoretically generate (6) [21], it would require starting with the trivial cutting plane 1 2 3 4 1 15 x x x x x + + + + ≤ and furthermore have a perfect guess of proper weights.Consequently, inequality merging yields inequalities similar to (6), which are extremely unlikely to be produced by lifting techniques.
The general inequality merging presented by Hickman and Easton in [34] did not merge multiple donor covers simultaneously, and it could not obtain (6).Inequality merging is also fundamentally different from other popular cutting plane generation techniques such as C-G cuts (Chvátal [36] and Gomory [37]), disjunctive cuts (Balas and Perregaard [38]), Gomory cuts (Gomory [37]), or superadditive cuts (Gomory and Johnson [39] and Wolsey [40]).Theoretically, these methods could generate ( 6), but they would require numerous iterative applications to find this cutting plane.Such a result is unlikely to occur without the consultation of an oracle to select initial inequalities, weights or other necessary input.
A single call to DCSA creates (6) and requires ( ) O nm effort.Thus, merging over cover inequalities is a new method to obtain previously unknown inequalities.Given the large size of most multiple knapsack problems, the flexibility of the construction algorithms are usually capable of finding strong candidate host C and donor C inequalities.The next section provides the results of a computational study, demonstrating the practical effectiveness of inequality merging on benchmark multiple knapsack problems.

Computational Study
This computational study compares solution times for multiple knapsack problems both with and without the use of merged inequalities.The instances chosen for this study are the MK instances from the OR-Library [41], developed by Chu and Beasley in 1998 [42].The majority of these instances are either trivially solved or too computationally intensive for an optimal solution.Thus, this study focuses on medium sized instances contained in files mknapcb2 ( 5 m = and 250 n = ) and mknapcb5 ( 10 m = and 250 n = ).Each file contains 30 instances divided into groups of 10 based upon a tightness ratio, which is equal to . The tightness ratio is approximately equal for all constraints and is 0.25 for the first 10 instances, 0.5 for the second ten instances, and 0.75 for the final ten instances.For this computational study, the first ten instances are only considered.When the tightness ratio is 0.5 or higher, host C tends to include too many variables.Since the variables in host C are prohibited from being in p N ψ , higher tightness ratios reduce the size of p N ψ , which decreases the likelihood of finding a suitable donor cover in any row.The study considers a variety of implementation strategies including the number of merged inequalities added, the possibility of overlapping rows when multiple cuts are added, the option to use the Donor Coefficient Strengthening Algorithm when constructing merged inequalities, and different pseudocosting techniques.The psuedocosting techniques provide an order for selecting indices for cover inequalities.Three options are considered: sorting on the reduced costs, sorting on the a coefficient values, and sorting on equal weights for both reduced costs and a coefficient values.More details of these methods and computational results are described in [43].
The experimentation compares computational effort to solve the MK instances with and without the inclusion of merged cover inequalities.CPLEX 12.5 [44] solves all of the instances at default settings, but writing node files out to memory is used for the larger instances.All results are obtained using a PC with an i7-4770 processor at 3.4 GHz with 8 GB of RAM.

Computational Results
The computational study considered the variations of each implementation strategy by testing both small and large instances.Solving all 10 smaller instances required from 10 to 15 minutes.Solving all 10 larger instances typically needed 1 to 2 days.Instead of reporting the time in seconds, the data below compares computational ticks in CPLEX, as this is more accurate.It should be noted that the time in seconds was highly correlated to ticks.The overall improvement in time was plus or minus two percent of the percent improvement in ticks.Ticks provide a more accurate comparison between the experimental runs because the computational time in seconds is subject to variability on different computers.Fischetti, et al. argue the benefit of using ticks in [45].Ju, et al. use a similar process to report their computational results [46].Since the two categories of MK test problems included 10 multiple knapsack subordinate instances, most of the tables compare the aggregate total ticks required to solve all 10 problems using the baseline CPLEX 12.5 and the inequality merging technique.

Computation Results for Smaller Problems
Problems from the smaller MK instances (file mknapcb2) offered an excellent opportunity for extensive experimentation with each of the implementation strategies.Table 2 and Table 3 show the best known results  from these experiments on the smaller MK instances.Since there are 5 rows in the smaller test problems, each implementation strategy was tested with the inclusion of 1 -5 merged inequalities.Table 2 shows the results for iterations with 1, 2, or 3 merged inequalities added.Table 3 shows the results with 4 or 5 merged inequalities added.
Observe that inequality merging outperformed the baseline CPLEX computational ticks for all strategies in Table 2 with 1, 2, or 3 added inequalities, and inequality merging also outperformed the baseline CPLEX by about 9% on average.The 4 and 5 cut strategies from Table 3 outperformed baseline CPLEX by about 4%.This demonstrates that adding more merged inequalities creates diminishing returns because of additional computational requirements as the A matrix and basis grow in size.Preferred implementation strategies should focus on including 1, 2, or 3 merged cutting planes.
Table 4 aggregates results from Table 2 and Table 3, and it reports the average results based upon different pseudo-costing strategies.Observe that many of the experimental runs in Table 2 and Table 3 included a pure strategy (all reduced costs, all a values, or all balanced cuts).However, some of the experimental runs include a mixture of strategies such as the 3 cut scenario with 1 cut of each pseudo-costing strategy.Experiments of this type are listed under "Mixture of Strategies" in Table 4. Notice that each of the three pure strategies performed well, at about the same level of improvement.However, there may be some additional benefit to mixing pseudocost strategies if multiple merged inequalities are generated.
Merged inequalities almost always improved the computational time, regardless of the overlapping strategy.It appears that deliberate overlapping of rows provides even stronger results if multiple cutting planes are added.This is consistent with the theory motivating Theorem 3. Overlapping allows the algorithm to search in rows that had previously been used to generate a host cover inequality for an earlier merged cut.If DCSA is employed, the algorithm may also search all candidate rows including those that had previously generated a host inequality.Thus, all future experimentation overlaps rows.

Computational Results for Larger Problems
As the problems increased in size, the computational time quickly increased.The same implementation strategies tended to yield the strongest results with larger problems, as shown in this section.Solving all 10 MK instances required from 1 to 2 days to solve.Table 5 shows the best known results for the large MK problems when the recommended implementation strategies are followed.
Table 5 shows that inequality merging continues to provide an average improvement of about 9% over the baseline CPLEX computational effort even on challenging instances.This is roughly the same level of average improvement observed in the smaller MK instances.Notice that following the recommended implementation strategies always improved the solution times.This provides strong evidence that inequality merging is a beneficial technique for MK problems, and the reduction of computational ticks correlates to hours of time savings for large problems.
Clearly a focus on reduced costs had the best impact for this particular grouping of larger MK instances, but that may not be the case in general.Previous analysis from Table 4 suggested that different pseudo-costing techniques may be preferred for particular problems, but focusing on reduced costs was actually the least preferred in that grouping of smaller MK instances.Identifying the reason that certain methods dominate other pseudo-costing techniques in particular problems is an excellent area for future research.
Table 6 shows the best solution times for each of the 10 MK instances in the larger files.In addition, the table also describes the implementation strategy that yields the best result for each problem.Merging improved the solution times for each of the 10 problems, with an average reduction of computational requirements by 25.8%.However, the best single result for each sub-problem came from a wide variety of implementation strategies.These include instances that search all donor rows with DCSA and other instances that consider only specified randomly-selected donor inequalities that define single overlaps.The two best results include both overlapping strategies and DCSA facilitated the single best percentage improvement in problem 1.It is clear that each strategy yields strong results in specific instances, and neither overlapping strategy dominates the other.These larger problems are excellent representatives of difficult, real-world problems.Thus, the observed reductions in computational requirements validated the theoretical advancements in this research as effective methods to help decrease computational effort for modern MK problems.

Conclusion and Future Work
This paper provides the theoretical foundations needed to build merged cover inequalities in MK instances.The theorems generate conditions for validity, using the p ψ term to identify candidate merging indices and simultaneously merging on all rows.Two algorithms support the newly-discovered theory, including an algorithm to reduce the size of p ψ and a second algorithm to find the strongest coefficients for each candidate index during simultaneous merging.
The computational study validates inequality merging as an effective technique that reduces computational time for multiple knapsack problems.Preferred implementation strategies should generate 1, 2, or 3 cuts and overlap the rows.These strategies provide the strongest results, yielding an average reduction of computational effort by about 9%.The computational study provides strong evidence that inequality merging yields productive cutting planes for MK problems, and it is likely that similar computational improvements will be achieved for other IPs.
Three ideas present themselves as excellent candidates for future research extensions.In this paper, inequality merging occurs on a single variable.The theory may be extended to merge on multiple variables.Since this paper focuses on cover inequalities and MK instances, another theoretical extension may merge other classes of cutting planes in general IPs.
All of the computational analysis in this research was performed on the first 10 problems of each file provided by Chu and Beasley [42] with a tightness ratio of 0.25.Other test problems exist in the same files with different tightness ratios, and future research should consider if varying tightness ratios tend to motivate different levels of computational improvement when merged cover inequalities are added to the MK instance.
An integer program (IP) is a common type of optimization problem, defined as maximize T c x subject to Ax b where m and n are integers both greater than or equal to the set of indices of an IP.
The multiple knapsack (MK) problem has multiple knapsack constraints and is defined as maximize T c x subject to Ax b ≤ and known technique to improve solution times for IP problems is the generation of valid inequalities. .If the valid inequality separates the linear relaxation solution from the convex hull of the IP, then it is called a cutting plane.The linear relaxation is the IP with the integrality restriction eliminated.The theoretically best cutting planes define facets of

.
The purpose of p ψ is to rapidly identify indices that can be used to create a donor cover from any row s.Define these potential donor indices as donor

.
provided.The output of this algorithm is a new host cover inequality and a new merging variable.These are denoted by When this happens, the last index q added to host C′ becomes the newly determined overlapped variable p x ′ , and acceptable additional variables for use in donor C .This increases the likelihood of achieving a valid donor C , thus increasing the opportunity for construction of a merged cutting plane inequality.In some instances, the Reducing p ψ Algorithm terminates successfully with a new cover host C′ and a new value p ψ ′ , but it may not have a sufficient number indices in p N ψ ′ to construct donor C .If this happens, the Reducing pψ Algorithm may be used iteratively until a suitable donor C is attained.
host C′ .When this happens, the size of host C′ may become undesirably large or fail to generate a cover.Including too many variables in the host cover results in fewer candidate indices in p N ψ .High values of τ may allow few (or zero) new candidate indices for inclusion in p N ψ .In such instances, it is more likely that the reducing p ψ algorithm fails to return a new host C′ and/or fails to reduce the value of p ψ .
cardinality cover from any row.
any cover from any constraint of the MK instance as long as one of the following conditions holds 1) 1 p N ψ .This translates into the stronger coefficients for each merging variable.The input to the Donor Coefficient Strengthening Algorithm (DCSA) is a MK instance, a host cover host C from row r and an index host p C ∈ .Donor Coeffcient Strengthening Algorithm DCSA identifies the smallest donor covers possible for each index in p N ψ from each row in the MK instance using the indices sorted in each row by the a values.Observe that DCSA does not guarantee a valid inequality, but it does identify the strongest possible merged inequality.If the reported merged inequality satisfies a condition of Theorem 3, then it is a valid inequality.DCSA's computational effort required for the initialization is

9 a 5 N
following example demonstrates the theoretical concepts discussed earlier.Consider multiple knapsack constraints of the form Ax bDesignate the first constraint as the host constraint, 1 r = and let the host cover be are all greater than or equal to 10, the candidate indices for the donor cover are restricted to ψ is a cover.The Reducing p ψ Algorithm is used to change the host cover to create a smaller ψ .Let 1 2 it requires five variables with coefficients in 12 N ψ to be set to one to arrive at a value strictly Conditions are provided to create valid inequalities from merging over three or more cover inequalities simultaneously.Another algorithm is presented to search for the strongest merging coefficients among multiple potential donor rows in the MK instance.
p N ψ .The inequality The algorithm has now determined a host and donor cover that can be merged.Merging the host with the donor on 12x yields (1), a valid inequality according to Theorem 2.
cover in the first knapsack as long as 1, 4

Table 1 .
The smallest covers are listed in the order in which DCSA adds indices to the cover.
12N ψ , DCSA forces this index as the first element in a cover and then adds other indices according to the sorted order for each row.Observe that { }

Table 1 .
Applying DCSA to find strongest coefficients.

Table 4 .
Average ticks of pseudo-costing strategies from Table2 and Table 3.

Table 6 .
Best merging performance by problem for