A Mixed-Variable Experimental Design Method

Zili Wan, Qi Wang^*

College of Aircraft Engineering, Nanchang Hangkong University, Nanchang, China.

DOI: 10.4236/ojapps.2024.142023                

Abstract

Mixed-variable problems are inevitable in engineering. However, few researches pay attention to discrete variables. This paper proposed a mixed-variable experimental design method (ODCD): first, the design variables were divided into discrete variables and continuous variables; then, the DVD method was employed for handling discrete variables, the LHD method was applied for continuous variables, and finally, a Columnwise-Pairwise Algorithm was used for the overall optimization of the design matrix. Experimental results demonstrated that the ODCD method outperforms in terms of the sample space coverage performance.

Keywords

Optimization; Experimental Design Methods, Discrete Variable

1. Introduction

With the advancement of computer technology, high-precision finite element simulation has become a pivotal method for addressing engineering challenges. While the accuracy of finite element simulation has gained wide acceptance in the industrial domain, its intensive computational time significantly hampers optimization efficiency. Consequently, surrogate models have garnered attention in the optimization domain due to their lower computational time costs [1] . Presently, they find extensive application in structural optimization design [2] [3] , aerodynamic optimization design [4] [5] , multidisciplinary optimization [6] , among other areas. Experimental design methods represent a crucial phase in surrogate model optimization. Efficient experimental design enables the acquisition of maximal information within limited experimental runs, thereby elucidating the impact of various experimental factors on outcomes while conserving computational resources [7] .

The concept behind experimental design is to select the minimum number of sample points to maximize the information obtained about the unknown space [8] . Common experimental design methods include full factorial design [9] , central composite design [10] , Box-Behnken design [11] , uniform design [12] , orthogonal design [13] , and Latin Hypercube sampling design [14] . In order to reduce experimental resource usage while fully revealing the impact of each experimental factor on the experimental results, researchers have introduced optimization criteria to optimize experimental samples, leading to the development of optimized experimental design methods. Morris and Mitchell targeted the φ_p criterion, utilizing a simulated annealing algorithm to optimize the design matrix [15] . Li proposed the use of the CP algorithm, with φ_p and maximum entropy as objectives, employing a column element exchange method to optimize the design matrix [16] . Saab introduced the SE algorithm, utilizing a threshold-based approach to determine the acceptance of a new solution [17] . Jin improved the SE algorithm and presented the ESE algorithm, which achieves self-adjustment through a complex combination of heating and cooling plans to adapt to different experimental design problems of varying categories, optimization criteria, and design scales [18] . XL Liu and Bo Zheng, using orthogonal measurement criteria and uniformity measurement criteria as optimization objectives, proposed a multi-objective optimization experimental design method based on the improved ESE algorithm [19] . Ye combined the continuous local enumeration method and the translation propagation algorithm to propose a rapid optimization Latin Hypercube experimental design method that significantly improves efficiency while ensuring accuracy [20] .

However, existing experimental design methods can only address continuous variable problems, and there are very few methods capable of handling mixed variables (involving both continuous and discrete variables). Currently, when encountering discrete variable issues, the typical approach involves sampling within the continuous domain and then rounding the obtained results to the nearest discrete points, rather than directly generating sample points within the mixed domain [21] . Due to the specific nature of discrete variables, this method results in poor spatial uniformity of samples. This paper proposes an Optimized Discrete and Continuous variable sampling Design (ODCD) method specifically tailored for mixed variables. This method can ensure stronger spatial uniformity.

2. General Methods for Handling Mixed Variables

Mixed variables encompass both continuous and discrete variables. While existing experimental design methods are applicable to continuous variables, the predominant approach for discrete variables involves sampling within the continuous domain and then rounding to the nearest discrete points [21] . However, this method often lacks in guaranteeing robust experimental design performance.

To illustrate this method with an example, let’s consider an example, y, which is a discrete variable taking values within the interval [w₁, w₂], containing three discrete values: a, b, and c. Using a continuous variable experimental design method, n sample points x₁, x₂, ..., x_n are selected within the range [w₁, w₂]. These sample points are then compared to the three discrete values, evaluating the distances L_ia, L_ib, and L_ic between each sample point x_i and the three discrete values. If L_ic is the smallest, x_i is assigned as c, and similarly for other cases.

For ease of description, this paper refers to the method of using the Latin Hypercube Design (LHD) for continuous variables and LHD sampling followed by rounding to the nearest discrete value for discrete variables to obtain the design matrix for mixed variables as Mixed variable Latin Hypercube sampling Design (MLHD).

3. The ODCD Method

In mixed variables, there are both discrete and continuous variables. Due to the unique nature of discrete variables, it’s imperative to sample discrete and continuous variables separately. The optimization experimental design method for handling continuous variables has become quite mature. Essentially, the design matrix for mixed variables is akin to that of continuous variables, as they both involve combinations of numbers. The ODCD method proposed in this paper segregates design variables into two parts, handling them separately using the Discrete Variable Design method (DVD) and the Latin Hypercube Design method. This is done with spatial coverage performance as a metric, leveraging random optimization techniques for holistic optimization. The process is outlined as follows:

1) Segregate design variables into discrete and continuous components.

2) Apply the DVD method to discrete variables to obtain the design matrix.

3) Employ the LHD method for continuous variables to derive the design matrix.

4) Combine the design matrices for discrete and continuous variables into a mixed variable design matrix.

5) Utilize Columnwise-Pairwise Algorithms to optimize the design matrix for mixed variables.

3.1. DVD

The challenge in mixed-variable experimental design lies in the presence of both discrete and continuous variables. Discrete variables have a finite number of possible values, and their distribution may be either uniform or non-uniform. Existing methods for discrete variable sampling typically involve selecting values in the continuous domain and rounding them to the nearest discrete points. This approach inevitably results in some values being excluded or duplicated, thereby compromising the uniformity of the samples. Therefore, a new experimental design method for discrete variables, referred to as Discrete Variable Sampling Design (DVD), needs to be developed based on the nature of discrete variables.

The implementation process of the DVD method is as follows:

Form a set K containing all combinations of discrete variables, with the total number of combinations denoted as m, and the desired number of design samples as N.

1) Calculate the quotient a by rounding down N/m, and determine the remainder b.

2) Duplicate the set K times to obtain the design sample set X₁.

3) Randomly select b points from the set K to form the design sample set X₂.

4) Combine X₁ and X₂ to create the complete design matrix.

To illustrate the DVD method more intuitively, consider a simple example with discrete variables x₁ and x₂, each having five and six discrete values, respectively. The goal is to generate 64 sample points in a two-dimensional space. Following the outlined steps, the total number of combinations m for x₁ and x₂ is 30, with set K comprising these 30 combinations. When N/m is rounded down, a is determined to be 2, and the remainder b is 4. Consequently, X₁ is created by duplicating K two times, and X₂ is formed by randomly selecting 4 points from K. Finally, X₁ and X₂ are combined to produce a design matrix containing 64 sample points.

3.2. Latin Hypercube Design

Latin Hypercube Design (LHD) is a representative method in the field of “space- filling” experimental design, renowned for its excellent uniformity and low-di- mensional projection performance. It has become one of the most popular sampling methods in computer experiments [22] . In this paper, we employ this method to address continuous variables. For a trial design problem of size N × d, where N is the number of samples, and d is the dimensionality of the samples, the process of the LHD method is outlined as follows:

Divide each dimension into N subsets, each subset having a length of 1/N. Select one point from each subset, resulting in N points for each dimension within the design space. Randomly permute the N values for each dimension, yielding an initial design matrix denoted as A.

$A=\left[\begin{array}{cccc}{x}_{11}& x_{12}& \cdots & x_{1d}\\ x_{21}& x_{22}& \cdots & x_{2d}\\ ⋮& ⋮& \ddots & ⋮\\ x_{N1}& x_{N2}& \cdots & x_{Nd}\end{array}\right]$

3.3. Composition of Mixed-Variable Design Matrix

A mixed-variable set comprises d continuous variables, x₁, x₂, ..., x_d, and e discrete variables, y₁, y₂, ..., y_e, with a total of N generated samples. The LHD experimental design method is employed to handle the continuous variable x₁, resulting in a set {x₁₁, x₂₁, ..., x_N1}, which is incorporated as the first column of matrix X. Similarly, other continuous variables are processed until x_d, yielding a set {x_1d, x_2d, ..., x_Nd}, which forms the dth column of X.

The DVD experimental design method is then applied to handle the discrete variable y₁, obtaining a set {y₁₁, y₂₁, ..., y_N1}, which is included as the (d + 1)th column of X. This process is repeated for other discrete variables until y_e, resulting in a set (y_1e, y_2e, ..., y_Ne), which is added as the (d + e)th column of X. At this point, the complete design matrix X is obtained.

$X=\left\{\begin{array}{cccccc}{x}_{11}& \cdots & x_{1d}& y_{11}& \cdots & y_{1e}\\ x_{21}& \cdots & x_{2d}& y_{21}& \cdots & y_{2e}\\ ⋮& \ddots & ⋮& ⋮& \ddots & ⋮\\ x_{N1}& \cdots & x_{Nd}& y_{N1}& \cdots & y_{Ne}\end{array}\right\}$

3.4. Columnwise-Pairwise Algorithms

Li proposed the Columnwise-Pairwise Algorithms (CP) with the objective of maximizing entropy, aiming to optimize the design matrix. CP algorithms are effective in handling permutation optimization problems. The method is particularly easier to keep the structure properties of a design in relation to columns, such as the balance and orthogonality properties. This article will utilize the Columnwise-Pairwise Algorithms to optimize the design matrix X. In contrast to Li’s proposed Columnwise-Pairwise Algorithms, this paper adopts a different set of convergence criteria. Specifically, the criteria employed here include the maximum and minimum distance criterion, the φp criterion, and the minimum potential energy criterion.

The algorithm for searching optimal CP is summarized as follows:

1) Start with the X.

2) Each iteration has L steps. At the ith step, the best two simultaneous exchanges within column i are found. The design matrix is updated accordingly.

3) If the resulting design is better with respect to the criterion, repeat Step 2. Otherwise, it is considered to be an “optimal design”, and the search is terminated.

4. Comparison and Result Analysis of Experimental Sample Uniformity

To validate the effectiveness of the ODCD method, this study conducts an analysis using experimental design examples and compares the results with the LHD method.

4.1. Qualitative Comparison of Sample Uniformity

With a sample size of 20 and two dimensions, where in x₁ is a continuous variable and x₂ is a discrete variable encompassing 5 discrete values, both processed using the ODCD and MLHD methods. The resulting sample distributions are illustrated in Figure 1 and Figure 2. It is evident that the sample uniformity obtained by the ODCD method is superior to the MLHD method.

4.2. Quantitative Comparison of Sample Uniformity

To quantitatively characterize the spatial uniformity of the samples, this study employed three criteria: the Maximum and minimum distance criterion (d_min),

Figure 1. ODCD.

Figure 2. MLHD.

φ_p criterion(φ_p), and minimum potential energy criterion (U). Larger values of d_min indicate stronger spatial uniformity, while smaller values of φ_p and U indicate greater spatial uniformity. Below is a brief description of the three criteria.

1) Maximum and minimum distance criterion [23] . The objective of the minimum distance criterion is to maximize the minimum distance between experimental sample points, satisfying the condition:

$\max \left\{d_{\min }\right\}=\max \left\{\underset{1\le i,j\le m,i\ne j}{\min }d\left(x_i,x_j\right)\right\}$ (1)

where m is the number of sample points, and d(x_i, x_j) represents the distance between any two sample points, x_i and x_j

$d\left(x_i,x_j\right)=d_{ij}={\left[{\displaystyle {\sum }_{k=1}^n\left|x_{ik}-x_{jk}\right|t}\right]}^{\frac{1}{t}},t=1\text{or}2$ (2)

2) φ_p criterion [15] . Morris and Mitchell extended the application of the minimum distance criterion, introducing the φ_p criterion. If an experimental design is referred to as a φ_p-optimal design, it satisfies the following conditions:

$\min \left\{{\varphi }_p\right\}=\min \left\{{\left[{\displaystyle {\sum }_{i=1}^s{J}_i{d}_i^{-p}}\right]}^{\frac{1}{p}}\right\}$ (3)

For a given experimental design, calculate the distance d_ij between any two sample points (here, t = 1). Sort these distances to obtain a list of distance function values (d₁, d₂, ..., d_s) and their corresponding index list (J₁, J₂, ..., J_s). The distances d_i represent different distance values, and $d_{\text{1}}<d_{\text{2}}<\cdots <d_s$ , where J_i is the number of pairs with a distance of d_i, and S is the number of distinct distance values. In this formula, p is a positive integer set to 50.

3) Minimum Potential Energy Criterion [20] . This criterion is inspired by physical principles: when the system’s potential energy reaches a minimum, it is in an equilibrium state. The expression for potential energy U is given by:

$U={\displaystyle {\sum }_{i=1}^{m-1}{\displaystyle {\sum }_{j=i+1}^m{d}_{ij}^{-2}}}$ (4)

Ensuring identical variables, generate experiment samples of the same size using the ODCD method and MLHD method. To reduce experimental errors, repeat the process 100 times. Evaluate the test samples using d_min, φ_p, and U criteria, recording the optimal, worst, and average values for d_min, φ_p, and U. Table 1 presents the results for four variables, including two continuous variables and two discrete variables.

From the table, it can be observed that the average values of the performance criteria φ_p and U for the experiment samples generated using the ODCD method are smaller than those using the MLHD method, while the average value for d_min is greater than that of the MLHD method. This indicates that the ODCD method yields experiment samples with superior performance compared to the MLHD method. The results for optimal and worst values align with the average values.

To further illustrate the sample performance of the ODCD method, generate experiment samples of different scales using ODCD and MLHD methods, particularly in scenarios with numerous design variables. Evaluate the samples using the three criteria, and average the results over 50 iterations. The outcomes are presented in Table 2, where N denotes the sample quantity, A represents the sample dimensionality, a denotes the dimensionality of continuous variables, and b represents the dimensionality of discrete variables. Similar to the low-dimensional cases, the ODCD method demonstrates superior sample performance compared

Table 1. Four-dimensional samples.

Table 2. Medium-to-high-dimensional samples.

to the MLHD method.

5. Conclusion

Many engineering design problems involve mixed variables. However, conventional experimental design research has seldom focused on discrete variables. The ODCD method proposed in this paper offers an effective solution for problems with mixed variables. The method utilizes the DVD method for handling discrete variables, employs the LHD method for continuous variables, and integrates them with random optimization methods. The proposed ODCD method demonstrates high reliability. Results indicate that, compared to methods involving continuous domain sampling followed by rounding to discrete points, the ODCD method performs better in terms of spatial coverage. However, due to inheriting the characteristics of the CP algorithm, this method tends to require a relatively longer computation time.

NOTES

*Corresponding author.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	Xie, B.C., Zhang, Y. and Xu, Z.Y. (2022) Review on Multidisciplinary Optimization Key Technology of Electrical Machine Based on Surrogate Models. Transactions of China Electrotechnical Society, 20, 5117-5143.
[2]	Wang, W., Wang, D.F. and Liao, X.D. (2016) Multi-Objective Optimization Design of Sub-Frame Using Surrogate Model. Journal of Nanjing University of Technology (Natural), 4, 130-136.
[3]	He, H., Zhu, G.R. and He, C. (2014) Crashworthiness Optimization Based on Kriging Metamodeling. Journal of Nanjing University of Aeronautics & Astronautics, 2, 297-303.
[4]	Yang, J., Liu, Y.Q. and Ai, C.A. (2013) Application of Improved Kriging-Model-Based Optimization Method in Solid Rocket-Ramjet Missile’s Aerodynamic Design. Journal of Solid Rocket Technology, 36, 590-593.
[5]	Xia, L., Wang, D. and Zhang, Y. (2016) Aerodynamic Optimization Method Based on Adaptive Surrogate Model. Acta Aerodynamica Sinica, 34, 433-440.
[6]	Han, Y.Z., Gao, X.S. and Li, L.Z. (2007) Kriging Model-Based Multidisciplinary Design Optimization for Turbine Blade. Journal of Aerospace Power, 22, 1055-1059.
[7]	Yang, J.W., Wu, Z.P. and Wang, W.J. (2021) A Surrogate-Based Optimization Method for Mixed-Variable Aircraft Design. Engineering Optimization, 54, 113-133. https://doi.org/10.1080/0305215X.2020.1855156
[8]	Han, Z.H. (2016) Kriging Surrogate Model and Its Application to Design Optimization: A Review of Recent Progress. Acta Aeronautica et Astronautica Sinica, 37, 3197-3225.
[9]	Kim, D.Y., Nam, J.K. and Jang, G.H. (2013) Reduction of Magnetically Induced Vibration of a Spoke-Type IPM Motor Using Magneto Mechanical Coupled Analysis and Optimization. IEEE Transactions on Magnetics, 49, 5097-5105. https://doi.org/10.1109/TMAG.2013.2255307
[10]	Semon, A., Melcescu, L. and Craiu, O. (2019) Design Optimization of the Rotor of a V-Type Interior Permanent Magnet Synchronous Motor Using Response Surface Methodology. 11th International Symposium on Advanced Topics in Electrical Engineering (ATEE), Bucharest, 28-30 March 2019, 1-4. https://doi.org/10.1109/ATEE.2019.8724856
[11]	He, J., Li, G.L. and Zhou, R. (2020) Optimization of Permanent-Magnet Spherical Motor Based on Taguchi Method. IEEE Transactions on Magnetics, 56, 1-7. https://doi.org/10.1109/TMAG.2019.2947863
[12]	Zhang, J.T., Zang, T.M. and Liang, L. (2011) Nonlinear Modeling and Generalized Predictive Control of Ultrasonic Motor. Electric Machines and Control, 15, 50-56.
[13]	Duan, H.B. and Gan, L. (2015) Orthogonal Multi-Objective Chemical Reaction Optimization Approach for the Brushless DC Motor Design. IEEE Transactions on Magnetics, 51, 1-7. https://doi.org/10.1109/TMAG.2014.2347962
[14]	Park, Y.U., Cho, J.H. and Kim, D.K. (2015) Cogging Torque Reduction of Single-Phase Brushless DC Motor with a Tapered Air-Gap Using Optimizing Notch Size and Position. IEEE Transactions on Industry Applications, 51, 4455-4463. https://doi.org/10.1109/TIA.2015.2453131
[15]	Morris, M.D. and Mitchell, T.J. (1995) Exploratory Designs for Computational Experiments. Journal of Statal Planning & Inference, 43, 381-402. https://doi.org/10.1016/0378-3758(94)00035-T
[16]	Li, C.F., William, W. and Jeff, W. (1997) Column Wise-Pairwise Algorithms with Applications to the Construction of Supersaturated Designs. Technometrics, 39, 171-179. https://doi.org/10.1080/00401706.1997.10485082
[17]	Saab, Y.G. and Rao, V.B. (1991) Combinatorial Optimization by Stochastic Evolution. IEEE Transactions on Computer, 10, 525-535. https://doi.org/10.1109/43.75636
[18]	Jin, R., Chen, W. and Sudjianto, A. (2016) An Efficient Algorithm for Constructing Optimal Design of Computer Experiments. Journal of Statal Planning and Inference, 134, 268-287. https://doi.org/10.1016/j.jspi.2004.02.014
[19]	Zhang, X.L. and Zhang, B. (2010) Multi-Objective Experimentation Design Optimization Based on Modified ESE Algorithms. Systems Engineering and Electronics, 32, 410-414.
[20]	Ye, P.C., Pan, G. and Gao, S. (2019) Sampling Design Method of Fast Optimal Latin Hypercube. Journal of Northwestern Polytechnical University, 37, 714-723. https://doi.org/10.1051/jnwpu/20193740714
[21]	Yang, J.W. (2020) A Novel ISLE-ESE Method for Mixed-Variable Experiment Design. International Conference on Quality, Reliability, Risk, Maintenance, and Safety Engineering, Xi’an, 8-11 October 2020, 22-43.
[22]	Cho, I., Lee, Y. and Ryu, D. (2017) Comparison Study of Sampling Methods for Computer Experiments Using Various Performance Measures. Structural & Multidiplinary Optimization, 55, 221-235. https://doi.org/10.1007/s00158-016-1490-6
[23]	Johnson, M.E., Moore, L.M. and Ylvisaker, D. (1990) Minimax and Maximin Distance Designs. Journal of Statal Planning & Inference, 26, 131-148. https://doi.org/10.1016/0378-3758(90)90122-B