A Kaleodoscopic View of Fuzzy Stochastic Optimization *

The last three decades have witnessed development of optimization under fuzziness and randomness also called Fuzzy Stochastic Optimization. The main objective of this new field is the need for basing many human decisions on information which is both fuzzily imprecise and probabilistically uncertain. Consistency indexes providing a union nexus between possibilities and probabilities of uncertain events exist in the literature. Nevertheless, there are no reliable transformations between them. This calls for new paradigms for coping with mathematical models involving both fuzziness and randomness. Fuzzy Stochastic Optimization (FSO) is an attempt to fulfill this need. In this paper, we present a panoramic view of Fuzzy Stochastic Optimization em-phasizing the methodological aspects. The merits of existing methods are also briefly discussed along with some related theoretical aspects.


Introduction
Mathematical programming is a very useful tool in the hands of a decision maker. As a matter of fact, many decision problems including allocation of resources, transportation, affectation, production, etc. may be cost into an optimization framework. An interested reader may consult the paper by Dan [1], where a complex deterministic optimization problem is discussed. According to Zadeh's incompatibility principle [2], when the complexity of a system increases, our aptitude to make precise statements on it decreases up to a threshold from *Authors express their gratefulness feelings to an anonymous Referee, whose comments helped increase the quality of the paper.
For details on probability of a fuzzy event, the interested reader is invited to consult [9].

Probabilistic Set
Given a set X and a probability space Ω , a probabilistic set A on X is a fuzzy set of X × Ω , whose membership function: For readers interested in more details on uncertain probabilities, we recommend the authoritative scientific papers by Buckley and Eslami [11] [12]. X α α ∈ are random intervals and frvs possesses the Radon-Nikodym property. A considerable body of literature has grown out of the concept of frv, in a wide range of fields (e.g. [13] [14]).

Random Fuzzy Variable
A random fuzzy variable is a map from a possibilistic set to a probabilistic one, verifying some measurability conditions [15]. An interested reader may consult [16] for details on random fuzzy variables.

Other Approaches
Other approaches to blending epistemic and aleatory forms of uncertainty include imprecise probabilities in the style of Peter Walley [17] and p-boxes defined by Ferson [18]. These approaches may be used to enrich models and methods for Fuzzy Stochastic Optimization.

Preamble
Fuzzy Stochastic Optimization (FSO) models are numerous and varied (e.g. [19]). They aim at stretching applicability of Optimization models to situations where both fuzziness and randomness are in the state of affairs. The back bone of FSO models is "the whole man doctrine" that urges us to bring everything we have to bear in our subject. Thwarting, suppressing or down playing uncertainty or imprecision in a mathematical model offers no other chance to that model but to churn out meaningless outcomes. This is in tune with the well-known Computer science rule: "Garbage in, Garbage out".

Types of FSO Models
In this paper, we consider 5 categories of FSO models. This classification is far form being exhaustive. It just reflects the main models encountered in the literature. The classification has been done according to the following criteria: the way fuzziness and randomness enter into the problem, the mathematical formalism used to account of involved hybrid uncertainty (see above), the structure of the optimization problem, the number of objective functions, the number of stages and the numbers of levels. The first class (C 1 ) contains mathematical programming problems under fuzziness and randomness. Models on this class have one objective function and a limited number of constraints. Moreover the decision is taken here and now. For examples of models in category C 1 , we refer the reader to papers [20] [21]. If we consider temporal and level dimensions, then we get: fuzzy stochastic version of Dynamic programming (category C 2 ) and multilevel optimization under randomness and fuzziness (category C 3 ).
Examples of models in Categories C 2 and C 3 may be found in references [22].
Another class of Fuzzy stochastic optimization models that came to the fore during the 1990s is the Fuzzy stochastic robust category (C 4 ). Here, instead of specifying feasibility of an action by a set of inequalities, like in conventional optimization, the feasible region is defined via random set containments. All the above mentioned classes assure that the optimization problem has one objective function. If one considers several conflictual objective functions, one falls on the realm of fuzzy stochastic versions of multi objective programming problems (C 5 ). It is also worth mentioning that problems in classes 1 4 , , C C  may be extended to the multi objective case, leading to 1 4 , , l l C C  (l denoting the number of objectives considered). The reader is referred to [23] for an example of fuzzy stochastic multi objective model.

Characteristics of FSO Models
FSO models have the following common features: 1) Fuzziness and randomness are, in some way, involved in these models.
2) There is no agreed definition of optimum.
3) There are different perspectives about the problem situation. 4) The fuzzy stochastic optimization problem should be approximated by a deterministic one.
5) The closedness of the appropriate deterministic problem to the original one as well as the tractability of the former is desirable traits.

First Generation of FSO Methods
Early attempts to solve decision problems involving fuzziness and randomness dated back to the 1980's [24] [25]. The notion of probabilistic set coupled to Bellman-Zadeh's confluence principle, served as the backdrop to these approaches. This line of research has been quickly extended to optimization problems with discrete random variables and vague relationships [26]. The above mentioned developments on FSO were followed by systematic comparisons between Stochastic Programming and Fuzzy Optimization [27]. These comparative studies displayed many similarities and differences that have been put in good use to deal with some complex stochastic programs through simple and relevant fuzzy optimization techniques [28]. By the same token, a method for incorporating random variables with flexible distributions in a mathematical program was obtained [29]. Research efforts have also been triggered for treating, in a synergetic way, fuzzy and stochastic constraints in a same Optimization problem [30]. Without a shadow of doubt, the concept of fuzzy random variable [31] has served as a catalyst that boosted FSO development. This hybrid tool provided a gold mine of opportunities for coping with situations where fuzziness and randomness co-occur in an optimization setting [32]. In the next section, we briefly survey solution procedures for Fuzzy Stochastic Optimization models mentioned in section 3.

General Procedure for Solving a FSO Model
As suitable models are central to conceptual formulation of FSO problems, so suitable mathematical methods are central to their quantitative treatment. The presence of both possibilistic and probabilistic information within an Optimization framework is a harbinger of computational nightmares if one were to approach the problem without any simplifications. The game is to come up with approximations which tradeoff faithful representation of reality for computational tractability. Although the trajectory of FSO follows a situation-specific approach that do not lead to a generalization, we introduce below a general scheme from which most of existing FSO methods boil down. This algorithmic framework unifies a variety of seemingly different methods that have been derived from disparate approaches.
Before proceeding further, we need the following notations: DO : Transformation that converts the original problem into a form suitable for treatment.
If the problem at hand is ready for treatment, then DO = I (identity transformation).

DF : Deffuzification.
That is a process that associates to a given fuzzy system a deterministic counterpart.

DR : Derandomization.
That is a procedure that immunizes from randomness.
DFR : Transformation that deals simultaneously with fuzziness and randomness.
DL(P) : stands for the problem resulting in applying DL to P, where L may be O, F, R or FR.
We are now in a position to depict a general scheme for solving a FSO problem.
Step 0: make a choice of DO, DF, DR, DFR.
Step 1: Apply 0 T DO = to the original problem (P) to obtain ( ) 0 T P .
Step T T P .
Step 3: Choose 2 T as follows: - -Otherwise take Step 4: Solve the resulting deterministic problem

T Poss Nec
∈ and β is a prescribed threshold.
Poss and Nec stand for possibility and necessity respectively.
α and δ stand for prescribed thresholds.
where E stands for the expectation operator.
where Prob is the probability measure and β a fixed threshold.
For the sake of space, we briefly discuss in the section below, section (5.2.2). Only some approaches for models of different categories.      and " ∧ " stands for an operator used to translate the semantic meaning of the "and" connective. It is worth noticing that

Solving Approaches for Models in the Category
are memberships functions of probabilistic sets representing the constraints: ; 0,1, , .
Solving this optimization problem yields a satisfying solution of Problem (M 1 ). The term symmetrical refers to the fact that the objective function and the constraints are considered as equivalent concepts.
To solve problem ( ) 1 M ′′′ , we need an analytical expression of the distribution of ( ) The following result, the proof of which may be found in [33], is helpful in this regard.
where γ stands for a coefficient of compensation ranging between 0 and 1, then For a perspective on the asymmetrical approaches for solving Problem (M 1 ). We refer the reader to [34]. Asymmetrical approaches have the advantage of confining the objective function to its classical role of ranking alternatives and the constraints to their role of delineating the feasible set. Ongoing research includes the use of other more effective paradigms, like the chance constrained or the multistage ones instead of merely sticking on the expectation operator to immunize from randomness. Another issue researchers are busy considering is the trade-offs between the two classes of approaches. Results along this line would make it possible to decide which approach is most suitable under given circumstances.
(b) Mathematical programming with fuzzy random coefficients Advances on elucidating properties of fuzzy random variables have triggered considerable work on how to deal with mathematical programs with fuzzy random (fr) or random fuzzy (rf) coefficients. It is worth mentioning that the general scheme described in section (5.2.1) applies here as well. Typical, but by no means unique, transformations are used. To provide a taste of what is done on optimization with fuzzy random data, we consider the following Program (M 2 ). We deliberately restrict ourselves to the linear case so that basic ideas are illustrated in a relatively simple context. For non linear programs, we refer the reader to [34] [35].
Consider the following fuzzy stochastic program: All transformations used in this table are well explained in Table 2 except DR3, DR4 and DR5 that are respectively the two stages, the random simplex and the distribution paradigms applied to a stochastic program. For solving resulting deterministic problems, one use techniques from simple ones like the simple method [40] to sophisticated ones like hybrid intelligent algorithms [41] via metaheuristics, like Genetic algorithms [42] and Tabu search [43]. A quick look at Table 2 may convey the wrong impression that methods listed on that table do not offer much scope for theoretical results. Upon closer examination one may realize that these approaches are not pedestrian or brute force methods. They rely on deep theoretical insights. We mention below some of them.
It is well known that constraints on probability induce non convexities which preclude application of powerful convex optimization techniques. So the viability of approach 1 in Table 2 depends on whether it is possible to convert constraints Table 2. Some instances of approaches for solving (2).

Transformation
Resulting deterministic problem ( P′ ) Approach references Find the fuzzy probability distribution of W and its expectation given respectively by: Use the random simplex method to Y. T. Mangongo et al.
on probability on a feasible convex set. Moreover, approach 2 relies on the following result.
Theorem 2. Consider the Problem (P) and all related notation below.
( ) where , n c x∈  and * means that ij a and i b are either fuzzy quantities or random variables, " * " takes "+" if the data is random and it takes "−" if the data is fuzzy; then the distribution of H α * and its expectation are respectively: For the proof of this result, we invite the reader to consult [33]. We must also stress the fact that several methods have been developed over the past few decades due to the efforts of many researches, e.g. [44] [45]. Surveying all these developments is however beyond the scope of this paper.
Important part of research in this realm includes the generalization of existing ideas by using other transformations and the driving up to the performance analysis in the side of effectiveness rather than sticking on computational efficiency. Something that should also be pursued with focused attention is the search for solutions of fuzzy stochastic nature, in a way to reflect the fuzzy random structure of the problem.
The case where one has both fuzzy and random data in a same optimization problem, can be converted into the case one has fuzzy random data, by considering involved fuzzy and random quantities as degenerate fuzzy random variables (e.g. [46]). Another notion that has energized FSO development is that of random fuzzy variable [47]. This notion that has been moulded by Liu [48], generalizes quite naturally the notion of random variable. A fascinating approach based on this notion and fuzzy random simulation that integrates neural network and genetic algorithm has been pushed forward in [49], to solve a mathematical program with random fuzzy parameters.
In addition to shedding light on analysis of Optimization problems under hybrid uncertainty, the proposed algorithm may be the only one resort for solving optimization problems under randomness and fuzziness, where random fuzzy variable is the only one format available for data. It is our belief that exploring the duality between a mathematical program with fuzzy random data and its counterpart with related random fuzzy parameters, will help fostering a more deep understanding of characteristics of Fuzzy Stochastic Optimization problems.
(c) Mathematical program with random variables having fuzzy parameters We now move to solution procedures for optimization problems, where another specific tool, namely uncertain probability, is used for modeling hybrid imprecision in the data of the problem. The mathematical program on which ideas will be illustrated is as follows.
where " * " means the daturn is random with some fuzzy parameters. This model has many applications in situations where experts who provide data of the optimization problem feel more comfortable in coupling their vague perception with hard statistical data. To this end, they may prefer to represent these data in the form of random variables with vague parameters. As an example, consider a portfolio selection problem where, due to stock expects' judgments and investors' different opinions, the security returns are modeled as random variables with fuzzy parameters. Model (M 3 ) has not received much attention in the literature. The art here is to find an approximation scheme simple enough to be computable, but not so simple that useful detail is lost. This is done in [50] by making use of the fuzzy version of the well known chance constrained programming approach. Uncertain probabilities of constraints are enforced to be larger than a specified level fixed by the Decision maker.
This means one has to consider the following problem.
( ) where P  stands for uncertain probability and , 1, , i i m δ =   are fuzzy thresholds fixed by the Decision maker. The question that immediately arises is that of converting Problem ( ) 3 M ′ into a standard mathematical program.
Fascinating results in connection with this issue have been obtained in the literature. For the sake of space, we only give below one of these results. To this end, we need the following lemma. .
where φ stands for the density function of the standard normal distribution.
It is worth noting that Problem ( ) 3 M ′′ has infinitely many constraints and it can be handled by semi-infinite mathematical programming techniques [49]. The prevailing research directions in this setting include analysis of non linear variants of Problem (M 3 ) along with methods for solving them and consideration of other distribution than the normal one. Another worthy avenue for research in this framework, is to carry out case studies in a way to validate existing approaches.

A method for Solving a Model in the Category C 4
One key ingredient for dealing with models in the category C 4 is the lattice isomorphism between the set of fuzzy random sets and the family of their α -level sets [51]. It makes a lot of sense to take a defuzification approach, the transformation that immunizes the original robust fuzzy stochastic program from fuzziness by making use of the above mentioned isomorphism and by exploring the fact that α -level sets of a fuzzy random set are random intervals. A suitable derandomization procedure would then be a stochastic programming technique like 1 2 , DR DR or 3 DR (see Table 1).
Here is an example of a fuzzy stochastic robust programming model.
To tackle randomness surrounding B , we resort to chance constrained programming approach. This approach requires that the constraints need to be satisfied with some higher probability. For instance, for a stochastic constraint: where l is a fixed level of probability. Constraint (2) can be merely written: being the cumulative distribution of e. For details on Chance Constrained programming, we refer the reader to [52]. It is also well known that α -levels of ij a and i b are real intervals. We denote them by P′′′ is a standard linear program, that can be solved by existing linear programming software.

A method for Solving a Model in the Category C 2
To illustrate how a Fuzzy Stochastic dynamic model can be solved, we briefly discuss a multistage decision process, involving randomness and fuzziness. The state space of the process at hand is We assume that the termination set, denoted by T, consists of states ( ) x + is a random variable characterized by the conditional probability function ( ) Given a policy we consider the following vectors: Let' also introduced the following transition matrices: with these notations, the system of equations in (3) can be put in a more compact form.
Π is said to be proper, if there is a natural number K such that The literature on multiperiod programming under fuzziness and randomness is rather limited. This could be due to the lack of tractable methodologies. The optimality principle, so profusely used in deterministic dynamic programming, does not lend itself better in a fuzzy stochastic environment. Nevertheless attempt to integrate fuzziness and randomness in a dynamic programming framework is found in [54]. In that paper, authors described an approach for solving a dynamic programming involving stochastic state transition, fuzzy constraints, fuzzy goal and an implicitely defined termination time. This approach relies heavily on the Bellman-Zadeh's confluence principle [37] [55]. Some interesting results on the robustness of solutions obtained by this way are also outlined. Another approach which deserves more emphasis has been presented by Ganji et al. [56]. Here the picture is made even more complicated by the presence of different decision-making mind-sets. More recently, a new approach named stochastic fuzzy neural network [57] has been introduced. It aims at overcoming some of the limitations of the traditional methods like the difficulties related to the curse of dimensionality due to the increase in the number of decision and state variables. This approach consists of training stochastically a neuro fuzzy system, in a way to represent the system operational strategy.
The prevailing research direction in the Fuzzy Stochastic dynamic programming framework is the improvement of this Neural network approach in a way to address drawbacks of existing models like the impossibility of considering simultaneously related fuzzy and stochastic characteristics.

Algorithms for Dealing with Models in Category C 3
Hierarchical decision making process is extremely practical to such decentralized systems as agriculture, government policy, economic systems, finance, etc. For more realism on their multilevel decision models, researches have investigated ways for incorporating fuzziness into these settings. Related solution concepts like Stackelberg strategy have been elaborated and interesting results have been obtained. Powerful algorithms for singling out desired solutions have also been pushed forward. These approaches are at the root of extensions to Fuzzy stochastic multilevel methodologies. In a method that integrate fuzzy random simulation, neural network training and Genetic algorithm has been developed to find a satisfying solution of a fuzzy random multilevel program. Another interesting approach for the two-level case is found. Here essential ingredients used are fuzzy random versions of the Nash and the Stackelberg-Nash equilibria. The former is used for the lower level problem, while the latter is used for the overall problem. These approaches give rise to a number of challenging issues including effectiveness and convergence of proposed methods. Currently researchers are busy pondering these thorny issues.
In a way to illustrate how a multilevel optimization problem under randomness and fuzziness can be solved, we briefly discuss below an approach proposed by Sakawa, Nishizaki and Katagiri [58] for a two-level linear programming problem.
To illustrate how algorithms for dealing with Fuzzy stochastic multilevel programs work, consider the following optimization problem:

Minimize for
Minimize for subject to : ; 0, where i DM stands for Decision maker i,  . Using Zadeh extension principle; we have that: To handle imprecision inherent to objective functions, we consider fuzzy goals, say the objective value should be substantially less than some given value.
are target values and , 1, 2 l l θ = are permissible probability levels, specified by the decision makers. Before discussing how to find a satisfying solution of the original multilevel program using the auxiliary problem ( ML P′′ ), we need the following notations. We assume that the random parameter associated to the fuzzy random variable l C  is denoted by l d and is expressed as where l t is a random variable with probability distribution l T . Moreover, we denote by F * the pseudo inverse of a real-valued function F of  . That is for θ ∈  , . It is shown that Problem ( ML P′′ ) is equivalent to the following program: Maximize min , subject to It is an easy matter to see that the above program is equivalent to: A part from this level-set based fractile approach for a multilevel optimization problem under fuzziness and randomness, there is also a possibility-based probability approach. 5 Significant progress has been made on ways for dealing with situations in which randomness and fuzziness are inextricably involved in a multi objective programming framework. These advances have generated more interest for this class of optimization problems. In a theoretical ground several concepts of generalized Pareto optimality have been introduced and properly characterized (see [59] [60]). As far as algorithms are concerned, powerful methods based on tools ranging from simple optimization algorithms to sophisticated ones (like hybrid intelligent algorithm [29]) via meetaheuristics [61] have been devised. The most common methods include:

Methods for Solving Models in the Category C
• methods for generating Pareto optimal solutions, • interactive methods, • goal programming methods.
As an example of a method for solving a fuzzy stochastic multi objective program. Consider the following multi objective programming problem: One may use the expectation operator to convert the original problem into deterministic terms. In this case, one maximizes the expected degree of possibility that each of the original functions involving fuzzy random coefficients attains the fuzzy goals. In this case, the resulting deterministic problem is:

Fuzzy Stochastic Optimization Applications
Fuzzy stochastic optimization is currently applied in a wide variety of domains spanning engineering Economy, Finance, Ecology, Energy, etc. The relevance of FSO was recognized once it became clear that it can be used as a unifying framework for handling situations where fuzziness and randomness co-occur in an optimization setting.
To assert that it is more useful to conceive imprecision in an optimization setting as a variegated whole is not to minimize important research works that have done in Fuzzy Optimization and Stochastic Programming. It is instead to assert that new perspectives for coping with complex real life problems may be gained by integration of both approaches than exclusion. It is virtually impossible to encompass the full spectrum of FSO applications within the scope of a paper. In Table 3, we list some examples of FSO applications in the above mentioned domains.

Concluding Remarks and Suggestions for Future Research
Fuzzy stochastic system is a fascinating area that allows generating satisfying outputs from fuzzy random inputs subject to a given system restriction. Investigating such systems represents an important research avenue in the discipline of mathematical modeling under uncertainty. Apart from being helpful in several applications, fuzzy stochastic systems can help shed some light on intricate Table 3. Some applications of FSO.

Domain
Example of applications Engineering Planning of solid waste management [62] Economy Marketing [63] Fiance Bank hedging decision problems [63] Ecology Environmental system planning [64] Energy Electric power planning [65] Water resources Reservoir strategies [66] Reliability Time-dependent reliability [67] Network Shortest path problem [68] Renewal processes Queuing applications [69] theoretical issues like impacts of infinitesimal changes on inputs of a mathematical model under uncertainty. When systems under scrutiny boil down to mathematical programs, we fall into the realm of fuzzy stochastic optimization, the subject matter of this paper. Fuzzy Stochastic Optimization has triggered, during the last two decades, an explosion of research worldwide both on theoretical issues, algorithmic aspects and concrete applications. It is therefore worth-while to pause once a while and ask ourselves, what we have done, what is done currently and what are the perspectives in this field. This paper has been written in this spirit. Its coverage has been made with a broad target audience in view. We have highlighted the modeling power of FSO, its broad applicability as well as significant advances made in terms of both the quantity and the quality of results obtained. The variety of related subjects and the current profusion of scientific publications in the field, reflect its vitality and diversity. Some lines for further developments on this fascinating field are briefly outlined below: 1) Despite the fact that most existing approaches for solving FSO problems bow to epistemologically based prescriptive procedures for dealing with problems involving uncertainty, the model validation has been treated mostly from the perspective of efficiency (doing things right). Efforts should be devoted to strike a good balance between efficiency and effectiveness of designed FSO methods. Some indexes, including poss-prod and prod-poss that serve as windows through which one can appraise the desirability of a given action under considered transformations, have been introduced, but more is needed.
2) Decision makers would perhaps be best served if from a deep comparative analysis of good FSO methods, user-friendly Decision Support Systems (DSSs) could be pushed forward. Such (DSSs) would be of great help advise Decision makers with the choice of the most suitable methods for their problems.
3) In some sensitive cases, it would be preferable to have as a solution to a FSO problem, a fuzzy stochastic quantity in a way to reflect the vagueness and the randomness surrounding the problem. So an interesting direction for future research is that of defining and characterizing fuzzy random solutions for a Fuzzy Stochastic Optimization problem. Algorithms for singling out such solutions should also be designed and implemented. 4) Description of high quality case studies is also needed, in order to have more solid arguments in favour of the underniable importance of FSO methods.
We very much hope to see FSO advances so far, so fast that its influence transforms both the theory and practice of mathematical programming under uncertainty.

Conflicts of Interest
The authors declare no conflicts of interest regarding the publication of this paper.