Application of Stochastic Control Technique in Production and Inventory Model for a Fixed or Constant Demand Rate

Nathaniel Kayode Oladejo

Department of Industrial Mathematics, School of Mathematical Sciences, C.K. Tedam University of Technology and Applied Sciences, Navrongo, Ghana.

DOI: 10.4236/ojop.2025.142003                

Abstract

This paper deal with the application of stochastic optimal control technique in production and inventory model for a fixed or constant demand rate where the stochastic differential equations are considered as ordinary differential equations driven by white noise justified by the connection of the Ito’s integral and white noise in the case of non-random integrands interpreted as cost functional. Numerical evaluation of such optimal stochastic control theory in production and inventory model of fixed or constant demand rates with its sensitivity effects were illustrated.

Keywords

Optimal, Stochastic, Control, Production, Inventory, Technique

1. Introduction

The impact and importance of optimal stochastic and dynamic control techniques cannot be underrated as it is well applicable in engineering, management, mathematics and economics to examine and evaluate optimal industrial processes such that the cost of production is determined and ascertain maximum production at the minimum and bearable cost as many questions of stochastic dynamic models and optimal control arise in engineering, management and economics. For example, it may be required to calculate the conditions of operation of an industrial process that gives the maximum output or quality or that gives the minimum cost. Investment and production planning models have a relatively long tradition in dynamic optimization theory [1]. Proposed the first major contribution in this field by applying the calculus of variation principle to solve a production investment model [2]. Provide a nice introduction to the applications of optimal control theory to investment and production models. A very good introduction to the dynamics of economics and management models, as well as their stochastic version, can be found in the books of [3] and [4] [5].

Examine optimizing production and inventory model for time varying demand rate using stochastic control technique emphasising the increasing role of stochastic processes and its wide applications in sciences, engineering, business management, military and space technology. They consider the numerical evaluation of an optimal stochastic control theory in production and inventory model of general time demand rates and examine its sensitivity effects [6].

Characterized the optimal inter-temporal price and production decisions depending on the sum of the adjoint variable of the inventory level and the Lagrange multiplier of the state constraint by using the optimal control theory. Production has attracted growing attention due to its increasing importance in today’s highly competitive environment, as propounded by [7] and [8].

The last three decades have seen a growing number of optimal control theory and applications in the field of investment as [9] considered the optimal control of the Vidale-Wolfe investment model. In his study, the optimal control is the rate of investment expenditure to achieve a terminal market share within specified limits in a way which maximizes the current value of net profit streams over a finite time horizon [10]. Considered the problem of optimal investment for the Nerlove-Arrow model under a replenish-able budget where an optimal control problem with two state variables for the dynamics of the model and the optimal control is the rate of investment expenditure that is required to maximize the present value of net streams over an infinite time horizon subject to a replenish-able budget [11].

Applied optimal investment model under a replenish-able budget with two state variables to maximize the present value of net streams over an infinite time horizon subject to a replenish-able budget while [12] Applied Hamilton–Jacobi–Bellman principle to determine the expected output rate with the expected inventory level having special cases for specific parameter values.

[13] examined the mathematical model of total production cost for a deteriorating production process by minimized production cost and maximized the profit and [14] examined the optimal control of production-inventory systems with correlated demand inter-arrival and processing times to minimize the expected average cost of production.

2. Stochastic Differential Equations

Here, we let our controlled stochastic differential equation (SDE) be of the form:

$\stackrel{•}{X}\left(t\right)=f\left(t,x\left(t\right)\right)dt+\sigma \left(t,\left(x\left(t\right)\right)\right)dw\left(t\right)$ (1)

With initial condition$x_{(0)}$ , this leads to integral equation of the form:

$x\left(t\right)=x_{\left(0\right)}+{\displaystyle {\int }_0^{t}f\left(s,x\left(s\right)\right)ds+{\displaystyle {\int }_0^t\sigma \left(s,x\left(s\right)\right)dw\left(s\right).}}$ (2)

The solution $x(t)$ of the equation (2) possess the differential equation of the form:

$\stackrel{•}{X}\left(t\right)=\overline{f}\left(t\right)dt+\overline{\sigma }\left(t\right)dw\left(t\right)$ (3)

Where

$\overline{f}=f\left(t,x\left(t\right)\right)$ and $\overline{\sigma }\left(t\right)=\sigma \left(t,x\left(t\right)\right)$ (4)

But,

${\displaystyle {\int }_0^t\overline{f}\left(s\right)ds<\infty }$ , and ${\displaystyle {\int }_0^t{\overline{\sigma }}^2\left(s\right)ds<\infty }$ (5)

It is remarkable to note that equation (1) can be written as

$dx\left(t\right)=f\left(t,x\right)+\sigma \left(t,x\right)\psi$ (6)

Where $\psi$ represent the white noise.

Equation (1) can be interpreted as a scalar equation or as a vector equation.

$x,f$ Represent $n-$ vectors,$w$ represent $r-$ vector.

Where$r$ , is the Wiener process serves as component and$\sigma$ represent $n\times r$ matrix.

By linearity, the scalar equation (1) can be read as:

$\stackrel{•}{X}=\left(\alpha +\beta x\right)dt+\left(\lambda +\delta x\right)dw.$ (7)

Where $\alpha ,\beta ,\lambda ,$ and $\delta$ are constants or time-dependent scalar.

Equation (7) can be replaced in the linear case by

$\stackrel{•}{X}=\left(a+Ax\right)dt+{\displaystyle \sum _{i=1}^r\left(b_i\text{'}+B_{i}x\right)d{w}_i}$ (8)

Where $a,b_i$ are vectors and $A,B_i$ represent matrices.

2.1. Stochastic Optimal Control Theory

For any given system represented by a differential equation of the form;

$dx=f\left(t,x_t,u_t,z_t\right)=f\left(x_t,u_t,t\right)+Gdz\left(t\right).$ (9)

An optimal control problem, is specified by giving a performance criterion that grades the possible control function $u$ in order of preference by attaching a number $J\left(u\right)$ to anyone.

$J\left(u\right)$ is refers to and called the cost of $u$ ,so that we can choose the control that minimizes it. If $J\left(u\right)$ represents the profit then we can maximize it by minimizing$-Ju$ , meanwhile the distinction between the minimizing and maximizing is purely notational.

In the optimal control theory, the type of cost function considered is almost invariably written as:

$J\left(u\right)=E\left\{{\displaystyle {\int }_0^{T}H\left(t,x_t,u_t\right)dt+G\left(x_t\right)}\right\}.$ (10)

Where $T$ is infinite termination time and not fixed.

In investments problems where $x_t$ is the value of one’s asset.$H$ represents the consumption rate, but in engineering perspective, $H$ is usually chosen to be the cost deviation from some desired trajectory of $x_t$ or the use of too much control forces or energy.$G\left(x_t\right)$ is the cost failure to reach some special target set at terminal.

Many deterministic optimal control problems can be formulated in other to have a cost

${\displaystyle {\int }_0^{T}H\left(t,x_t,u_t,z_t\right)}dt+G\left(x_t\right).$ (11)

Solving the stochastic optimal control problems defined in the equations (10) and (11).

We let $V\left(x,t\right)$ refers to as the current value function be the expected value of the objective function of the equation (11) from $t$ to $T:$

$V\left(x,t\right)=\underset{u}{\max }E\left\{{\displaystyle {\int }_t^{T}H\left(x,u,z,t\right)dt+G\left(x_t\right)}\right\}.$ (12)

When an optimal policy is followed from $t$ to$T$ ,

Given $X_t=x.$ then by the optimality principle,

$V\left(x,t\right)=\underset{u}{\max }E\left[H\left(x,u,z,t\right)dt+V\left(x+d{x}_t,t+dt\right)\right].$ (13)

Applying Taylor’s expansion we get;

$V\left(x+d{x}_t,t+dt\right)=V\left(x,t\right)+V_{t}dt+V_{x}d{x}_t+\frac{1}{2}{V}_{xx}{\left(d{x}_t\right)}^2+\frac{1}{2}{V}_{tt}{\left(dt\right)}^2+\frac{1}{2}{V}_{xt}d{x}_{t}dt+.$ (14)

Using the equation (10) we write formally thus:

${\left(\stackrel{•}{X}(t)\right)}^2=f^2{\left(dt\right)}^2+G^2{\left(d{z}_t\right)}^2+2fGd{z}_{t}dt.$ (15)

$dx-tdt=f{\left(dt\right)}^2+Gd{z}_{t}dt\mathrm{..}$ (16)

Here, it is sufficient to know the multiplication rules of the stochastic calculus, thus;

${\left(d{z}_t\right)}^2=dt,d{z}_{t}dt=0,{\left(dt\right)}^2=0.$ (17)

Substitute from the equation (14) into the equation (13), apply the equations (10), (11) and equation (12) we get:

$V\left(x,t\right)=\underset{u}{\max }\left[Hdt+V\left(x,t\right)+V_{t}dt+V_{x}fd{X}_t+\frac{1}{2}{V}_{xx}{G}^{2}dt+0\left(dt\right)\right].$ (18)

Conceding the term $V\left(x,t\right)$ on both sides of the equation (18), dividing the remainder by $dt$ .

Given that $dt\to 0$ , then following the Hamilton-Jacob-Bellman equation can be derived

$0=\underset{u}{\max }\left[H+\frac{\partial V}{\partial t}+\frac{\partial V}{\partial x}f+\frac{1}{2}{G}^2\frac{{\partial }^{2}V}{\partial x^2}\right].$ (19)

For the current value function$V(x,t)$ , with the boundary condition

$V\left(x,T\right)=S\left(x,T\right)$ (20)

2.2. Optimal Stochastic Consumption Model

Here we formulate the deterministic consumption model. We assume that $w\left(t\right)$ denotes the wealth at time$t$ , which represents the state variable, and $C\left(t\right)$ is the rate of consumption, which represents the control variable at time$t$ .

From the differential equation of the form:

$dw\left(t\right)=rw\left(t\right)-C\left(t\right),w\left(0\right)=w_0$ (21)

Where $r$ is the compounded rate.

We formulate the optimal control problem of the form:

$\max \left\{J={\displaystyle {\int }_0^T{e}^{-\rho t}In\left[C\left(t\right)\right]dt+e^{-\rho T}BW\left(T\right)}\right\}$ (22)

Subject to:

$dw=rw-C,w\left(0\right)=w_0,w\left(T\right)\ge 0$ (23)

Where $T$ is the time,$B$ is constant,$W\left(T\right)$ denote the wealth.

We let $R_0$ be the initial price of an investment, $r$ the interest rate and $R_t$ be an accumulated changing rate at time $t$ .

Then,

$d{R}_t=\frac{d{R}_t}{dt}=r{R}_t,R_0=R\left(0\right)$ (24)

By applying separation of variable method, the equation yields;

$d{R}_t=r{R}_{t}dt,$ $R_0=R\left(0\right)$ (25)

Solving, we get the accumulated amount as a fraction of time;

Thus; $R_t=R_0{e}^{rt}$

According to Merton (1971) and Black-Scholes (1973) the stock price $p_t$ can be formulated by Ito’ stochastic differential equation:

$d{p}_t=\alpha p_{t}dt+\sigma p_{t}d{z}_t$ $p_0=p\left(0\right)$ (26)

Where $\alpha$ represents the expected value of the return rate on stock,${\sigma }^2$ is the variance associated with the return and $z_t$ represents a standard wiener process.

3. Parameters for Production and Inventory Model in a Fixed or Constant Demand Rate

We consider and adopt the following parameters to formulate the stochastic optimal control model for a constant or fixed demand rate.

S/N	Parameter	Interpretation
1	$Y_t$	Wealth at time t
2	$K_t$	Consumption rate at time t
3	$Z_t$	Fraction of the wealth invested in stock at time t
4	$1-Z_t$	Fraction of the wealth in saving account at time t
5	$V_c$	Utility of consumption at consumption rate c
7	$\rho$	Discount rate applied to consumption utility
8	$B$	Bankruptcy parameter

Then we develop the dynamics of the consumption model by considering the wealth equation of the form:

$d{Y}_t=\left[\left(\alpha -r\right)Z_t{Y}_t+\left(r{Y}_t-K_t\right)\right]dt+\sigma Z_t{Y}_{t}d{z}_t$ $Y_0=Y\left(0\right)$ (27)

Where

$\alpha Z_t{Y}_{t}dt$ is the expected return from the risky investment$Z_t{Y}_t$ at time $t$ to $t+dt$ .

$\sigma Z_t{Y}_{t}dt$ is the risk involved in investing $Z_t{Y}_t$ in the stock.

$r\left(1-Z_t\right)Y_{t}dt$ is the amount of interest earned on the balance of $\left(1-Z_t\right)Y_t$ .

$K_{t}dt$ is the amount of consumption during the time interval from $t$ to $t+dt$ .

Solving equation (27) shows that one can trade continuously in time without incurring any broker’s commission and the changes $d{Y}_t$ in wealth from$t$ to $t+dt$ is due to the gain from changes in share and to consumption.

We then formulate the problem of optimal control of stochastic consumption model.

$\underset{K_t>0}{\max }\left\{J=E\left[{\displaystyle {\int }_0^T{e}^{-\rho t}V\left(K_t\right)dt+B{e}^{-\rho T}}\right]\right\}$ (28)

Subject to;

$d{Y}_t=\left[\left(\alpha -r\right)Z_t{Y}_t+\left(r{Y}_t-K_t\right)\right]dt+\sigma Z_t{Y}_{t}d{z}_t$ , $Y_0=Y\left(0\right)$ ,$K_t>0$ (29)

The bankruptcy parameter $B$ can be positive if there is a social welfare system in place and can also be negative if there is a remorse associated with bankruptcy.

We let $V\left(x\right)$ be the value function associated with an optimal policy with $Y_t=x$ at time $t$ .

Then, the Hamilton-Jacobi-Bellman equation satisfied by the value function$V\left(x\right)$ is of the form:

$\rho V\left(x\right)=\underset{K_t\ge 0}{\max }\left\{\left(\alpha -r\right)qx\frac{dV}{dx}+\left(rx-C\right)\frac{dV}{dx}+\frac{1}{2}{q}^2{\sigma }^2{x}^2\frac{d^{2}V}{d{x}^2}+V\left(C\right)\right\},V\left(0\right)=B.$ (30)

We then assumed that

$V\left(C\right)=\sqrt{C}$ (31)

$\frac{dV}{dC}\left|C=0=\frac{1}{2\sqrt{C}}\right|C=0=\infty$ (32)

Differentiating equation (30) with respect to $Q$ and $C$ , equating it to zero yield:

$\begin{array}{l}\left(\alpha -r\right)x\frac{dV}{dx}+Z{\sigma }^2{x}^2\frac{d^{2}V}{d{x}^2}=0\\ 1-c\frac{dV}{dx}=0\end{array}\}$ (33)

Solving equation (33) with respect to the function$Z(x)$ and$K(x)$

$Z\left(x\right)=-\left(\frac{r-\alpha }{{\sigma }^{2}x}\right)\left(\frac{dV}{dx}\right){\left(\frac{d^{2}V}{d{x}^2}\right)}^{-1}$ (34)

And

$K\left(x\right)={\left(\frac{dV}{dx}\right)}^{-1}$ (35)

Substituting equations (35) and(34) into the equation(30) yields;

$\left[rx\frac{dV}{dx}-In\left(\frac{dV}{dx}\right)\right]\frac{d^{2}V}{d{x}^2}-\lambda {\left(\frac{dV}{dx}\right)}^2-\rho V=0$ (36)

Where

$\lambda =\frac{{\left(\alpha -r\right)}^2}{2{\sigma }^2}$

3.1. Solution to Stochastic Consumption Model

The nonlinear ordinary differential equation (18) was solved and it take the form:

$V\left(x\right)=aIn\left(bx\right)+D$ (37)

Where $a,D$ are constant and determined by:

$\frac{dV}{dx}=\frac{1}{ax},\frac{d^{2}V}{d{x}^2}=-\frac{1}{a{x}^2}$ (38)

Find the constants $a,b,D$ by substituting equations (38) and (37) into the equation (36).

Thus;

$a=\frac{1}{\rho },b=\rho ,D=\frac{r-\rho +\lambda }{{\rho }^2}$ (39)

And the solution to the equation (65) is given as:

$V\left(x\right)=\frac{1}{\rho }In\rho x+\frac{r-\rho +\lambda }{{\rho }^2}$ (40)

Substituting equation (40) into the equation (34) then, the wealth invested in the stock is;

$Z=\frac{\alpha -r}{{\sigma }^2}$ (41)

Then, the optimal consumption rate $Vc$ is;

$V=\rho x$ (42)

3.2. Numerical Example and Illustration

Here we find the optimal consumption rate and the fraction of the wealth invested in the stock in a fixed or constant demand (Tables 1-2 and Figures 1-2).

Stochastic consumption model

Table 1. shows the utility against consumption rate $Vc=\sqrt{V}$ .

0	0
100	10

Continued

200	14.142
300	17.321
400	20
500	22.361
600	24.495
700	26.458

Figure 1. shows the utility against consumption rate.

Table 2. shows the expected optimal inventory level against time (t).

T	Ed(x)	Ee(x)	Ef(x)
0	10000	10000	2250
3	5600.15	6479.1	1589.83
6	5072.07	6057.5	1510.78
9	5008.65	6006.92	1501.299
12	5001.20	6000.89	1500.199
15	5001.40	6000.649	1500.389
18	5010.75	6004.593	1503.094
20	5044.20	6018.861	1512.729

Figure 2. Shows the expected optimal inventory level E(x) against time. (t)

This indicates that as the time increases, then, the expected optimal inventory level decreases but slightly increases and is normalized as the time keeps on increasing.

4. Sensitivity Analysis

Here we evaluate the current values function of the system parameters ($\lambda ,r$ ) in order to determine its’ sensitivity against rate level (r) (Figures 3-5).

4.1. Expected Current Values Against Rate Level in a Fixed Demand

Illustration 1.

Given that $\rho =75,\lambda =100,x=50$ .

Then Table 3 below shows the expected current values against a fixed demand rate.

Table 3. Expected current values against a fixed demand rate.

R	Va[x]	Vb[x]	Vc[x]
0	0.08267	0.30145	0.080262
10	0.08367	0.31745	0.080956
20	0.08467	0.33345	0.081656
30	0.08567	0.34945	0.082346
40	0.08667	0.36545	0.083039
50	0.08767	0.38145	0.083734
60	0.08867	0.39745	0.084429

Figure 3. Shows the expected optimal inventory level against time.

4.2. Current Value Against Discount Rate

Illustration 2.

Table 4 shows the expected current values against the discount rate.

Table 4. Expected current values against discount rate.

γ	Va[x]	Vb[x]	Vc[x]
0	0.460517	0.405045	0.080171
10	0.560517	0.430045	0.081171

Continued

20	0.660517	0.455045	0.082171
30	0.760517	0.480045	0.083171
40	0.860517	0.505045	0.084171
50	0.960517	0.530045	0.085171

Figure 4. Shows the expected optimal inventory level against time.

4.3. Current Level against Fixed Demand Rate

Illustration 3. Given that $\rho =10,\lambda =10,x=10$ .

Table 5 shows the expected current values against fixed demand rate.

Table 5. Expected current values against.

ρ	Va[x]	Vb[x]	Vc[x]
1	21.30258	66.9957	58.6052
2	5.997866	17.5944	15.8992
3	3.02262	8.2537	7.679
4	1.92221	4.908	4.6854
5	1.3824	3.321	3.2429

Figure 5. Shows the expected optimal inventory level against time with respect to fixed demand rate.

5. Discussion and Interpretation

Table 1 and Figure 1 above show that at a fixed demand rate as time increases, then the utility consumption at the expected optimal inventory level against time increases. Also, in Table 2 and Figure 2 as the time increases, then, the expected optimal inventory level decreases but slightly increases and normalized as the time keep on increasing at a fixed demand rate.

The sensitivity analysis of the model was conducted by investigating the effect of changes in the system parameters on the expected optimal cost and the expected inventory level value of the system parameters on the expected current values against rate level in a fixed demand, current value against constant demand rate and shows that the expected optimal inventory level increases. While the current level against fixed demand rate in Table 4 and Figure 4 shows that as the expected optimal inventory level decreases then, the expected optimal inventory value decreases at a constant or fixed demand rate.

6. Conclusion

Optimal stochastic control techniques in production and inventory model in time varying demand rates have been successfully examined and evaluated. Also, its sensitivity effects on various parameters mentioned in the model were examined and it was discovered that the inventory level and consumption are clearly stochastic in nature at vary demand rate.

Conflicts of Interest

The author declares no conflicts of interest regarding the publication of this paper.

References

[1]	Sethi, S.P. and Lee, S.C. (1981) Optimal Advertising for the Nerlove‐Arrow Model under a Replenishable Budget. Optimal Control Applications and Methods, 2, 165-173. https://doi.org/10.1002/oca.4660020205
[2]	Merton, R.C. (1969) Lifetime Portfolio Selection under Uncertainty: The Continuous-Time Case. The Review of Economics and Statistics, 51, 247-257. https://doi.org/10.2307/1926560
[3]	Merton, R.C. (1971) Optimum Consumption and Portfolio Rules in a Continuous-Time Model. Journal of Economic Theory, 3, 373-413. https://doi.org/10.1016/0022-0531(71)90038-x
[4]	Derzko, N.A. and Sethi, S.P. (1981) Optimal Exploration and Consumption of a Natural Resource: Stochastic Case. International Journal of Policy Analysis, 53, 185-200.
[5]	Oladejo, N.K., Agana, F. and Ogunlaran, O.M. (2024) Optimizing Production and Inventory Model for Time Varying Demand Rate Using Stochastic Control Technique. Far East Journal of Applied Mathematics, 117, 115-132. https://doi.org/10.17654/0972096024006
[6]	Dockner, E.J. and Feichtingor, G. (1993) Cyclical Consumption Patterns and Rational Addiction. Journal of American Economic Review, 83, 256-263.
[7]	Novak, A. and Feichtinger, G. (1992) Optimal Consumption, Training, Working Time, and Leisure over the Life Cycle. Journal of Optimization Theory and Applications, 75, 369-388. https://doi.org/10.1007/bf00941474
[8]	Sethi, S.P. and Thompson, G.L. (2000) Optimal Control Theory: Applications to Management Science and Economics. 2nd Edition, Kluwer Academic Publishers.
[9]	Pontryagain (1974) Necessary Conditions for Optimality for the Optimal Control Problem. Journal of Economics and Statistics, 7, 136-156.
[10]	El-Gohary, A. (2006) Chaos and Optimal Control of Cancer Self Remission and Tumor System Steady-States, A Stochastic Approach. Journal of Chaos, Solutions and Fractals, 9, 817-822.
[11]	El-Gohary, A. (2001) Optimal Control of the Genital Herpes Epidemic. Chaos, Solitons & Fractals, 12, 1817-1822. https://doi.org/10.1016/s0960-0779(00)00012-6
[12]	Alshamrani, A.M. (2013) Optimal Control of a Stochastic Production-Inventory Model with Deteriorating Items. Journal of King Saud University-Science, 25, 7-13. https://doi.org/10.1016/j.jksus.2012.01.004
[13]	Hossein, M., et al. (2016) Project Portfolio Selection via Harmony Search Algorithm and Modern Portfolio Theory. Procedia-Social and Behavioral Sciences, 226, 51-58.
[14]	Manafzadeh Dizbin, N. and Tan, B. (2020) Optimal Control of Production-Inventory Systems with Correlated Demand Inter-Arrival and Processing Times. International Journal of Production Economics, 228, Article 107692. https://doi.org/10.1016/j.ijpe.2020.107692