A Stochastic Optimal Control Theory to Model Spontaneous Breathing

Respiratory variables, including tidal volume and respiratory rate, display significant variability. The probability density function (PDF) of respiratory variables has been shown to contain clinical information and can predict the risk for exacerbation in asthma. However, it is uncertain why this PDF plays a major role in predicting the dynamic conditions of the respiratory system. This paper introduces a stochastic optimal control model for noisy spontaneous breathing, and obtains a Shrödinger’s wave equation as the motion equation that can produce a PDF as a solution. Based on the lobules-bronchial tree model of the lung system, the tidal volume variable was expressed by a polar coordinate, by use of which the Shrödinger’s wave equation of inter-breath intervals (IBIs) was obtained. Through the wave equation of IBIs, the respiratory rhythm generator was characterized by the potential function including the PDF and the parameter concerning the topographical distribution of regional pulmonary ventilations. The stochastic model in this study was assumed to have a common variance parameter in the state variables, which would originate from the variability in metabolic energy at the cell level. As a conclusion, the PDF of IBIs would become a marker of neuroplasticity in the respiratory rhythm generator through Shrödinger’s wave equation for IBIs.


Introduction
Classical physiology is grounded on the principle of homeostasis, in which regulatory mechanisms act to reduce variability and to maintain a steady state [1].Cherniack et al. [2] applied a systems engineering approach to the control of respiration, describing a controller (brain stem respiratory pattern generator), sensors (chemo-and mechanoreceptors), and a plant (airways, chest wall, muscles, and pulmonary tissue).With this model, fluctuations are often dismissed as "noise" of little or no significance.However, since many systems in nature, including respiration, operate away from an equilibrium point, the importance of taking fluctuations into account was well known from early models of the respiratory control mechanism.For example, measured interbreath intervals of a preterm baby at 39 and 61 weeks of postconceptional age have shown that the baby's breathing pattern was highly irregular at 39 weeks, and that the fluctuations were significantly reduced by 61 weeks [2].
For constructing realistic models of control mechanisms with biological variability in spontaneous breathing, one is faced with the problem of finding suitable ways to characterize them.A characteristic feature of fluctuations is the impossibility of precisely predicting their future values, and thus some researchers have tried to use statistical concepts to model fluctuations.From this statistical viewpoint, Frey et al. and Suki have suggested three points on noisy biological variables: 1) the fluctuations obey their own probability distribution; 2) irregular fluctuations can carry information through the probability distribution; and 3) the probability distribution may be sensitive to physiological or pathological changes [3,4].Thus, to define the physiological or pathological meaning of biological variability, it is important to show why the probability distribution of noisy breathing variables is sensitive to physiological or pathological changes.This paper introduces a stochastic optimal control theory to model spontaneous breathing.By implementing a stochastic process, the method reveals that the probability density function of noisy spontaneous breathing obeys a Shrödinger's wave function, which was introduced for describing motions of a quantum particle.Based on the wave function for noisy breathing, this paper concludes that the probability density function of inter-breath intervals will be a marker of neuroplasticity in the central rhythm generator.

Fluctuations as a Sequence of Random Variables
A characteristic feature of fluctuations is the impossibility of precisely predicting their values.A successful attempt is to model a disturbance as a sequence of random variables or a stochastic process.A stochastic process can be defined as a family of random variables is a function of time which is called a sample function or a trajectory.The trajectories can be regarded as elements of the sample function space Ω.For ordinary random variables whose sample function spaces are Euclidean spaces, probability measures can be assigned by ordinary distribution functions and denoted by P.
Let us assign a probability function to the multidimensional random variable for any k and arbitrary time with a distribution function F as follows, , , , ; , , , , , , which satisfies the conditions of symmetry in all pairs  ,  j j t  and consistency.The consistency condition is expressed by , , , ; , , , lim , , , ; , , , Thus, the mean value of a stochastic process m(t) is defined by use of the probability distribution density The symbol E[ ] denotes expectation, that is, integration with respect to the measure P. The covariance of   X s and   X t are also given by When both the mean value function m(t) and the covariance r(s, t) exist, the stochastic process is said to be of second order.

A Wiener Process and a Markov Process
Let us consider the stochastic process of second order , and .When the set elements are mutually independent, the process is called a process with independent increments.If the variables are only uncorrelated, the process {X(t)} is called a process with uncorrelated or orthogonal increments.A Wiener process is one with orthogonal increments defined by the following conditions: 1) X(0) = 0, 2) X(t) is normal, 3) m(t) = 0 for all t > 0, and 4) the process has independent stationary increments.Since a Wiener process has independent stationary increments and X(0) = 0, the variance of the process is , and the covariance of the process is r(s, t) = c × (the minimal difference between t and s), where the parameter c is called the variance parameter.
A stochastic process is given by the Bayes' rule as follows, , , , ; 2) shows that a Markov process is defined by both the initial probability distribution and the transition probabilities.

Stochastic State Models
State models, i.e., systems of first order difference or differential equations, are very convenient for the analysis of systems.An extension of this concept to stochastic state models requires that the probability distribution of the state variable x at future times should be uniquely determined by the actual value of the state.If X(t + 1) is a random variable which depends on the state variable x at the time t

Stochastic Differential Equations of State Models
Starting with the difference where the term o(h 2 ) denotes the omit terms of higher order than 2. One can easily obtain a stochastic difference equation by adding a disturbance   When the disturbance is a Markov process with independent increments, the conditional distribution of where     w t is a Wiener process with unit variance parameter.Thus, the stochastic state model is obtained for the stochastic process Therefore, the expectation and (2.4.6) respectively, Then, let h go to zero in (2.4.4) and one obtains the following formal expression (2.4.7) , b x t is called a forward drift function of the state x at e t.The stochastic differ the tim ential (2.4.7) is defined as the limit of (2.4.4).However, another expression is possible for dX(t) as follows, The difference , and the variance of pe (2.4.9 where

bles in Noisy Breathing
es of tidal he stochastic given x at the time t.

Breathing . State Varia
Spontaneous breathing is described as a seri volumes or changes in respiratory rhythm.A series of tidal volumes is produced from the neural activity of the respiratory center in the brain.The neural activities of the respiratory center induce changes in the length of respiratory muscles, which are transformed into changes in the pleural pressure through the architectural properties of the ribcage.The changes in the pleural pressure are transformed to the alveolar pressure through the lung parenchyma.The alveolar pressure is transformed into airway pressure by the pulmonary lobule, and goes into the environment by producing airflows through the fractal bronchial tree (Figure 1).It is important to note in Figure 1 that there are two origins of fluctuations in this process: in the respiratory rhythm generator (the neural center of respiration) and in the fractal airway modulator (the phasic asynchronous contractions of airway smooth muscles in the lobular bronchioles) [6].Then, based on that bronchial flow F(t) is composed of N-number of phasic lobular flow (q), a tidal volume V T is defined as following,

A Stochastic State Model
The spontaneous breathing is chara respiratory variables   T V .One will consider the series  

Optimal Controlled Cond
Optimal control deals with the problem of finding a control that a certain optimality law for a given system such criterion is achieved.The optimality criterion includes a value of H similar to the total energy of a mechanical system.In the case of noisy breathing, a cost function H(t) should be of equilibrium at optimal controlled conditions as follows, where U(x) is a potential function of the respiratory system.By use of the probability density function ρ(x, t), the stochastic optimal controlled conditions are expressed by the following, When the series of stochastic variables Here, let us introduce two functions, v(x, t) and u(x, t) as follows Then, the functional relationships of  

ger's Wave Equation as Optimal Controlled Conditions
According to (3.3.1), the optimal condition of noisy breath- It is possible to transform (4.1.1)to the following Open Access AM K. MIN 1542 equatio dix for details regarding the reference [7]), n (see Appen g to Einstein's diffusion equation of (3.3.7), the following equation is obtained, The probability density function  , the cost function would be equal to an optimal value of H. Thus, the wave function of noisy ventilations is defined by the following If the potential function U(x) is dependent on only the variable τ, (4.3.1) can be transformed to the Equation (4.2.3) after rewriting the wave function as meter  or  , thus each side term of (4.2.3) should be a constant  .An equation for  is obtained from the right side term of (4.2.3) as follows, That is, each solution is dependent on k as follows, would relate to patterns of temporal and regional ventilations emerging as a result of phasic of smoot the lobular br contractions h muscles in onchioles.The parameter  would be a marker for em pattern of regional ventilations in the lung.

Inter-Breath Intervals
From the left side te erging

Shrödinger's Wave Equation for
rm of (4.2.3) an equation is obtained as follows, , the following equation is obtained as another wave equation for One can produce a distribution density by     probability of tween at the optimal value of H, which is a inter-breath intervals (IBIs) observed be- For an optimal condition of H, one assumes that the wave function P(τ) is expressed by two functions calculating, one obtains the following equation: Thus, the state of the rhythm generator is uniquely determined with dependence on the value of H.
The Equation (4.3.6)explains how the probability density function (PDF) relates to the function of the central rhythm generator.

iscuss bility?
Biological processes in the body provide endless and astounding source of complexity.This variability is not ply attributable to random noise superimposed on reguprocesses.Instead, some researchers have suggested ral structures which y be important markers of numerous acute and chronic seases [1].Frey et al. and Suki have suggested that the probability distribution of noisy physiologica ould have biological information [3,4].However, little work has been done in regard to the orig s study, fluctuations in respiratory state variables are assumed to be r Planck constant in physics.

Temporal and Regional Distribution of Lobular Ventilations
By using the lobule and fractal bronchial tree model (LBT model) [6], the state variable of spontaneous noisy breathing was expressed by the polar coordinate x, which is composed of two dimensional variables τ and θ.The biological variability of θ may be determined through the variability in the amplitude of tidal volume V T based on (3.1.3).The Legendre Equation (4.2.5) would describe the temporal and regional distribution of pulmonary ventilations, which Venegas et al. recently demonstrated as the images of positron emission tomography (PET) [9].f biological variability originates at the level of the cell, the biological variability of perfusion in the lung is also expected to have the same motion equation as (4.2.5).In the case of pulmonary perfusion, it is necessary to define the state variable from the stroke volume (SV).If the biological variability in both V T and SV is measured simultaneously, one would be able to describe the ventilation-perfusion matching in the lung according to (4.2.6).

Neuroplasticity of the Respiratory Rhythm Generator (RRG)
Respiratory rhythm generation arises in neurons that initiate rhythmic inspiratory and expir activity.Several studies suggest that the pre-Bötz plex, a discrete group of pr ventrolateral medulla, plays a critical role in respiration rhythm generation, although this hypothesis is not without controversy [10].Pattern-forming neurons include premotoneurons and motoneurons in the brain stem or spinal cord, where complex activation patterns arise from interactions between their intrinsic properties and synaptic inputs.Pattern formation establishes the detailed spatio-temporal motor output of respiratory muscles, coordinating their activation to produce a breath with the appropriate characteristics.These coordinated, complex in ractions among groups of neurons in the brain produce an optimal breathing rhythm which is described by P(τ) in (4.3.6).
Mitchell and Johnson have stated that a comprehensive conceptual framework of neuroplasticity in the respiratory control system is lacking [10].However, the Equation (4.3.6) can provide a comprehensive framework for respiratory rhythm generation since this expression includes an optimal total energy H of the respiratory system, the topographical distribution parameter λ of re gional ventilation in the lung, and the probability density reat Frey et al. [3] and Fadel et al. [11] This potential of the RRG shows that development of the RRG in infants leads to a change in parameters α and λ, but no change in the structure of the potential function.If a change in the structure of the potential function signals neuroplasticity, the developmental change of the RRG is not a neuroplastic process.

Conclusion
ave equand ano ned as a complex function including probability density functions of biological v both rhythm and amplitude of spontaneous noisy in Variability in spontaneous breathing is not simply attributable to random noise superimposed on a regular respiratory process.Biological variability should originate from energetic fluctuations at the level of the cell, and thus it is acceptable to assume that biological variability is a universal constant amongst all physiological variables.Under this assumption, a stochastic state model for spontaneous noisy breathing produced Shrödinger's w tion as the motion equation.Based on the lobule and fractal bronchial tree model of the lung, two wave equations were obtained from the Shrödinger's equation: one for the respiratory rhythm generator a ther for the modulator of airway smooth muscles in the lung.From these equations, the function of the respiratory rhythm generator was defi ariability in breathg.The stochastic control model analysis in this study can thus provide a new tool applicable for the analysis of any noisy biological processes.The first term of (4.1.1)is calculated as follows,

Appendix
The third term of (4.1.1)is expressed by following, The Equation (4.1.2) is obtained as the necessity for the criterion of control (A.4) as follows,

Figure 1 . 2 )
Figure 1.Components of respiratory system and produ ing of breathing motions.(a) Co ystem: c mponents of respiratory s the ribcage consists of thoracic structures and the diaphragm, the right lung parenchyma consists of many lobules, a sliced face of right upper lobe lobules with a single bronchiole, and a fractal bronchial tree integrates many lobules; (b) A series of tidal volumes is produced from the neural activity of the respiratory center in the brain.The neural activities of the respiratory center induce changes in the length of respiratory muscles, which are transformed into changes in the pleural pressure through architectural properties of the ribcage.The changes in the pleural pressure are transformed into alveolar pressure through the lung parenchyma, which is composed of a large number of lobules.The alveolar pressure is transformed into airway pressure by the pulmonary lobule, and goes into the environment by producing airflows through the fractal bronchial tree, each branch of which has own bundle of smooth muscles.Bundles of airway smooth muscles dynamically change in length-tension to adapt with conditions of breathing.sinT V qN


is the interbreath interval (IBI), and sin is the pr tion of simultaneously relaxed lobular br cterized by a series of opor onchioles in the lung during a breath.
is called the forward drift function or the backward drift function of state variable x at the giv spect en t, re ively as follows,

Figure 2 .
Figure 2. Probability distribution of regional ventilations in the lung.Each distribution density    

2 
in vitro experim colleagues have proposed that e u ell are essential ponents of biological variability[8].If any biological variability originates from the variability of ene of the cell, it will be acceptable to hypothesize t the fluctuati ental studies, Suki nergetic and metactuations at the level of the c rgy at the hat ons of physiological state variables are described by the single quantity of2   .That is, a living cell would produce biological variability through molecular fluctuations, and this ability could e universal constant biological vari be modeled by th ) of inter-b h intervals (IBIs).
4.1) and (3.4.2), the following relation is necessary if the function f(x) is arbitrary, demonstrated the fractal properties of PDFs of IBIs in preterm, term babies and a third of adults at rest.When there are fractal properties in PDFs of IBIs as follows   1)The second term of (4.1.1)is also transformed as follows, d , By combining (A.1), (A.2) and (A.3), the criterion of optimal control is expressed by the following,