The Markovian Approach for Probabilistic Life-Cycle Assessment of Existing Structures ()
1. Introduction
Since any structural system in aggressive environment suffers from deterioration over time, the damage increases, while some structural capabilities decreases [1, 2]. Thence system’s performance is reduced. Usually deterioration laws governing environmental aggression are unknown; therefore, just a stochastic approach enables to assess the life-cycle of the system [3,4].
Biondini and Garavaglia [5] refer to the life-cycle of a deteriorated structure as a reliability problem, where a loss of performance greater than a prescribed threshold value is considered as a failure. When a failure occurs, the system moves from the current state to another one characterized by a lower level of performance. Maintaining and/or repairing lead to the opposite process: the system moves from an initial low state of performance to a higher one. Both cases describe the failure process as a transition process through different states, due to environmental attacks and/or maintenance actions.
Assuming the Markov Renewal Process (MRP) as the most suitable stochastic dynamic method, the transition process can be modelled [6,7].
MRP is usefully applied in predicting natural hazard and catastrophic events (e.g. earthquakes and typhoons) [7-13] as well as in forecasting material deterioration over time [14-17]. Thence MRP seems to be the best approach for modelling the deterioration of structures in aggressive environment.
Basing on MRP, a deteriorated steel truss has been chosen as the case study for assessing life cycle reliability and planning selective maintenance strategies. Material damage is described in probabilistic terms within a Markov renewal model [14]. The probability of the system’s transition through different performance states has been investigated. Regarding results from Markov method, some selective maintenance scenarios have been discussed. Repair interventions have just been applied to seriously deteriorated members and/or characterized by a low safety margin. Thence following Biondini et al. [7] approach, a suitable condition index is used. Considering the effects of selective maintenance, the here proposed procedure has been applied to life-cycle reliability analysis and maintenance planning of a steel truss structure [18].
2. Markov Renewal Processes and Structural Deterioration Processes
The deterioration process can be seen as a loss of performance affecting a material system. Whenever deterioration reaches a given threshold, the system suffers a “failure” and a loss of performance. This phenomenon can be described as the transition of the system (i.e. the material) through different service states characterized by different levels of performance. Thus, the deterioration process (failure process) may be defined as a transition process [14].
Every transition depends on:
• Magnitude of attack (stress cycle).
• System’s capability to withstand the attack.
Both these parameters depend on a large number of time dependent random variables (r.v.); therefore the transition process is rightly interpreted as a stochastic process. A possible r.v. which can be assumed to describe a transition process is the service lifetime ti defined as “the waiting time spent by the material in the performance state i before a transition” [14].
From this point of view the reliability function 
can be defined as:
. (1)
Equation (1) is the probability that a transition from state i to a next state doesn’t happen by waiting time ti [19], and it can be described as the survival function of the r.v. ti.
If the deterioration process is assumed to be a transition process, a stochastic dynamic process can be assumed to represent it, and Biondini and Garavaglia [5] have shown that the Markov Renewal Process (MRP) seems to be suitable method to describe it [6,7]. MRP are a class of stochastic processes generalizing the Markov jump process as well as the renewal process. It can illustrate the evolution of the system’s life through different service states with different waiting times, and it also enables to consider the age t0 of the system, i.e. the time already spent by the system in the current service state before the prediction is made. This aspect is very important in reliability analysis when maintenance must be planned.
2.1. Introducing Markov Renewal Processes
Here has been proposed a brief review of both the definition and some properties of the MRP with a finite state space where, for 
with 
denotes a matrix of the distribution functions on
, 
denotes a transition matrix on A, and 
a probability distribution on A
(i.e. 
and
).
Considering a two-dimensional stochastic process
defined in a complete probability space
satisfying [6,7]:
1) Initial conditions of t0 = 0 a.s.
2) 
for every k Î A.
a.s. for every 
and
.
The probabilities 
are the transition probabilities of the Markov chain 
and 
are the distribution functions associated to waiting times in the state Jn−1 before moving to the next state
.
DEF. The process 
is called the MRP and is determined by
.
Some consequences of that definition are recalled hereunder:
1) 
is a A-valued Markov chain with transition matrix P and initial distribution a.
2) for every n>1, τ1,
,τn are conditionally independent, given 
and
a.s. (2)
The Markov chain 
represents the states successively visited, and the process 
represents the successive waiting times. The application here proposed assumes the performance levels reached by a structural system along its life as the transitory states. Those levels have been classified with respect to different damage severity. τn are the waiting times between successive losses of performance. Basing on 2) consequence, if damage is classified by severity i Î A and the next damage severity is j Î A, the time between the two losses of performance is a positive random variable with distribution Fij, and density fij, for every i, j = 1, 
, S.
Supposing (J0, J1, t1, ..., tq−1, Jq) describes the MRP within the time-window [0,T], q represents the number of states visited over [0,T]. Thence the last event Jq has an incomplete waiting time tq, which represents the censored data: 
Thus, the conditional likelihood is:
(3)
where 
and 
are the transition probabilities defined above, 
and 
are the density and the cumulative distribution of the probability distribution function chosen and 1 stands for indicator functions.
Maximizing the log-likelihood over the time-window
, the maximum likelihood for 
and for the parameters of 
has been estimated.
2.2. Crossing State Prediction
Knowing the last transition state i as well as the time t0 passed since the transition occurred, the j-state of next transition in a lower performance level has been predicted.
When the MRP is defined (points I-III), considering t0 as the time already passed by the system in the current i-state, the probability that the system will pass to state j is:
(4)
Jn is the state of the present performance, Jn+1 is the state of next reduced performance, τij is the waiting time spent by the system in the state i before moving on j, under the condition that no transition has already occurred (defined by the condition:
); Dt is the discrete time when the prediction can be obtained.
Once distribution functions 
have been defined and
, Equation (4) results in:
(5)
Experimental observations give the probabilities 
using the ratio:
(6)
2.3. Deterioration Process as a MRP
The stress value s recorded and evaluated at every monitoring stage is the random variable chosen for describing the deterioration process of the here proposed system. It is the ratio between the internal forces and the deteriorated area of the members’ cross-sections. Referring to Markov renewal model, the time evolution modeling of such variable need for the following assumptions [5]:
• The structure is undamaged at the initial time
.
• The damaged structure is considered to be in a state 
when
, where 
and 
are the lower and upper thresholds, respectively, which characterize the state i.
• The structure evolves from a state
to another state
, characterized by a lower level of performance
, 
, over a time interval τij. Of course, the condition
, with
, is also possible if some maintenance is performed.
Under the hypothesis of MRP, the time evolution of structural behavior is then represented as transitions between different states of performance.
Choosing an appropriate PDF, the waiting time τij has been modeled for each transition. This choice is not simple and it may be “not unique”. It should be made on the basis of physical knowledge of phenomenon and on the characteristics of the distribution in their tail area where, usually, not much data are recorded.
Physical knowledge suggests that the deterioration follows a damage law that increases over time; therefore the distributions satisfying this tendency are distributions with increasing hazard rate functions 
(7)
Although Weibull distributions and Gamma distributions are distributions characterized by a time-dependent increasing hazard rate function, their behavior is quite different. Regarding Weibull distributions at the increasing of t, with t ® ¥, the function 
tends to an infinite value. Otherwise, considering Gamma for t ® ¥, the function 
tends to an asymptotic value. Then applying the abovementioned general knowledge to the case-study here discussed, for Weibull it comes out that higher is the waiting time spent by the system into the present state i highest the immediate transition into another state of j with lower performance (alert for an immediate failure). Whereas for Gamma we have that higher is the waiting time in the state i highest is the probability of transition but it tend to become almost constant (alert for a probable failure).
Experimental data recorded and their modeling brings out that the most probable behavior is the one modeled by a Gamma distribution (Figure 1).
Basing on previous considerations, a Gamma distribution has been chosen in order to model the waiting times τij. The density of the Gamma distribution is:
, (8)
where the shape parameter aij and the inverse scale parameter 
(rate parameter) are positive parameters.
Thus, though without great emphasis, rare events within the distribution tail have been taken into consideration. After a long waiting time τij, if the transition from the state i into j isn’t occurred yet, the occurrence probability in the next Δt increases with a rise in time, τij. Anyway since values usually refer to a quasi-assured occurrence, the probability never tends to an infinite value.
Although the credibility, or validation, of one model on another is a relevant and largely debated topic [20-22] and mainly one the authors deals with it [23] the issue hasn’t been examined in this paper.
Figure 1. Comparison between hazard rate behaviors in the experimental evidence modeling.
The maximum likelihood estimates for the parameters pij, aij, rij, here presented results maximizing the loglikelihood (3) over the time window [0,T]. Using a computer code, where the routine ROSE of IMSL Fortran Library, involving the Rosenbrock’s optimization method, was implemented, the maximum likelihood assessment has been performed.
3. Deterioration Modeling
Materials deterioration due to aggressive environments or material fatigue compromises the performance of a structural system and its reliability over time. Moreover damage induced by environmental attacks suffers from uncertainty and requires to be appropriately modeled. In the following, a general approach for deterioration modeling of structural members is proposed [17].
Damage Index
Assuming Q as a generic material property, its deterioration over time t is expressed below:
(9)
where 0 denotes the initial undamaged state, and the deterioration over time t is measured by a time-variant damage index
. The following damage model is assumed [17]:
(10)
where
, 
is the normalized time instant of reaching the failure threshold
, w and r are damage parameters defining the shape of the damage curve.
The damage parameters r and w must be chosen according to the actual evolution of the damage process. Damage rates may be associated with the aggressiveness of the environment, as well as with the active stress level.
The following linear relationship is assumed [17]:
(11)
(12)
where subscript a refers to damage associated with environmental aggression, subscript b refers to damage associated with loading effects, and x refers to the ratio between the level of active stress and the limit state value.
In this way, the damage law proposed is able to represent damage mechanisms induced by environmental deterioration, like carbonation of concrete, corrosion of steel or material fatigue. Generally, these mechanisms are present and interacting, and a proper calibration of the damage parameters is required based on experimental observations and/or laboratory accelerated test data.
4. Application
Attention is focused on truss systems subjected to a deterioration process involving a reduction of both the crosssectional area A and material strength 
of each structural member. Without loss of generality, in this study it is assumed that these properties undergo the same damage process:
(13)
(14)
where 0 denotes the initial undamaged state, and the damage rates may be associated with the aggressiveness of the environment, as well as with the level of acting stress s with respect to the material strength
, or
. The damage model assumed is Equation (10).
Application is made on a statically indeterminate truss system with a members’ area of A = 2500 mm2 and total volume of V = 0.0723 m3. The allowable material strength is 
140 MPa. Buckling failures are assumed to be avoided. The structure is subjected to a set of forces F = 25 kN as shown in Figure 2. The initial value of the member cross-sectional area A and of the material strength
, as well as the force F are assumed as deterministic.
Each probabilistic approach requires several experi-
Figure 2. Statically indeterminate truss system with member area A = 2500 mm2 and total volume V = 0.0723 m3.
mental data, which are usually difficult to collect. Using the Monte Carlo simulation the samples were obtained. Based on prescribed probability distribution, the simulation process models the damage parameters ra, rb, ωa, ωb, and TC, as random variables. Considering physical knowledge of phenomena investigated, each distribution has been chosen. Starting from a mean and standard deviation, the model was defined [24,25].
Each probabilistic approach requires several experimental data that is usually difficult to collect. In this study, the samples were obtained by using the Monte Carlo simulation. In this simulation process the damage parameters ra, rb, ωa, ωb, and TC, are modeled as random variables with a prescribed probability distribution. Each distribution is chosen on the basis of physical knowledge of phenomena investigated and the model is made starting from a mean and standard deviation [24,25].
During its life the structure moves into different service states; here three different states are assumed: State 1 relates to low damage, State 2 relates to moderate damage and State 3 relates to heavy damage. The boundaries of the states are chosen on the basis of expert judgment and are presented in Table 1.
For each simulation cycle, a time-variant structural analysis is carried out by updating step-by-step updating of the stiffness matrix of the deteriorated structural members. Referring to the limit state i, 
, when a member reaches the upper bound 
of the state i, the system’s transition occurs when a member reaches the upper bound 
of the state i. For each transition the crossing time is recorded and it is interpreted as the waiting time tij spent by the system in state i before the transition into the next state j.
Considering the deterioration process, if the states are significant, the transition can move state by state, otherwise the transition can involve two or more states; in other words, the transition can pass through two or more states at each step. It is clear that the choice of the size of the state is crucial and it can compromise the model. In the case study proposed, the transition always occurs state by state until failure.
With reference to the limit state
, the system failure is reached when the number of failed members is able to activate a mechanism. The time in which the failure occurs is called failure time. Furthermore whenever the system reaches a given damage threshold,
the time to reach this threshold value is called crossing time.
Then the abovementioned probabilistic approach has been applied to probabilistically investigate the next transition of the system, from the actual state i since t0 to the state j in the next interval Δt. Basing on MC simulation, a population of 5000 samples has been built. For each sample, the failure time and the crossing time related to each damage threshold have been recorded and modeled with Gamma distributions (Equation (8)). Following Equation (5), the probabilities 
are evaluated. Since the transition between i and j could be imminent, probabilities 
with
of 
(n = positive number) are assumed to be dangerous for the structure. Therefore maintenance should be planned at instants close to which this probability is recorded.
5. Selective Maintenance Schedules
Generally, for maintaining two approaches can be performed:
1) Repairing all damaged members: the initial system reliability is fully restored and next maintenance can be applied at constant time intervals.
2) Repairing several damaged members: the initial system reliability is only partially restored.
The application of the MRP can lead to decide “When to operate” but not “Where to operate”. However, considering complex structural systems, it is crucial to decide which members require maintenance. Otherwise investigating where to repair scenarios just involves the deteriorated members. Selective maintenance requires suitable life-cycle performance indicators able to select among the whole system which members require maintenance. To achieve this aim, Biondini et al. in 2008 have pointed out the following condition index m has been defined [17]:
(15)
where d is the damage index, 
is the limit strength assumed for each member k, and 
is the corresponding loading demand. The index 
can be evaluated for each member k at each t-time over the structure’s lifetime. Basing on a suitable threshold μmin, the condition 
can be used to identify members which are seriously deteriorated and/or characterized by a low safety-margin.
Considering results provided by the reliability analysis, different maintenance strategies can be scheduled and different maintenance scenarios can be investigated.
In MRP the initial process instant (i.e. initial state) is relevant. Therefore if the system is new and State 1 = undamaged is the initial state, or the system is an existing building and the initial state is a generic damaged state k, the application of the method changes.
For new buildings, when the probability 
reaches a value less than or equal to
, maintenance can be performed on the whole system or on some members with a m less than a certain value. Thence the system will return to State 1 = undamaged or will be improved in a service state of major performance. The next maintenance depends on the state reached by the system after it. Then the next transition process will be modeled again as MRP.
Considering previous observations, the life-cycle assessment of the system in Figure 2 has been performed. Supposing the system “new” and assuming t0 initially equal to zero and the State 1 undamaged, Equation (5) describes the occurrence of the transition probability from State 1 to State 2, i.e. 
in the next year (Δt = 1) after 17 years of construction (t0 = 17). Furthermore the transition from State 2 into State 3 could occur 2 years after the last transition with a probability
. When the structure is 25 years old, the prediction of failure in the following year becomes very probable,
.
Therefore, maintenance scenarios involve:
a) The repair of the whole system after 17 years of construction. The system will return to State 1 = undamaged, the deterioration process becomes a renewal process and maintenance will be carried out every 17 years.
b) The repair of the whole system at the time of failure (25 years). The process becomes again a renewal process.
c) The repair of just some members of the system will move it into a state j equal or different to the initial state State 1 = undamaged. The next maintenance will occur with t0 difference in comparison to the initial one (ex.: t0 = 11 years). The process can be modeled again as a MRP and the new probability of transition is investigated. Considering this scenario, the use of condition index m is crucial.
Figure 3 shows the comparison between the behaviors of the survival functions 
of scenarios a) and b) over the system’s service life. Whenever maintenance is carried out, the system renews itself and the process restarts with the same survival function. Concerning scenarios a) and b), Figure 4 shows the condition index’s behavior over the life of some members of the system when the danger limits assumed are
and
respectively.
Scenario c) enables to propose several other possible scenarios connected with selective maintenance and
Figure 3. Comparison between survival functions of scenario (a) and (b) throughout system’s service life; in both the scenarios when maintenance is carried out the system renews itself.
(a)
(b)
Figure 4. Condition index’s behavior for scenario (a) and scenario (b) throughout the service life and concerning members 6, 8, 9 and 10 of the system.
supported by the behavior of condition index m. Thus, for instance, the first scenario (scenario (1)) foresees maintenance performed after 17 years
on those members with a condition index
. This maintenance moves the system from State 2 into State 1 = undamaged, but the probability of the next transition into State 2 is higher than the probability connected with the new system. MRP suggests that the new transition into State 2 could happen in Dt = 1 year and t0 = 11 years with a probability
.
At the new transition (11 year after the first one) there are 12 members with
; once again it seems to be convenient to replace only these twelve members.
After maintenance, the system moves from State 2 into State 1. Running a new Monte Carlo simulation and applying MRP, the occurrence of the new transition from State 1 into State 2 just 6 years after the last maintenance
comes out. Furthermore a replacement of the whole system is necessary. Since maintaining the whole system after 6 years may be not convenient, scenario (2) is approached. In this case the maintenance of the whole system is performed at the 28th year and the system renews itself after the second maintenance.
Figure 5 shows the comparison between the behavior of the survival functions for scenarios (1) and (2) through the service life of the system. Assuming a danger limit threshold of
, the time-dependent behavior of the condition index concerning some of the members has been investigated (Figure 6).