Expert judgement-based tuning of the system reliability neural network

ABSTRACT The neural network tuning procedure applied to reliability analyses of anthrop technical systems, based on judgements of experts - experienced operating practicians. Numerical and linguistic elicitation of the judgements, analyses of the network input and output data correlation and of the AHP method processing deviation are presented. Example of data elicitation and correlation analysis of a reliability arrangement of the seagoing ship propulsion system are included to the article.


INTRODUCTION
Neural networks (NNs) can be useful wherever there are difficulties with the formulation and/or solution of an analytical model but where the network tuning data are available.The data may be objective or subjective, i.e. derived from human memory.Such fields include, among other domains, reliability and safety of anthrop technical systems, in particular complex systems where formal models may be burdened with considerable uncertainty.
Tuning a neural network consists in determining the values of network input/output (I/O) parameters.When objective data are not available, the tuning may be based on expert judgement.There are fields of technology where experts can be found only among experienced operators.This is the case discussed in this paper.
The neural network I/O values must be correlated -the non-correlated values are obviously useless for tuning.Obtaining correlated subjective data is an essential difficulty in the considered case.There are several ways of effecting the level of correlation.First of all it is proper selection of experts and methods of judgement elicitation, and applying effective methods of processing the obtained data.

EXPERTS AND ELICITATION PROCEDURE
The expert is assumed to be a person well acquainted with the subject he is expected to formulate his judgement on.The knowledge is connected with the experience acquired by years-long practice.The expert should also be capable of formulating his judgement.This is connected with the level of his education and the language used in the elicitation process, particularly as regards the parameters the expert is expected to estimate.This may be the language of numerical or linguistic values.Numerical values are better but are more difficult to articulate -also errors in judgements are more likely.The analyst designing the reliability investigation method must in each case select properly the category of available experts, the number of experts and the elicitation language to be used.The number and qualifications of the available experts may be a limitation.
In the case of reliability, tuning pertains to characteristics expressed by probabilistic values, e.g.reliability function, unreliability function, failure rate, intensity function, or to physical values -operands in those expressions -e.g.failure frequency, time to failure or time between failures.
Preferred candidates for experts are persons having experience in observing the operation process of the elicitation objects for sufficiently long time and having proper theoretical knowledge.The reliability analyst must determine the elicitation language and choose the available expert category.For instance, in the reliability investigation of nuclear power stations, operators of those objects may be considered highclass specialists with knowledge of the calculus of probability, and on seagoing ships -members of the crew with various education levels, generally not familiar with the probability.
Man is not good as a probability estimator.His judgements show biases, weak calibration, incoherence, and overconfidence tendency.Dependences may occur between expert judgements.These flaws cannot be fully removed in the elicitation phase [4,9].Table 1 contains data on presentation forms of probabilistic judgements.The type of probability distribution is connected with the character of the respective event.For instance, the up time or maintenance time distributions are continuous and the human error probability is generally estimated by discrete distributions.Distributions of dangerous event circumstances, appearing in the event trees, are generally estimated by discrete two-point distributions.The distribution dimension may be essential in the case of estimation of probability distributions of different categories of losses and also in distributions of state parameters of the risk analysis object environment, e.g.meteorological conditions.

Models of probability distributions
Empirical.Formal -e.g.exponential, normal or Markov processes.

Distribution dimension
Single-dimensional or multidimensional.

Calibration
With or without calibration by objective data.
Differentiation of frequent and rare events is essential.The latter may be out of the experience of experts, who have not observed them.The estimation of the probability of occurrence of rare events is based on intuition.
As regards the information in the last row of Table 1, significant is the fact of having or not having objective information which could be used for the calibration of expert judgements.Without such information the estimation results may bear considerable uncertainty.
Reference [4] describes conditions to be fulfilled in the expert judgement elicitation phase.The main conditions pertain to the selection of experts, instructions, questionnaires and the way they should be filled-in, and also to the independence of judgements and the interview duration.Experts are chosen according to the subject of investigation.They are informed about the purpose and procedure of the investigation, data processing method and other possible questions.They are asked to formulate their judgements in a straightforward and honest way.Experts present their judgements by filling in the prepared questionnaires; give numbers or linguistic values by marking the appropriate fields.They cannot answer questions on subjects they have no knowledge about.The questionnaires should be as simple as possible.
The expert judgements must be formulated independently, which means that the experts must not contact with each other during the elicitation process.They are supposed to formulate their judgements entirely on their own, relying on their personal experience [1,4,9].

TUNING THE NEURAL NETWORKS OF RELIABILITY
The anthrop technical object of interest will be treated as a reliability system.It may be a no-repairable or repairable system with negligible or non-negligible renewal time.The catastrophic failure state will be modeled as the absorbing state.
Let's assume that the task to be done by the neural network is to determine system reliability model parameters.If we choose a specific reliability model then the basic problem is to determine its parameters.In general, the uncertainty of a model is connected mainly with the uncertainty of its parameters.By identifying parameters of the model we obtain sufficient material to be able to control its reliability, and the neural network model becomes simpler than that of the system as a whole.
The first step in programming an investigation is to define its objective and assumptions concerning the investigation subject (definitions of the system and its operational states, formal reliability model, characteristics of the environment).With these assumptions the system fault tree (FT) can be constructed.The fault tree allows determining the sets of elements effecting the system reliability, and also indirect relations if it appears helpful in the elicitation process.
We shall continue the consideration with, for instance, the exponential reliability system and the Markov chain.The task of the neural network will be to determine parameters of those systems.

Exponential distribution
where: λ = 1/MTTF -the failure rate, MTTF -the mean time to failures, which can be estimated by an expert without difficulties, t -is the time.

Markov chain
The graph presents the simplest case of a chain with two states: 1 -the state of operational use with λ(t) failure rate, and 2 -the maintenance state with μ(t) repair rate of the reliability system.It is a nonhomogeneous process with finite renewal time.When the transition rates are not time-dependent then the process becomes homogeneous and the distributions of the state 1 and 2 duration times are exponential.The availability formula for an asymptotic homogenous version of the system takes the form: (2) Parameters of model (2) are the rates of failure λ and repair μ.In general, they are time-dependent, but may be approximated by constant or constant in time intervals.Statistical verification of such simplification is recommended [6,11].From the renewal equations: (3) where: -the number of failures in time interval t, T 0 = MTBF.
From formula (3) -after sufficiently long time: (4) where: -the mean number of failures in time interval t, which can be easily determined from the expert judgements.In the case of identical exponential distributions of times between failures λ = 1/MTBF (5) It is generally assumed that the maintenance times have also exponential distributions with time-independent transition rates μ.This assumption pertains to direct maintenance work time without organizational preparation time and waiting time for beginning the work.In practice, that preparation period may be chaotic, which makes probabilistic estimation of the parameter μ difficult or even impossible.The following approach to the estimation of parameter μ is proposed: 1) adopting the model with negligible renewal time when that time is short compared with the usage time; 2) adopting constant renewal times for individual devices; 3) determining μ from the formula: (6) where: -the mean renewal time, to be estimated by the experts.
In general, the reliability model parameters are functions of operands -physical values -like the time to failure, time between failures, duration times of specific reliability or operational states, and/or the number of failures in a time interval (event frequencies).These values are easier to be determined by an expert than the probabilistic model parameters.They are suggested to be used in elicitation.
The above presented considerations deal with the neural network output parameters, i.e. the top event (TE) in the tree FT, which is the system failure.Now we shall deal with the determination of the input parameters, i.e. basic events (BEs) in relation to TE.These are the system element failures.It is possible to obtain linguistic values of shares of the basic events in the top event frequencies.The linguistic variables have associated sets of values (very rare, rare, occasional, frequent, and very frequent).The linguistic values determined in the elicitation process allow to apply the pairwise comparison method in order to determine the preferences linking individual pairs and then, using the AHP method, the corresponding numerical values [10,12].The correlation level between the neural network output and input parameters should then be verified.
In the case of a large reliability system, the elicitation process of the shares of system elements in the failure frequencies may be subdivided into "layers", for instance in the case of two FT system layers -higher and lower -first determine the shares of higher layer elements in the system failure frequencies and then the shares of lower layer elements in the failure frequencies of It was proved [14] that with this particular value of c the difference between the linguistic judgment matrix and the matrix derived from the priority vector is at a minimum.Table 2 shows the Xu scale data with parameter c = 2.

DATA CORRELATION PROBLEM
The data elicited from the experts may have a numerical or linguistic form.The latter are adjectives valuating the intensity of the measured variable of an object's feature, process or phenomenon.These adjectives may be assigned natural numbers, ascending with the increasing intensity, i.e. perform ranking of the measured value.The estimates of the linguistic values are done by means of ordered scales.The scales have order relations.As indicated above, in the case of physical objects observed in the operation process the numerical values pertain to independent variables in expressions defining the reliability model parameters.They are estimated in interval scales.Such scales have a constant unit of measurement, the order relation and an optionally chosen zero point.
The correlation analysis of the values measured on the above presented scales is carried out using non-parametric methods.
the higher layer objects.The principles of completeness and disjointness should be maintained.This procedure will increase the correctness of expert estimates.
Let φ be an parameter of a NN and p = (p 1 , p 2 , ..., p n ) Tvector of parameter sequences of BS event shares in the system fault tree, being the network inputs determined by the AHP method.Then the input vector will take the form [10,12]: The preferred method of determining the sequence vector is the logarithmic least squares method [10].
Fig. 1 presents the flow diagram of the elicitation and data processing algorithm for obtaining correlated sets suitable for NN tuning.

PROCESSING OF EXPERT DATA BY THE AHP METHOD
Linguistic estimates of the shares of reliability system elements in the system failure frequency consist in expert choice of the share value from the set of five values.The estimates are given numbers from 1 to 5. Differences of experts' judgements indicate the scale of preferences in the pairwise comparison of the linguistic estimates.Depending on these differences, the preferences are assigned weights r(s) in accordance with a scale function.Then the linguistic judgment matrix R is determined and transformed into the priority vector p: (8) where: R -the linguistic judgement matrix r ij -the preference of i-th to j-th share (i,j = 1, 2,…, n), with properties: r ij > 0, ∀ i,j r i,j = 1/r j,i .
Matrix R is consistent if its elements fulfill the condition: r i,j r j,k = r ik ∀i,j,k = 1, 2, ..., n.
The priority vector p = (p 1 , ..., p n ) T is determined by approximation of matrix R with matrix P, where: The measure of consistency of the Xu processing is the difference between matrices P and R [14]: (10) The Xu scale [14] is a geometrical scale with parameter c = 2, in the form: (11) where: I(s) -the index of the preference symbol s, c -the parameter.
They compensate the effects of the standing-out measurements and the non-normality of the elicited values [13].
To the correlation analysis of the data obtained from the elicitation process the R. Spearman's method is applied.The output and input data are ranked by assigning them ascending natural numbers starting from 1.The numbers are ranks.The ranking process may be performed also with the decreasing sequence.The Spearman's rank correlation coefficient is determined from the following formula [13]: (12) where: d i -the difference between the ranks of corresponding characteristic values.The correlation coefficient ranges within -1 ≤ r s ≤ 1.
The correlation coefficient determined by formula ( 11) is applicable only to a random sample, and the correlation of the general population should also be checked.For that the zero hypothesis H o : ρ = 0, where ρ is the correlation coefficient of the general population, is verified against the alternative hypothesis H 1 : ρ ≠ 0. The t-Student test is used for verification.It is assumed that the population has the Student distribution with n -1 degrees of freedom.The test has the form: (13) where: n -the size of the sample.

Fig. 3. Distribution of propulsion system ICF type failure numbers
The test value t is compared with the critical value t p determined from the Student tables for an assumed significance level p and n -2 degrees of freedom.If t > t p then hypothesis H 0 is rejected and hypothesis H 1 is accepted (the correlation exists also in the general population) [13].

Object of analysis, elicitation
The example illustrates the reliability analysis of a container carrier propulsion system (PS) with slow-speed piston internal combustion engine and screw propeller, operating in the North Atlantic (*).The reliability was analysed for immediate catastrophic failures (ICF) of the PS; it is substantial in the case of the risk analysis of PS.Fig. 2 presents the FT of the PS.It was assumed that ICFs can occur only during the active usage state of the system, i.e. during the ship sea voyage.Detailed data of this example can be found in [1,2,3].
A questionnaire was presented with definitions of the investigated object, "immediately catastrophic failure of PS" and "sea traffic", as well as tables to be filled in by the experts and suggestions how to do it.The questionnaire was filled in by 50 experts -ship engineers with multi-year experience.
Questions were asked about the annual frequency of propulsion system ICF type events, the share of subsystem (SS) failures in the PS system failure frequency and the share of module (set of devices -SD) ICF type failures in the SS failure frequencies.Appendix 1 shows fragments of the questionnaire.
The data elicitated by the experts have been multiplied in the neural network tuning process to 2 -3 hundreds base on uniform distribution.

Analysis of the correlation of elicitation results
As regards the propulsion system as a whole, the experts gave their subjective estimates of the ICF type failures per year in numbers, and linguistically by marking one of fields in the order scale containing numbers and descriptions of that failure frequency (see Appendix 1).For instance, Fig. 3 presents a histogram of the system PS failure frequency per year.The histogram shows a distribution close to the normal distribution, which may be considered correct in the case of observation of dangerous events with more or less steady frequency of occurrence.
The elicitation of the ICF type failures of SS subsystems and their SD modules consisted in marking appropriate fields in the questionnaire order scales (see Appendix 1).They indicated the share of a given SS or SD in the ICF type failure frequency of a direct higher level object, i.e. of the propulsion system in the case of SSs and of a specific SS in the case of SD modules.The shares were pairwise compared and respective differences of numbers were treated as numerical estimates of experts' preferences.The preferences were assigned values in accordance with the geometrical scale function.

CONCLUSION
The above text presents the method to tune the neural network.The network is only directed to determine subjective parameters of the complex reliability system model.The numerical elicitation of the system's failure frequency is performed, with further linguistic elicitation of the shares of failures in system's elements in that value.The first value serves for determining the network's output parameter and the second one serves for determining the set of network's input parameters.The linguistic values are subjected to pairwise comparison and assigned numerical values by means of the AHP method.The correlation between the output and input parameters is investigated.When it is positive we can find that the tuning parameters are corrected for general population.
The content presented above allows concluding that the used neural network tuning procedure gives correct results.It appears appropriate when makes use of experts who are experienced operators of the reliability analysis objects.It may be useful for network tuning in reliability analyses and for the technical system risk management.

Tab. 1 .
Taxonomy of the forms of probabilistic judgements used in reliability Distribution type Discrete or continuous.If discrete then two-point or multipoint.If continuous then in known functional form or empirical.

Fig. 1 .
Fig. 1.Network of the algorithm of data elicitation and processing for tuning the neural network