Improved generalized Fourier amplitude sensitivity test (FAST) for model assessment

The Fourier amplitude sensitivity test (FAST) can be used to calculate the relative variance contribution of model input parameters to the variance of predictions made with functional models. It is widely used in the analyses of complicated process modeling systems. This study provides an improved transformation procedure of the Fourier amplitude sensitivity test (FAST) for non-uniform distributions that can be used to represent the input parameters. Here it is proposed that the cumulative probability be used instead of probability density when transforming non-uniform distributions for FAST. This improvement will increase the accuracy of transformation by reducing errors, and makes the transformation more convenient to be used in practice. In an evaluation of the procedure, the improved procedure was demonstrated to have very high accuracy in comparison to the procedure that is currently widely in use.     

Density modification-based reliability sensitivity analysis

Sensitivity analysis (SA) of a numerical model, for instance simulating physical phenomena, is useful to quantify the influence of the inputs on the model responses. This paper proposes a new sensitivity index, based upon the modification of the probability density function (pdf) of the random inputs, when the quantity of interest is a failure probability (probability that a model output exceeds a given threshold). An input is considered influential if the input pdf modification leads to a broad change in the failure probability. These sensitivity indices can be computed using the sole set of simulations that has already been used to estimate the failure probability, thus limiting the number of calls to the numerical model. In the case of a Monte Carlo sample, asymptotical properties of the indices are derived. Based on Kullback–Leibler divergence, several types of input perturbations are introduced. The relevance of this new SA method is analysed through three case studies.     

Uncertainty and sensitivity analysis techniques as tools for the quality assessment of composite indicators

Summary.  Composite indicators are increasingly used for bench-marking countries' performances. Yet doubts are often raised about the robustness of the resulting countries' rankings and about the significance of the associated policy message. We propose the use of uncertainty analysis and sensitivity analysis to gain useful insights during the process of building composite indicators, including a contribution to the indicators' definition of quality and an assessment of the reliability of countries' rankings. We discuss to what extent the use of uncertainty and sensitivity analysis may increase transparency or make policy inference more defensible by applying the methodology to a known composite indicator: the United Nations's technology achievement index.     

A sensitivity analysis of probabilistic sensitivity analysis in terms of the density function for the input variables

Probabilistic sensitivity analysis (SA) allows to incorporate background knowledge on the considered input variables more easily than many other existing SA techniques. Incorporation of such knowledge is performed by constructing a joint density function over the input domain. However, it rarely happens that available knowledge directly and uniquely translates into such a density function. A naturally arising question is then to what extent the choice of density function determines the values of the considered sensitivity measures. In this paper we perform simulation studies to address this question. Our empirical analysis suggests some guidelines, but also cautions to practitioners in the field of probabilistic SA.     

On two conjectures about perturbations of the stochastic growth rate

The stochastic growth rate describes long-run growth of a population that lives in a fluctuating environment. Perturbation analysis of the stochastic growth rate provides crucial information for population managers, ecologists and evolutionary biologists. This analysis quantifies the response of the stochastic growth rate to changes in demographic parameters. A form of this analysis deals with changes that only occur in some environmental states. Caswell put forth two conjectures about environment-specific perturbations of the stochastic growth rate. The conjectures link the stationary distribution of the stochastic environmental process with the magnitude of some environment-specific perturbations. This note disproves one conjecture and proves the other.     

Measuring what's missing: practical estimates of coverage for stochastic simulations

Stochastic sensitivity analyses rarely measure the extent to which realized simulations cover the search space. Rather, simulation lengths are typically chosen according to expert judgement. In response, this paper recommends a novel application of Good-Turing estimators of missing distributional mass. Using the United Nations Development Programme's Human Development Index, the empirical performance of such coverage metrics are compared to alternative measures of convergence. The former are advantageous – they provide probabilistic estimates of simulation coverage and permit calculation of strict bounds on estimates of pairwise dominance (for all possible weight vectors, how often country X dominates country Y).     

Reliability of global sensitivity indices

Uncertainty and sensitivity analysis is an essential ingredient of model development and applications. For many uncertainty and sensitivity analysis techniques, sensitivity indices are calculated based on a relatively large sample to measure the importance of parameters in their contributions to uncertainties in model outputs. To statistically compare their importance, it is necessary that uncertainty and sensitivity analysis techniques provide standard errors of estimated sensitivity indices. In this paper, a delta method is used to analytically approximate standard errors of estimated sensitivity indices for a popular sensitivity analysis method, the Fourier amplitude sensitivity test (FAST). Standard errors estimated based on the delta method were compared with those estimated based on 20 sample replicates. We found that the delta method can provide a good approximation for the standard errors of both first-order and higher-order sensitivity indices. Finally, based on the standard error approximation, we also proposed a method to determine a minimum sample size to achieve the desired estimation precision for a specified sensitivity index. The standard error estimation method presented in this paper can make the FAST analysis computationally much more efficient for complex models.     

Small sample sensitivity analysis techniques for computer models.with an application to risk assessment

As modeling efforts expand to a broader spectrum of areas the amount of computer time required to exercise the corresponding computer codes has become quite costly (several hours for a single run is not uncommon). This costly process can be directly tied to the complexity of the modeling and to the large number of input variables (often numbering in the hundreds) Further, the complexity of the modeling (usually involving systems of differential equations) makes the relationships among the input variables not mathematically tractable. In this setting it is desired to perform sensitivity studies of the input-output relationships. Hence, a judicious selection procedure for the choic of values of input variables is required, Latin hypercube sampling has been shown to work well on this type of problem.

However, a variety of situations require that decisions and judgments be made in the face of uncertainty. The source of this uncertainty may be lack ul knowledge about probability distributions associated with input variables, or about different hypothesized future conditions, or may be present as a result of different strategies associated with a decision making process In this paper a generalization of Latin hypercube sampling is given that allows these areas to be investigated without making additional computer runs. In particular it is shown how weights associated with Latin hypercube input vectors may be rhangpd to reflect different probability distribution assumptions on key input variables and yet provide: an unbiased estimate of the cumulative distribution function of the output variable. This allows for different distribution assumptions on input variables to be studied without additional computer runs and without fitting a response surface. In addition these same weights can be used in a modified nonparametric Friedman test to compare treatments, Sample size requirements needed to apply the results of the work are also considered. The procedures presented in this paper are illustrated using a model associated with the risk assessment of geologic disposal of radioactive waste.     

Sensitivity,uncertainty, and decision analyses in the prioritization of research

The evaluation of hazards from complex, large scale, technologically advanced systems often requires the construction of computer implemented mathematical models. These models are used to evaluate the safety of the systems and to evaluate the consequences of modifications to the systems. These evaluations, however, are normally surrounded by significant uncertainties related to the uncertainty inherent in natural phenomena such as the weather and those related to uncertainties in the parameters and models used in the evaluation.

Another use of these models is to evaluate strategies for improving information used in the modeling process itself. While sensitivity analysis is useful in defining variables in the model that are important, uncertainty analysis provides a tool for assessing the importance of uncertainty about these variables. A third complementary technique, is decision analysis. It provides a methodology for explicitly evaluating and ranking potential improvements to the model. Its use in the development of information gathering strategies for a nuclear waste repository are discussed in this paper.     

Energy performance evaluation of OECD countries using Bayesian stochastic frontier analysis and Bayesian network classifiers

More recently a large amount of interest has been devoted to the use of Bayesian methods for deriving parameter estimates of the stochastic frontier analysis. Bayesian stochastic frontier analysis (BSFA) seems to be a useful method to assess the efficiency in energy sector. However, BSFA results do not expose the multiple relationships between input and output variables and energy efficiency. This study proposes a framework to make inferences about BSFA efficiencies, recognizing the underlying relationships between variables and efficiency, using Bayesian network (BN) approach. BN classifiers are proposed as a method to analyze the results obtained from BSFA.