New ASA Fellows — 1964

This is an invited expository article for The American Statistician. It reviews the nonparametric estimation of statistical error, mainly the bias and standard error of an estimator, or the error rate of a prediction rule. The presentation is written at a relaxed mathematical level, omitting most proofs, regularity conditions, and technical details.     

Multivariate Calibration — Direct and Indirect Regression Methodology

This paper tries first to introduce and motivate the methodology of multivariate calibration. Next a review is given, mostly avoiding technicalities, of the somewhat messy theory of the subject. Two approaches are distinguished: the estimation approach (controlled calibration) and the prediction approach (natural calibration). Among problems discussed are the choice of estimator, the choice of confidence region, methodology for handling situations with more variables than observations, near-collinearities (with counter-measures like ridge type regression, principal components regression, partial least squares regression and continuum regression), pretreatment of data, and cross-validation vs true prediction. Examples discussed in detail concern estimation of the age of a rhinoceros from its horn lengths (low-dimensional), and nitrate prediction in waste-water from high-dimensional spectroscopic measurements.     

Aggregated functional data model for near-infrared spectroscopy calibration and prediction

Calibration and prediction for NIR spectroscopy data are performed based on a functional interpretation of the Beer–Lambert formula. Considering that, for each chemical sample, the resulting spectrum is a continuous curve obtained as the summation of overlapped absorption spectra from each analyte plus a Gaussian error, we assume that each individual spectrum can be expanded as a linear combination of B-splines basis. Calibration is then performed using two procedures for estimating the individual analytes’ curves: basis smoothing and smoothing splines. Prediction is done by minimizing the square error of prediction. To assess the variance of the predicted values, we use a leave-one-out jackknife technique. Departures from the standard error models are discussed through a simulation study, in particular, how correlated errors impact on the calibration step and consequently on the analytes’ concentration prediction. Finally, the performance of our methodology is demonstrated through the analysis of two publicly available datasets.     

Bayesian inference and prediction of the Rayleigh distribution based on ordered ranked set sampling

Based on ordered ranked set sample, Bayesian estimation of the model parameter as well as prediction of the unobserved data from Rayleigh distribution are studied. The Bayes estimates of the parameter involved are obtained using both squared error and asymmetric loss functions. The Bayesian prediction approach is considered for predicting the unobserved lifetimes based on a two-sample prediction problem. A real life dataset and simulation study are used to illustrate our procedures.     

Bayesian estimation and prediction based on lognormal record values

In this paper we consider the problems of estimation and prediction when observed data from a lognormal distribution are based on lower record values and lower record values with inter-record times. We compute maximum likelihood estimates and asymptotic confidence intervals for model parameters. We also obtain Bayes estimates and the highest posterior density (HPD) intervals using noninformative and informative priors under square error and LINEX loss functions. Furthermore, for the problem of Bayesian prediction under one-sample and two-sample framework, we obtain predictive estimates and the associated predictive equal-tail and HPD intervals. Finally for illustration purpose a real data set is analyzed and simulation study is conducted to compare the methods of estimation and prediction.     

Bayes inference for the modulated power law process

The Modulated Power Law process has been recently proposed as a suitable model for describing the failure pattern of repairable systems when both renewal-type behaviour and time trend are present. Unfortunately, the maximum likelihood method provides neither accurate confidence intervals on the model parameters for small or moderate sample sizes nor predictive intervals on future observations.

This paper proposes a Bayes approach, based on both non-informative and vague prior, as an alternative to the classical method. Point and interval estimation of the parameters, as well as point and interval prediction of future failure times, are given. Monte Carlo simulation studies show that the Bayes estimation and prediction possess good statistical properties in a frequentist context and, thus, are a valid alternative to the maximum likelihood approach.

Numerical examples illustrate the estimation and prediction procedures.     

Conditional covariance penalties for mixed models

The prediction error for mixed models can have a conditional or a marginal perspective depending on the research focus. We introduce a novel conditional version of the optimism theorem for mixed models linking the conditional prediction error to covariance penalties for mixed models. Different possibilities for estimating these conditional covariance penalties are introduced. These are bootstrap methods, cross-validation, and a direct approach called Steinian. The behavior of the different estimation techniques is assessed in a simulation study for the binomial-, the t-, and the gamma distribution and for different kinds of prediction error. Furthermore, the impact of the estimation techniques on the prediction error is discussed based on an application to undernutrition in Zambia.     

Prediction in multilevel generalized linear models

Summary.  We discuss prediction of random effects and of expected responses in multilevel generalized linear models. Prediction of random effects is useful for instance in small area estimation and disease mapping, effectiveness studies and model diagnostics. Prediction of expected responses is useful for planning, model interpretation and diagnostics. For prediction of random effects, we concentrate on empirical Bayes prediction and discuss three different kinds of standard errors; the posterior standard deviation and the marginal prediction error standard deviation (comparative standard errors) and the marginal sampling standard deviation (diagnostic standard error). Analytical expressions are available only for linear models and are provided in an appendix . For other multilevel generalized linear models we present approximations and suggest using parametric bootstrapping to obtain standard errors. We also discuss prediction of expectations of responses or probabilities for a new unit in a hypothetical cluster, or in a new (randomly sampled) cluster or in an existing cluster. The methods are implemented in gllamm and illustrated by applying them to survey data on reading proficiency of children nested in schools. Simulations are used to assess the performance of various predictions and associated standard errors for logistic random-intercept models under a range of conditions.     

Estimation and prediction for low degree polynomial models under measurement errors with an application to forest harvesters

Summary.  In a modern computer-based forest harvester, tree stems are run in sequence through the measuring equipment root end first, and simultaneously the length and diameter are stored in a computer. These measurements may be utilized for example in the determination of the optimal cutting points of the stems. However, a problem that is often passed over is that these variables are usually measured with error. We consider estimation and prediction of stem curves when the length and diameter measurements are subject to errors. It is shown that only in the simplest case of a first-order model can the estimation be carried out unbiasedly by using standard least squares procedures. However, both the first- and the second-degree models are unbiased in prediction. Also a study on real stem is used to illustrate the models that are discussed.     

The Bootstrap and Kriging Prediction Intervals

Kriging is a method for spatial prediction that, given observations of a spatial process, gives the optimal linear predictor of the process at a new specified point. The kriging predictor may be used to define a prediction interval for the value of interest. The coverage of the prediction interval will, however, equal the nominal desired coverage only if it is constructed using the correct underlying covariance structure of the process. If this is unknown, it must be estimated from the data. We study the effect on the coverage accuracy of the prediction interval of substituting the true covariance parameters by estimators, and the effect of bootstrap calibration of coverage properties of the resulting 'plugin' interval. We demonstrate that plugin and bootstrap calibrated intervals are asymptotically accurate in some generality and that bootstrap calibration appears to have a significant effect in improving the rate of convergence of coverage error.     

Prediction Error Estimation Under Bregman Divergence for Non-Parametric Regression and Classification

Abstract.  Prediction error is critical to assess model fit and evaluate model prediction. We propose the cross-validation (CV) and approximated CV methods for estimating prediction error under the Bregman divergence (BD), which embeds nearly all of the commonly used loss functions in the regression, classification procedures and machine learning literature. The approximated CV formulas are analytically derived, which facilitate fast estimation of prediction error under BD. We then study a data-driven optimal bandwidth selector for local-likelihood estimation that minimizes the overall prediction error or equivalently the covariance penalty. It is shown that the covariance penalty and CV methods converge to the same mean-prediction-error-criterion. We also propose a lower-bound scheme for computing the local logistic regression estimates and demonstrate that the algorithm monotonically enhances the target local likelihood and converges. The idea and methods are extended to the generalized varying-coefficient models and additive models.     

Volatility estimation from short time series of stock prices

We consider estimation of the historical volatility of stock prices. It is assumed that the stock prices are represented as time series formed as samples of the solution of a stochastic differential equation with random and time-varying parameters; these parameters are not observable directly and have unknown evolution law. The price samples are available with limited frequency only. In this setting, the estimation has to be based on short time series, and the estimation error can be significant. We suggest some supplements to the existing nonparametric methods of volatility estimation. Two modifications of the standard summation formula for the volatility are derived. In addition, a linear transformation eliminating the appreciation rate and preserving the volatility is suggested.     

Progressively Type-I interval censored competing risks data for the proportional hazards family

In this article, we consider some problems of estimation and prediction when progressive Type-I interval censored competing risks data are from the proportional hazards family. The maximum likelihood estimators of the unknown parameters are obtained. Based on gamma priors, the Lindely's approximation and importance sampling methods are applied to obtain Bayesian estimators under squared error and linear–exponential loss functions. Several classical and Bayesian point predictors of censored units are provided. Also, based on given producer's and consumer's risks accepting sampling plans are considered. Finally, the simulation study is given by Monte Carlo simulations to evaluate the performances of the different methods.     

Estimation and prediction for Chen distribution with bathtub shape under progressive censoring

We consider estimation of the unknown parameters of Chen distribution [Chen Z. A new two-parameter lifetime distribution with bathtub shape or increasing failure rate function. Statist Probab Lett. 2000;49:155–161] with bathtub shape using progressive-censored samples. We obtain maximum likelihood estimates by making use of an expectation–maximization algorithm. Different Bayes estimates are derived under squared error and balanced squared error loss functions. It is observed that the associated posterior distribution appears in an intractable form. So we have used an approximation method to compute these estimates. A Metropolis–Hasting algorithm is also proposed and some more approximate Bayes estimates are obtained. Asymptotic confidence interval is constructed using observed Fisher information matrix. Bootstrap intervals are proposed as well. Sample generated from MH algorithm are further used in the construction of HPD intervals. Finally, we have obtained prediction intervals and estimates for future observations in one- and two-sample situations. A numerical study is conducted to compare the performance of proposed methods using simulations. Finally, we analyse real data sets for illustration purposes.     

On Prediction for Correlated Domains in Longitudinal Surveys

In the survey sampling estimation or prediction of both population’s and subopulation’s (domain’s) characteristics is one of the key issues. In the case of the estimation or prediction of domain’s characteristics one of the problems is looking for additional sources of information that can be used to increase the accuracy of estimators or predictors. One of these sources may be spatial and temporal autocorrelation. Due to the mean squared error (MSE) estimation, the standard assumption is that random variables are independent for population elements from different domains. If the assumption is taken into account, spatial correlation may be assumed only inside domains. In the paper, we assume some special case of the linear mixed model with two random components that obey assumptions of the first-order spatial autoregressive model SAR(1) (but inside groups of domains instead of domains) and first-order temporal autoregressive model AR(1). Based on the model, the empirical best linear unbiased predictor will be proposed together with an estimator of its MSE taking the spatial correlation between domains into account.     

Total least squares solution for compositional data using linear models

The restrictive properties of compositional data, that is multivariate data with positive parts that carry only relative information in their components, call for special care to be taken while performing standard statistical methods, for example, regression analysis. Among the special methods suitable for handling this problem is the total least squares procedure (TLS, orthogonal regression, regression with errors in variables, calibration problem), performed after an appropriate log-ratio transformation. The difficulty or even impossibility of deeper statistical analysis (confidence regions, hypotheses testing) using the standard TLS techniques can be overcome by calibration solution based on linear regression. This approach can be combined with standard statistical inference, for example, confidence and prediction regions and bounds, hypotheses testing, etc., suitable for interpretation of results. Here, we deal with the simplest TLS problem where we assume a linear relationship between two errorless measurements of the same object (substance, quantity). We propose an iterative algorithm for estimating the calibration line and also give confidence ellipses for the location of unknown errorless results of measurement. Moreover, illustrative examples from the fields of geology, geochemistry and medicine are included. It is shown that the iterative algorithm converges to the same values as those obtained using the standard TLS techniques. Fitted lines and confidence regions are presented for both original and transformed compositional data. The paper contains basic principles of linear models and addresses many related problems.     

Statistical inference based on generalized Lindley record values

This paper addresses the problems of frequentist and Bayesian estimation for the unknown parameters of generalized Lindley distribution based on lower record values. We first derive the exact explicit expressions for the single and product moments of lower record values, and then use these results to compute the means, variances and covariance between two lower record values. We next obtain the maximum likelihood estimators and associated asymptotic confidence intervals. Furthermore, we obtain Bayes estimators under the assumption of gamma priors on both the shape and the scale parameters of the generalized Lindley distribution, and associated the highest posterior density interval estimates. The Bayesian estimation is studied with respect to both symmetric (squared error) and asymmetric (linear-exponential (LINEX)) loss functions. Finally, we compute Bayesian predictive estimates and predictive interval estimates for the future record values. To illustrate the findings, one real data set is analyzed, and Monte Carlo simulations are performed to compare the performances of the proposed methods of estimation and prediction.     

New simulation results for the calibration and inverse median estimation problems

This paper gives the results of a new simulation study for the familiar calibration problem and the less familiar inverse median estimation problem. The latter arises when one wishes to estimate from a linear regression analysis the value of the independent variable corresponding to a specified value of the median of the dependent variable. For example, from the results of a regression analysis between stress and time to failure, one might wish to estimate the stress at which the median time to failure is 10,000 hours. In the study, the mean square error, Pitman closeness, and probability of overestimation are compared for both the calibration problem and the inverse median estimation problem for (1) the classical estimator, (2) the inverse estimator, and (3) a modified version of an estimator proposed by Naszodi (1978) for both a small sample and a moderately large sample situation.     

Characterization of an optimal matrix estimator under convex loss function

Characterization of an optimal vector estimator and an optimal matrix estimator are obtained. In each case appropriate convex loss functions are considered. The results are illustrated through the problems of simultaneous unbiased estimation, simultaneous equivariant estimation and simultaneous unbiased prediction. Further an optimality criterion is proposed for matrix unbiased estimation and it is shown that the matrix unbiased estimation of a matrix parametric function and the minimum variance unbiased estimation of its components are equivalent.     

A risk set calibration method for failure time regression by using a covariate reliability sample

Regression parameter estimation in the Cox failure time model is considered when regression variables are subject to measurement error. Assuming that repeat regression vector measurements adhere to a classical measurement model, we can consider an ordinary regression calibration approach in which the unobserved covariates are replaced by an estimate of their conditional expectation given available covariate measurements. However, since the rate of withdrawal from the risk set across the time axis, due to failure or censoring, will typically depend on covariates, we may improve the regression parameter estimator by recalibrating within each risk set. The asymptotic and small sample properties of such a risk set regression calibration estimator are studied. A simple estimator based on a least squares calibration in each risk set appears able to eliminate much of the bias that attends the ordinary regression calibration estimator under extreme measurement error circumstances. Corresponding asymptotic distribution theory is developed, small sample properties are studied using computer simulations and an illustration is provided.