Optimality of the quasi-score estimator in a mean–variance model with applications to measurement error models

We consider a regression of y$y$ on x$x$ given by a pair of mean and variance functions with a parameter vector θ$\theta$ to be estimated that also appears in the distribution of the regressor variable x$x$. The estimation of θ$\theta$ is based on an extended quasi-score (QS) function. We show that the QS estimator is optimal within a wide class of estimators based on linear-in-y$y$ unbiased estimating functions. Of special interest is the case where the distribution of x$x$ depends only on a subvector α$\alpha$ of θ$\theta$, which may be considered a nuisance parameter. In general, α$\alpha$ must be estimated simultaneously together with the rest of θ$\theta$, but there are cases where α$\alpha$ can be pre-estimated. A major application of this model is the classical measurement error model, where the corrected score (CS) estimator is an alternative to the QS estimator. We derive conditions under which the QS estimator is strictly more efficient than the CS estimator.     

Minimum Hellinger distance estimation in a nonparametric mixture model

In this paper, we investigate the estimation problem of the mixture proportion λ$\lambda$ in a nonparametric mixture model of the form λF(x)+(1-λ)G(x)$\lambda F(x)+(1-\lambda )G(x)$ using the minimum Hellinger distance approach, where F and G are two unknown distributions. We assume that data from the distributions F and G   as well as from the mixture distribution λF+(1-λ)G$\lambda F+(1-\lambda )G$ are available. We construct a minimum Hellinger distance estimator of λ$\lambda$ and study its asymptotic properties. The proposed estimator is chosen to minimize the Hellinger distance between a parametric mixture model and a nonparametric density estimator. We also develop a maximum likelihood estimator of λ$\lambda$. Theoretical properties such as the existence, strong consistency, asymptotic normality and asymptotic efficiency of the proposed estimators are investigated. Robustness properties of the proposed estimator are studied using a Monte Carlo study. Two real data examples are also analyzed.     

Moderate deviations for LS estimator in simple linear EV regression model

In this paper, we consider the following simple linear Errors-in-Variables (EV) regression model _ηi=θ+β_xi+_?i${\eta }_i=\theta +\beta x_i+{?}_i$, _ξi=_xi+_δi${\xi }_i=x_i+{\delta }_i$, 1?i?n$1?i?n$. The moderate deviation principle for the least squares (LS) estimators of the unknown parameters θ$\theta$, β$\beta$ in the model are obtained.     

United statistics,confidence quantiles,Bayesian statistics

A survey of research by Emanuel Parzen on how quantile functions provide elegant and applicable formulas that unify many statistical methods, especially frequentist and Bayesian confidence intervals and prediction distributions. Section 0: In honor of Ted Anderson's 90th birthday; Section 1: Quantile functions, endpoints of prediction intervals; Section 2: Extreme value limit distributions; Sections 3, 4: Confidence and prediction endpoint function: Uniform(0,θ)$(0,\theta )$, exponential; Sections: 5, 6: Confidence quantile and Bayesian inference normal parameters μ$\mu$, σ$\sigma$; Section 7: Two independent samples confidence quantiles; Section 8: Confidence quantiles for proportions, Wilson's formula. We propose ways that Bayesians and frequentists can be friends!     

Efficiency of weighted averages

When combining estimates of a common parameter (of dimension d?1$d?1$) from independent data sets—as in stratified analyses and meta analyses—a weighted average, with weights ‘proportional’ to inverse variance matrices, is shown to have a minimal variance matrix (a standard fact when d=1$d=1$)—minimal in the sense that all convex combinations of the coordinates of the combined estimate have minimal variances. Minimum variance for the estimation of a single coordinate of the parameter can therefore be achieved by joint estimation of all coordinates using matrix weights. Moreover, if each estimate is asymptotically efficient within its own data set, then this optimally weighted average, with consistently estimated weights, is shown to be asymptotically efficient in the combined data set and avoids the need to merge the data sets and estimate the parameter in question afresh. This is so whatever additional non-common nuisance parameters may be in the models for the various data sets. A special case of this appeared in Fisher [1925. Theory of statistical estimation. Proc. Cambridge Philos. Soc. 22, 700–725.]: Optimal weights are ‘proportional’ to information matrices, and he argued that sample information should be used as weights rather than expected information, to maintain second-order efficiency of maximum likelihood. A number of special cases have appeared in the literature; we review several of them and give additional special cases, including stratified regression analysis—proportional-hazards, logistic or linear—, combination of independent ROC curves, and meta analysis. A test for homogeneity of the parameter across the data sets is also given.     

Robust median estimator in logistic regression

This paper introduces a median estimator of the logistic regression parameters. It is defined as the classical _L1$L_1$-estimator applied to continuous data _Z1,…,_Zn$Z_1,\dots ,Z_n$ obtained by a statistical smoothing of the original binary logistic regression observations _Y1,…,_Yn$Y_1,\dots ,Y_n$. Consistency and asymptotic normality of this estimator are proved. A method called enhancement is introduced which in some cases increases the efficiency of this estimator. Sensitivity to contaminations and leverage points is studied by simulations and compared in this manner with the sensitivity of some robust estimators previously introduced to the logistic regression. The new estimator appears to be more robust for larger sample sizes and higher levels of contamination.     

Sufficient conditions for Bayesian consistency

We introduce the Hausdorff α$\alpha$-entropy to study the strong Hellinger consistency of posterior distributions. We obtain general Bayesian consistency theorems which extend the well-known results of Barron et al. [1999. The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27, 536–561] and Ghosal et al. [1999. Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27, 143–158] and Walker [2004. New approaches to Bayesian consistency. Ann. Statist. 32, 2028–2043]. As an application we strengthen previous results on Bayesian consistency of the (normal) mixture models.     

Bayesian design in queues: An application to aeronautic maintenance

We exploit Bayesian criteria for designing M/M/c//r$M/M/c//r$ queueing systems with spares. For illustration of our approach we use a real problem from aeronautic maintenance, where the numbers of repair crews and spare planes must be sufficiently large to meet the necessary operational capacity. Bayesian guarantees for this to happen can be given using predictive or posterior distributions.     

Empirical Bayes regression analysis with many regressors but fewer observations

In this paper, we consider the prediction problem in multiple linear regression model in which the number of predictor variables, p, is extremely large compared to the number of available observations, n  . The least-squares predictor based on a generalized inverse is not efficient. We propose six empirical Bayes estimators of the regression parameters. Three of them are shown to have uniformly lower prediction error than the least-squares predictors when the vector of regressor variables are assumed to be random with mean vector zero and the covariance matrix (1/n)X^tX$(1/n){\mathit{X}}^{\mathrm{t}}\mathit{X}$ where X^t=(_x1,…,_xn)${\mathit{X}}^{\mathrm{t}}=({\mathit{x}}_1,\dots ,{\mathit{x}}_n)$ is the p×n$p\times n$ matrix of observations on the regressor vector centered from their sample means. For other estimators, we use simulation to show its superiority over the least-squares predictor.     

Design selection criteria for discrimination/estimation for nested models and a binomial response

The aim of an experiment is often to enable discrimination between competing forms for a response model. We investigate the selection of a continuous design for a non-sequential strategy when there are two competing generalized linear models for a binomial response, with a common link function and the linear predictor of one model nested within that of the other. A new criterion, _TE$T_E$-optimality, is defined, based on the difference in the deviances from the two models, and comparisons are made with T$T$-, _Ds$D_s$- and D$D$-optimality. Issues are raised through the study of two examples in which designs are assessed using simulation studies of the power to reject the null hypothesis of the smaller model being correct, when the data are generated from the larger model. Parameter estimation for discrimination designs is also discussed and a simple method is investigated of combining designs to form a hybrid design in order to achieve both model discrimination and estimation. This method has a computational advantage over the use of a compound criterion and the similar performance of the designs obtained from the two approaches is illustrated in an example.