Improving the estimators of the parameters of a probit regression model: A ridge regression approach

This paper considered the estimation of the regression parameters of a general probit regression model. Accordingly, we proposed five ridge regression (RR) estimators for the probit regression models for estimating the parameters (β)$(\beta )$ when the weighted design matrix is ill-conditioned and it is suspected that the parameter β$\beta$ may belong to a linear subspace defined by Hβ=h$H\beta =h$. Asymptotic properties of the estimators are studied with respect to quadratic biases, MSE matrices and quadratic risks. The regions of optimality of the proposed estimators are determined based on the quadratic risks. Some relative efficiency tables and risk graphs are provided to illustrate the numerical comparison of the estimators. We conclude that when q≥3$q\ge 3$, one would uses PRRRE; otherwise one uses PTRRE with some optimum size α$\alpha$. We also discuss the performance of the proposed estimators compare to the alternative ridge regression method due to Liu (1993).     

On the stability of the risk hull method for projection estimators

We consider in this paper the regularization by projection of a linear inverse problem Y=Af+εξ$Y=\mathit{Af}+\epsilon \xi$ where ξ$\xi$ denotes a Gaussian white noise, A   a compact operator and ε>0$\epsilon >0$ a noise level. Compared to the standard unbiased risk estimation (URE) method, the risk hull minimization (RHM) procedure presents a very interesting numerical behavior. However, the regularization in the singular value decomposition setting requires the knowledge of the eigenvalues of A$A$. Here, we deal with noisy eigenvalues: only observations on this sequence are available. We study the efficiency of the RHM method in this situation. More generally, we shed light on some properties usually related to the regularization with a noisy operator.     

New large-sample confidence intervals for a linear combination of binomial proportions

In this paper, we consider the problem wherein one desires to estimate a linear combination of binomial probabilities from k>2$k>2$ independent populations. In particular, we create a new family of asymptotic confidence intervals, extending the approach taken by Beal [1987. Asymptotic confidence intervals for the difference between two binomial parameters for use with small samples. Biometrics 73, 941–950] in the two-sample case. One of our new intervals is shown to perform very well when compared to the best available intervals documented in Price and Bonett [2004. An improved confidence interval for a linear function of binomial proportions. Comput. Statist. Data Anal. 45, 449–456]. Furthermore, our interval estimation approach is quite general and could be extended to handle more complicated parametric functions and even to other discrete probability models in stratified settings. We illustrate our new intervals using two real data examples, one from an ecology study and one from a multicenter clinical trial.     

Decomposable pseudodistances and applications in statistical estimation

The aim of this paper is to introduce new statistical criteria for estimation, suitable for inference in models with common continuous support. This proposal is in the direct line of a renewed interest for divergence based inference tools imbedding the most classical ones, such as maximum likelihood, Chi-square or Kullback–Leibler. General pseudodistances with decomposable structure are considered, they allowing defining minimum pseudodistance estimators, without using nonparametric density estimators. A special class of pseudodistances indexed by α>0$\alpha >0$, leading for α↓0$\alpha \downarrow 0$ to the Kullback–Leibler divergence, is presented in detail. Corresponding estimation criteria are developed and asymptotic properties are studied. The estimation method is then extended to regression models. Finally, some examples based on Monte Carlo simulations are discussed.     

A new estimation method for Weibull-type tails based on the mean excess function

Studying the right tail of a distribution, one can classify the distributions into three classes based on the extreme value index γ$\gamma$. The class γ>0$\gamma >0$ corresponds to Pareto-type or heavy tailed distributions, while γ<0$\gamma <0$ indicates that the underlying distribution has a finite endpoint. The Weibull-type distributions form an important subgroup within the Gumbel class with γ=0$\gamma =0$. The tail behaviour can then be specified using the Weibull tail index. Classical estimators of this index show severe bias. In this paper we present a new estimation approach based on the mean excess function, which exhibits improved bias and mean squared error. The asserted properties are supported by simulation experiments and asymptotic results. Illustrations with real life data sets are provided.     

Representing a function of two variables as a superposition over angles of ridge functions with a given shape via linear programming

A ridge function with shape function g   in the horizontal direction is a function of the form g(x)h(y,0)$g(x)h(y,0)$. Along each horizontal line it has the shape g(x)$g(x)$, multiplied by a function h(y,0)$h(y,0)$ which depends on the y-value of the horizontal line. Similarly a ridge function with shape function g   in the vertical direction has the form g(y)h(x,π/2)$g(y)h(x,\pi /2)$. For a given shape function g it may or may not be possible to represent an arbitrary   function f(x,y)$f(x,y)$ as a superposition over all angles of a ridge function with shape g   in each direction, where h=_hf=_hf,g$h=h_f=h_{f,g}$ depends on the functions f and g   and also on the direction, θ:h=_hf,g(·,θ)$\theta :h=h_{f,g}(·,\theta )$. We show that if g   is Gaussian centered at zero then this is always possible and we give the function _hf,g$h_{f,g}$ for a given f(x,y)$f(x,y)$. For highpass or for odd shapes g  , we show it is impossible to represent an arbitrary f(x,y)$f(x,y)$, i.e. in general there is no _hf,g$h_{f,g}$. Note that our problem is similar to tomography, where the problem is to invert the Radon transform, except that the use of the word inversion is here somewhat “inverted”: in tomography f(x,y)$f(x,y)$ is unknown and we find it by inverting the projections of f  ; here, f(x,y)$f(x,y)$ is known, g(z)$g(z)$ is known, and _hf(·,θ)=_hf,g(·,θ)$h_f(·,\theta )=h_{f,g}(·,\theta )$ is the unknown.     

Sharp upper bounds for the expected values of non-extreme order statistics from discrete distributions

We consider the problem of determining sharp upper bounds on the expected values of non-extreme order statistics based on i.i.d. random variables taking on N values at most. We show that the bound problem is equivalent to the problem of establishing the best approximation of the projection of the density function of the respective order statistic based on the standard uniform i.i.d. sample onto the family of non-decreasing functions by arbitrary N  -valued functions in the norm of L²(0,1)$L^2(0,1)$ space. We also present an algorithm converging to the local minima of the approximation problems.     

Posterior convergence rates for Dirichlet mixtures of beta densities

We consider Bayesian density estimation for compactly supported densities using Bernstein mixtures of beta-densities equipped with a Dirichlet prior on the distribution function. We derive the rate of convergence for α$\alpha$-smooth densities for 0<α?2$0<\alpha ?2$ and show that a faster rate of convergence can be obtained by using fewer terms in the mixtures than proposed before. The Bayesian procedure adapts to the unknown value of α$\alpha$. The modified Bayesian procedure is rate-optimal if α$\alpha$ is at most one. This result can be extended to two dimensions.     

On the mean residual life of records

Suppose that a technical system is subject to shocks, e.g. peaks of voltages from a sequence of identically independent voltages having a lower limit value v>0$v>0$. We propose a new definition for the mean residual life of the records of the sequence and study its various properties.     

Modified quasi-profile likelihoods from estimating functions

We discuss higher-order adjustments for a quasi-profile likelihood for a scalar parameter of interest, in order to alleviate some of the problems inherent to the presence of nuisance parameters, such as bias and inconsistency. Indeed, quasi-profile score functions for the parameter of interest have bias of order O(1)$\mathrm{O}(1)$, and such bias can lead to poor inference on the parameter of interest. The higher-order adjustments are obtained so that the adjusted quasi-profile score estimating function is unbiased and its variance is the negative expected derivative matrix of the adjusted profile estimating equation. The modified quasi-profile likelihood is then obtained as the integral of the adjusted profile estimating function. We discuss two methods for the computation of the modified quasi-profile likelihoods: a bootstrap simulation method and a first-order asymptotic expression, which can be simplified under an orthogonality assumption. Examples in the context of generalized linear models and of robust inference are provided, showing that the use of a modified quasi-profile likelihood ratio statistic may lead to coverage probabilities more accurate than those pertaining to first-order Wald-type confidence intervals.     

Mantel-Haenszel estimators for irregular sparse K2×J tables

This paper deals with sparse K2×J(J>2)$K2\times J(J>2)$ tables. Projection-method Mantel–Haenszel (MH) estimators of the common odds ratios have been proposed for K2×J$K2\times J$ tables, which include Greenland's generalized MH estimator as a special case. The method projects log-transformed MH estimators for all K2×2$K2\times 2$ subtables, which were called naive MH estimators, onto a linear space spanned by log odds ratios. However, for sparse tables it is often the case that naive MH estimators are unable to be computed. In this paper we introduce alternative naive MH estimators using a graph that represents K2×J$K2\times J$ tables, and apply the projection to these alternative estimators. The idea leads to infinitely many reasonable estimators and we propose a method to choose the optimal one by solving a quadratic optimization problem induced by the graph, where some graph-theoretic arguments play important roles to simplify the optimization problem. An illustration is given using data from a case–control study. A simulation study is also conducted, which indicates that the MH estimator tends to have a smaller mean squared error than the MH estimator previously suggested and the conditional maximum likelihood estimator for sparse tables.     

Laplace approximations to means and variances with asymptotic modes

The moment-generating function method, which is proposed by Tierney et al. [1989a. Fully exponential Laplace approximations to expectations and variances of nonpositive functions. J. Amer. Statist. Assoc. 84, 710–716], is an asymptotic technique of approximating a posterior mean of a general function by approximating the moment-generating function (MGF), and then differentiating it. In this article, we give approximations to the posterior means and variances by combining the MGF method and the Laplace approximations with asymptotic modes. We prove that asymptotic errors of the approximate means and variances are of order n^-2$n^{-2}$ and of order n^-3$n^{-3}$, respectively. Our approximation is closely related to a standard-form approximation, and is given without evaluating the exact posterior mode and third derivatives of the log-likelihood function. The MGF method also improves numerical instability of the fully exponential Laplace approximation for a predictive mean in logistic regression.     

Reduced-rank estimation of the difference between two covariance matrices

We consider m×m$m\times m$ covariance matrices, _Σ1${\Sigma }_1$ and _Σ2${\Sigma }_2$, which satisfy _Σ2-_Σ1=Δ${\Sigma }_2-{\Sigma }_1=\Delta$, where Δ$\Delta$ has a specified rank. Maximum likelihood estimators of _Σ1${\Sigma }_1$ and _Σ2${\Sigma }_2$ are obtained when sample covariance matrices having Wishart distributions are available and rank(Δ)$\text{rank}(\Delta )$ is known. The likelihood ratio statistic for a test about the value of rank(Δ)$\text{rank}(\Delta )$ is also given and some properties of its null distribution are obtained. The methods developed in this paper are illustrated through an example.     

Prediction in moving average processes

For the stationary invertible moving average process of order one with unknown innovation distribution F, we construct root-n   consistent plug-in estimators of conditional expectations E(h(_Xn+1)|_X1,…,_Xn)$E(h(X_{n+1})|X_1,\dots ,X_n)$. More specifically, we give weak conditions under which such estimators admit Bahadur-type representations, assuming some smoothness of h or of F. For fixed h it suffices that h   is locally of bounded variation and locally Lipschitz in _L2(F)$L_2(F)$, and that the convolution of h and F   is continuously differentiable. A uniform representation for the plug-in estimator of the conditional distribution function P(_Xn+1?·|_X1,…,_Xn)$P(X_{n+1}?·|X_1,\dots ,X_n)$ holds if F has a uniformly continuous density. For a smoothed version of our estimator, the Bahadur representation holds uniformly over each class of functions h that have an appropriate envelope and whose shifts are F-Donsker, assuming some smoothness of F. The proofs use empirical process arguments.     

The use of an identity in Anderson for multivariate multiple testing

Consider the model where there are I$I$ independent multivariate normal treatment populations with p×1$p\times 1$ mean vectors _μi${\mathit{\mu }}_i$, i=1,…,I$i=1,\dots ,I$, and covariance matrix Σ$\Sigma$. Independently the (I+1)$(I+1)$st population corresponds to a control and it too is multivariate normal with mean vector _μI+1${\mathit{\mu }}_{I+1}$ and covariance matrix Σ$\Sigma$. Now consider the following two multiple testing problems.