Efficient estimation for time series following generalized linear models

In this paper, we consider James–Stein shrinkage and pretest estimation methods for time series following generalized linear models when it is conjectured that some of the regression parameters may be restricted to a subspace. Efficient estimation strategies are developed when there are many covariates in the model and some of them are not statistically significant. Statistical properties of the pretest and shrinkage estimation methods including asymptotic distributional bias and risk are developed. We investigate the relative performances of shrinkage and pretest estimators with respect to the unrestricted maximum partial likelihood estimator (MPLE). We show that the shrinkage estimators have a lower relative mean squared error as compared to the unrestricted MPLE when the number of significant covariates exceeds two. Monte Carlo simulation experiments were conducted for different combinations of inactive covariates and the performance of each estimator was evaluated in terms of its mean squared error. The practical benefits of the proposed methods are illustrated using two real data sets.     

Model selection and post estimation based on a pretest for logistic regression models

ABSTRACT

This article addresses the problem of parameter estimation of the logistic regression model under subspace information via linear shrinkage, pretest, and shrinkage pretest estimators along with the traditional unrestricted maximum likelihood estimator and restricted estimator. We developed an asymptotic theory for the linear shrinkage and pretest estimators and compared their relative performance using the notion of asymptotic distributional bias and asymptotic quadratic risk. The analytical results demonstrated that the proposed estimation strategies outperformed the classical estimation strategies in a meaningful parameter space. Detailed Monte-Carlo simulation studies were conducted for different combinations and the performance of each estimation method was evaluated in terms of simulated relative efficiency. The results of the simulation study were in strong agreement with the asymptotic analytical findings. Two real-data examples are also given to appraise the performance of the estimators.     

Shrinkage and penalized estimators in weighted least absolute deviations regression models

In this paper, we consider the estimation problem of the weighted least absolute deviation (WLAD) regression parameter vector when there are some outliers or heavy-tailed errors in the response and the leverage points in the predictors. We propose the pretest and James–Stein shrinkage WLAD estimators when some of the parameters may be subject to certain restrictions. We derive the asymptotic risk of the pretest and shrinkage WLAD estimators and show that if the shrinkage dimension exceeds two, the asymptotic risk of the shrinkage WLAD estimator is strictly less than the unrestricted WLAD estimator. On the other hand, the risk of the pretest WLAD estimator depends on the validity of the restrictions on the parameters. Furthermore, we study the WLAD absolute shrinkage and selection operator (WLAD-LASSO) and compare its relative performance with the pretest and shrinkage WLAD estimators. A simulation study is conducted to evaluate the performance of the proposed estimators relative to that of the unrestricted WLAD estimator. A real-life data example using body fat study is used to illustrate the performance of the suggested estimators.     

Combining poisson means

The problem of estimating the Poisson mean is considered based on the two samples in the presence of uncertain prior information (not in the form of distribution) that two independent random samples taken from two possibly identical Poisson populations. The parameter of interest is λ₁ from population I. Three estimators, i.e. the unrestricted estimator, restricted estimator and preliminary test estimator are proposed. Their asymptotic mean squared errors are derived and compared; parameter regions have been found for which restricted and preliminary test estimators are always asymptotically more efficient than the classical estimator. The relative dominance picture of the estimators is presented. Maximum and minimum asymptotic efficiencies of the estimators relative to the classical estimator are tabulated. A max-min rule for the size of the preliminary test is also discussed. A Monte Carlo study is presented to compare the performance of the estimator with that of Kale and Bancroft (1967).     

Pooling means under uncertain prior information with application to discrete distributions

The problem of pooling means is considered based on two samples in presence of the uncertain prior information that these samples are taken from possibly identical populations. Two discrete models, Poisson and binomial are considered in particular. Three estimators, i.e. the unrestricted estimator, shrinkage restricted estimator and estimators based on preliminary test are proposed. Their asymptotic mean squared errors are derived and compared. It is demonstrated via asymptotic results that the range of the parameter space in which shrinkage preliminary test estimator dominates the unrestricted estimator is wider than that of the usual preliminary test estimator. A Monte Carlo study for Poisson model is presented to compare the performance of the estimators for small samples.     

On the estimation of reliabilty function in a Weibull lifetime distribution

The estimation of the reliability function of the Weibull lifetime model is considered in the presence of uncertain prior information (not in the form of prior distribution) on the parameter of interest. This information is assumed to be available in some sort of a realistic conjecture. In this article, we focus on how to combine sample and non-sample information together in order to achieve improved estimation performance. Three classes of point estimatiors, namely, the unrestricted estimator, the shrinkage estimator and shrinkage preliminary test estimator (SPTE) are proposed. Their asymptotic biases and mean-squared errors are derived and compared. The relative dominance picture of the estimators is presented. Interestingly, the proposed SPTE dominates the unrestricted estimator in a range that is wider than that of the usual preliminary test estimator. A small-scale simulation experiment is used to examine the small sample properties of the proposed estimators. Our simulation investigations have provided strong evidence that corroborates with asymptotic theory. The suggested estimation methods are applied to a published data set to illustrate the performance of the estimators in a real-life situation.     

Improved pretest nonparametric estimation in a multivariate regression model

Shrinkage pretest nonparametric estimation of the location parameter vector in a multivariate regression model is considered when nonsample information (NSI) about the regression parameters is available. By using the quadratic risk criterion, the dominance of the pretest estimators over the usual estimators has been investigated. We demonstrate analytically and computationally that the proposed improved pretest estimator establishes a wider dominance range for the parameter under consideration than that of the usual pretest estimator in which it is superior over the unrestricted estimator.     

Model selection and parameter estimation of a multinomial logistic regression model

In the multinomial regression model, we consider the methodology for simultaneous model selection and parameter estimation by using the shrinkage and LASSO (least absolute shrinkage and selection operation) [R. Tibshirani, Regression shrinkage and selection via the LASSO, J. R. Statist. Soc. Ser. B 58 (1996), pp. 267–288] strategies. The shrinkage estimators (SEs) provide significant improvement over their classical counterparts in the case where some of the predictors may or may not be active for the response of interest. The asymptotic properties of the SEs are developed using the notion of asymptotic distributional risk. We then compare the relative performance of the LASSO estimator with two SEs in terms of simulated relative efficiency. A simulation study shows that the shrinkage and LASSO estimators dominate the full model estimator. Further, both SEs perform better than the LASSO estimators when there are many inactive predictors in the model. A real-life data set is used to illustrate the suggested shrinkage and LASSO estimators.     

Meta-analysis,pretest, and shrinkage estimation of kurtosis parameters

An asymptotic theory for the improved estimation of kurtosis parameter vector is developed for multi-sample case using uncertain prior information (UPI) that several kurtosis parameters are the same. Meta-analysis is performed to obtain pooled estimator, as it is a statistical methodology for pooling quantitative evidence. Pooled estimator is a good choice when assumption of homogeneity holds but it becomes inconsistent as assumption violates, therefore pretest and Stein-type shrinkage estimators are proposed as they combine sample and nonsample information in a superior way. Asymptotic properties of suggested estimators are discussed and their risk comparisons are also mentioned.     

Nonparametric Shrinkage Estimation for Aalen's Additive Hazards Model

Aalen's nonparametric additive model in which the regression coefficients are assumed to be unspecified functions of time is a flexible alternative to Cox's proportional hazards model when the proportionality assumption is in doubt. In this paper, we incorporate a general linear hypothesis into the estimation of the time‐varying regression coefficients. We combine unrestricted least squares estimators and estimators that are restricted by the linear hypothesis and produce James‐Stein‐type shrinkage estimators of the regression coefficients. We develop the asymptotic joint distribution of such restricted and unrestricted estimators and use this to study the relative performance of the proposed estimators via their integrated asymptotic distributional risks. We conduct Monte Carlo simulations to examine the relative performance of the estimators in terms of their integrated mean square errors. We also compare the performance of the proposed estimators with a recently devised LASSO estimator as well as with ridge‐type estimators both via simulations and data on the survival of primary billiary cirhosis patients.     

Application of shrinkage estimation in linear regression models with autoregressive errors

In this paper, we consider the shrinkage and penalty estimation procedures in the linear regression model with autoregressive errors of order p when it is conjectured that some of the regression parameters are inactive. We develop the statistical properties of the shrinkage estimation method including asymptotic distributional biases and risks. We show that the shrinkage estimators have a significantly higher relative efficiency than the classical estimator. Furthermore, we consider the two penalty estimators: least absolute shrinkage and selection operator (LASSO) and adaptive LASSO estimators, and numerically compare their relative performance with that of the shrinkage estimators. A Monte Carlo simulation experiment is conducted for different combinations of inactive predictors and the performance of each estimator is evaluated in terms of the simulated mean-squared error. This study shows that the shrinkage estimators are comparable to the penalty estimators when the number of inactive predictors in the model is relatively large. The shrinkage and penalty methods are applied to a real data set to illustrate the usefulness of the procedures in practice.     

Shrinkage and pretest estimators for longitudinal data analysis under partially linear models

In this paper, we develop marginal analysis methods for longitudinal data under partially linear models. We employ the pretest and shrinkage estimation procedures to estimate the mean response parameters as well as the association parameters, which may be subject to certain restrictions. We provide the analytic expressions for the asymptotic biases and risks of the proposed estimators, and investigate their relative performance to the unrestricted semiparametric least-squares estimator (USLSE). We show that if the dimension of association parameters exceeds two, the risk of the shrinkage estimators is strictly less than that of the USLSE in most of the parameter space. On the other hand, the risk of the pretest estimator depends on the validity of the restrictions of association parameters. A simulation study is conducted to evaluate the performance of the proposed estimators relative to that of the USLSE. A real data example is applied to illustrate the practical usefulness of the proposed estimation procedures.     

Stein-type estimation using ranked set sampling

In this study, we consider the application of the James–Stein estimator for population means from a class of arbitrary populations based on ranked set sample (RSS). We consider a basis for optimally combining sample information from several data sources. We succinctly develop the asymptotic theory of simultaneous estimation of several means for differing replications based on the well-defined shrinkage principle. We showcase that a shrinkage-type estimator will have, under quadratic loss, a substantial risk reduction relative to the classical estimator based on simple random sample and RSS. Asymptotic distributional quadratic biases and risks of the shrinkage estimators are derived and compared with those of the classical estimator. A simulation study is used to support the asymptotic result. An over-riding theme of this study is that the shrinkage estimation method provides a powerful extension of its traditional counterpart for non-normal populations. Finally, we will use a real data set to illustrate the computation of the proposed estimators.     

Asymptotic normality of kernel estimators based upon incomplete data

In this paper, we are concerned with nonparametric estimation of the density and the failure rate functions of a random variable X which is at risk of being censored. First, we establish the asymptotic normality of a kernel density estimator in a general censoring setup. Then, we apply our result in order to derive the asymptotic normality of both the density and the failure rate estimators in the cases of right, twice and doubly censored data. Finally, the performance and the asymptotic Gaussian behaviour of the studied estimators, based on either doubly or twice censored data, are illustrated through a simulation study.     

Polynomial Histograms for Multivariate Density and Mode Estimation

Abstract. We consider the problem of efficiently estimating multivariate densities and their modes for moderate dimensions and an abundance of data. We propose polynomial histograms to solve this estimation problem. We present first‐ and second‐order polynomial histogram estimators for a general d‐dimensional setting. Our theoretical results include pointwise bias and variance of these estimators, their asymptotic mean integrated square error (AMISE), and optimal binwidth. The asymptotic performance of the first‐order estimator matches that of the kernel density estimator, while the second order has the faster rate of O(n^?6/(d+6)). For a bivariate normal setting, we present explicit expressions for the AMISE constants which show the much larger binwidths of the second order estimator and hence also more efficient computations of multivariate densities. We apply polynomial histogram estimators to real data from biotechnology and find the number and location of modes in such data.     

Estimation of Pr( x<y ) in the exponential case with common location parameter

This paper considers the problem of estimating the probability P = Pr(X < Y) when X and Y are independent exponential random variables with unequal scale parameters and a common location parameter. Uniformly minimum variance unbiased estimator of P is obtained. The asymptotic distribution of the maximum likelihood estimator is obtained and then the asymptotic equivalence of the two estimators is established. Performance of the two estimators for moderate sample sizes is studied by Monte Carlo simulation. An approximate interval estimator is also obtained.     

Non-penalty shrinkage estimation of random effect models for longitudinal data with AR(1) errors

In this paper, we consider the non-penalty shrinkage estimation method of random effect models with autoregressive errors for longitudinal data when there are many covariates and some of them may not be active for the response variable. In observational studies, subjects are followed over equally or unequally spaced visits to determine the continuous response and whether the response is associated with the risk factors/covariates. Measurements from the same subject are usually more similar to each other and thus are correlated with each other but not with observations of other subjects. To analyse this data, we consider a linear model that contains both random effects across subjects and within-subject errors that follows autoregressive structure of order 1 (AR(1)). Considering the subject-specific random effect as a nuisance parameter, we use two competing models, one includes all the covariates and the other restricts the coefficients based on the auxiliary information. We consider the non-penalty shrinkage estimation strategy that shrinks the unrestricted estimator in the direction of the restricted estimator. We discuss the asymptotic properties of the shrinkage estimators using the notion of asymptotic biases and risks. A Monte Carlo simulation study is conducted to examine the relative performance of the shrinkage estimators with the unrestricted estimator when the shrinkage dimension exceeds two. We also numerically compare the performance of the shrinkage estimators to that of the LASSO estimator. A longitudinal CD4 cell count data set will be used to illustrate the usefulness of shrinkage and LASSO estimators.     

Relative performance of stein-rule and preliminary test estimators in linear models least squares theory

In the classical (univariare) linear model, bearing the plausibility of a subset of the regression parameters being close to a pivot, shrinkage least squares estimation of the complementary subset is considered. Based on the usual James-Stein rule, shrinkage least squares estimators are constructed, and under an asymptotic setup (allowing the shrinkage parameters to be 'close to ' the pivot), the relative performance of such estimators and the prcliminary test estimators is studied. In this context, the normality of the errors is also avoided under the same asymptotic setup. None of the shrinkage and preliminary test estimators may dominate the other (in the light of the asymptotic distributional risk criterion, as has been developed here), though each of them fares well relative to the classical least squeres estimator. The chice of the shrinkage factor is also examined properly.     

Performance of some new preliminary test ridge regression estimators and their properties

The problem of estimation of the regression coefficients in a multiple regression model is considered under multicollinearity situation when it is suspected that the regression coefficients may be restricted to a subspace. We present the estimators of the regression coefficients combining the idea of preliminary test and ridge regression methodology. Accordingly, we consider three estimators, namely, the unrestricted ridge regression estimator (URRE), the restricted ridge regression estimator (RRRE), and finally, the preliminary test ridge regression estimator (PTRRE). The biases, variancematrices and mean square errors (mse) of the estimators are derived and compared with the usual estimators. Regions of optimality of the estimators are determined by studying the mse criterion. The conditions of superiority of the estimators over the traditional estimators as in Saleh and Han (1990) and Ali and Saleh (1991) have also been discussed.     

Preliminary test and Bayes Estimation of a Location Parameter Under Blinex Loss

In this article, the preliminary test estimator is considered under the BLINEX loss function. The problem under consideration is the estimation of the location parameter from a normal distribution. The risk under the null hypothesis for the preliminary test estimator, the exact risk function for restricted maximum likelihood and approximated risk function for the unrestricted maximum likelihood estimator, are derived under BLINEX loss and the different risk structures are compared to one another both analytically and computationally. As a motivation on the use of BLINEX rather than LINEX, the risk for the preliminary test estimator under BLINEX loss is compared to the risk of the preliminary test estimator under LINEX loss and it is shown that the LINEX expected loss is higher than BLINEX expected loss. Furthermore, two feasible Bayes estimators are derived under BLINEX loss, and a feasible Bayes preliminary test estimator is defined and compared to the classical preliminary test estimator.