Shrinkage inverse regression estimation for model-free variable selection

Summary.  The family of inverse regression estimators that was recently proposed by Cook and Ni has proven effective in dimension reduction by transforming the high dimensional predictor vector to its low dimensional projections. We propose a general shrinkage estimation strategy for the entire inverse regression estimation family that is capable of simultaneous dimension reduction and variable selection. We demonstrate that the new estimators achieve consistency in variable selection without requiring any traditional model, meanwhile retaining the root n estimation consistency of the dimension reduction basis. We also show the effectiveness of the new estimators through both simulation and real data analysis.     

Estimation of the Length Distribution of Marine Populations in the Gaussian-multinomial Setting using the Method of Moments

In this paper, the problem of estimation of the length distribution of marine populations in the Gaussian-multinomial model is considered. For the purpose of the mean and covariance parameter estimation, the method of moments estimators are developed. That is, minimum variance linear unbiased estimator for the mean frequency vector is derived and a consistent estimator for the covariance matrix of the length observations is presented. The usefulness of the proposed estimators is illustrated with an analysis of real cod length measurement data.     

On characterizations of pitman closeness of some shrinkage estimators

For the multivariate normal mean (vector) estimation problem, some characterizations of the Pitman closest property of a general class of shrinkage (or Stein-rule) estimators (including the so called positive-rule versions) are studied. Further, for the same model when the parameter is restricted to a positively homogeneous cone, Pitman closeness of restricted shrinkage maximum likelihood estimators is established.     

Pseudo-full likelihood estimation for prospective survival analysis with a general semiparametric shared frailty model: Asymptotic theory

In this work we present a simple estimation procedure for a general frailty model for analysis of prospective correlated failure times. Earlier work showed this method to perform well in a simulation study. Here we provide rigorous large-sample theory for the proposed estimators of both the regression coefficient vector and the dependence parameter, including consistent variance estimators.     

LASSO and shrinkage estimation in Weibull censored regression models

In this paper we address the problem of estimating a vector of regression parameters in the Weibull censored regression model. Our main objective is to provide natural adaptive estimators that significantly improve upon the classical procedures in the situation where some of the predictors may or may not be associated with the response. In the context of two competing Weibull censored regression models (full model and candidate submodel), we consider an adaptive shrinkage estimation strategy that shrinks the full model maximum likelihood estimate in the direction of the submodel maximum likelihood estimate. We develop the properties of these estimators using the notion of asymptotic distributional risk. The shrinkage estimators are shown to have higher efficiency than the classical estimators for a wide class of models. Further, we consider a LASSO type estimation strategy and compare the relative performance with the shrinkage estimators. Monte Carlo simulations reveal that when the true model is close to the candidate submodel, the shrinkage strategy performs better than the LASSO strategy when, and only when, there are many inactive predictors in the model. Shrinkage and LASSO strategies are applied to a real data set from Veteran's administration (VA) lung cancer study to illustrate the usefulness of the procedures in practice.     

Shrinkage ridge regression in partial linear models

ABSTRACT

In this paper, shrinkage ridge estimator and its positive part are defined for the regression coefficient vector in a partial linear model. The differencing approach is used to enjoy the ease of parameter estimation after removing the non parametric part of the model. The exact risk expressions in addition to biases are derived for the estimators under study and the region of optimality of each estimator is exactly determined. The performance of the estimators is evaluated by simulated as well as real data sets.     

Simultaneous estimation of restricted location parameters based on permutation and sign-change

The problem of simultaneously estimating location parameters is addressed, where the vector of location parameters belongs to a polyhedral cone including simple order, tree order and positive orthant restrictions and so forth. This paper proposes modified estimators based on orthogonal transformations such as sign-change and permutation and proves that, in a multivariate location family, the modified estimators are minimax under quadratic loss. Shrinkage minimax estimators improving on the modified estimators are obtained for a restricted mean vector of spherically symmetric distribution. An application of sign-change transformation is also given in estimation of a bounded normal mean.     

Minimum distance estimation in linear regression with strong mixing errors

Abstract

Minimum distance estimation on the linear regression model with independent errors is known to yield an efficient and robust estimator. We extend the method to the model with strong mixing errors and obtain an estimator of the vector of the regression parameters. The goal of this article is to demonstrate the proposed estimator still retains efficiency and robustness. To that end, this article investigates asymptotic distributional properties of the proposed estimator and compares it with other estimators. The efficiency and the robustness of the proposed estimator are empirically shown, and its superiority over the other estimators is established.     

固定效应部分线性变系数面板模型的快速有效估计

本文将最小二乘支持向量机(LSSVM) 和二次推断函数法(QIF) 相结合，为个体内具有相关结构的固定效应部分线性变系数面板模型提供了一种新的快速估计方法；在一定的正则条件下，论证了参数估计量的渐近正态性和非参数估计量的收敛速度；采用Monte Carlo模拟考察了估计方法在有限样本下的表现并将估计技术应用于现实数据分析。该方法不仅保证了估计的有效性和统计推断力，而且程序运行速度得到较大幅度提升。     

Improved pretest nonparametric estimation in a multivariate regression model

Shrinkage pretest nonparametric estimation of the location parameter vector in a multivariate regression model is considered when nonsample information (NSI) about the regression parameters is available. By using the quadratic risk criterion, the dominance of the pretest estimators over the usual estimators has been investigated. We demonstrate analytically and computationally that the proposed improved pretest estimator establishes a wider dominance range for the parameter under consideration than that of the usual pretest estimator in which it is superior over the unrestricted estimator.     

A Note on Support Vector Density Estimation for the Deconvolution Problem

The problem of support vector density estimation is studied when the sample observations are contaminated with random noise. A procedure based on support vector method and the Fourier transform is presented and is compared with kernel density estimators by the simulation study.     

第三类Tobit模型的半参数估计方法

本文给出第三类Tobit模型的一种新的半参数估计方法。在独立性假设下,利用主方程和选择方程中可观察受限因变量的条件生存函数所满足的关系式,构造第三类Tobit模型参数的一步联立估计量。在已知选择方程中参数一致性估计量的条件下,这种方法也可用于构造主方程模型参数 的两步估计量。本文证明了所提出的一步联立估计量和两步估计量的一致性和渐近正态性。实验模拟表明,我们提出的估计量在有限样本下具有良好表现,且一步联立估计量的有限样本表现优于或接近于Chen（1997）的估计量。     

Maximum likelihood estimation of variance components of heteroscedastic random anova model

the estimation of variance components of heteroscedastic random model is discussed in this paper. Maximum Likelihood (ML) is described for one-way heteroscedastic random models. The proportionality condition that cell variance is proportional to the cell sample size, is used to eliminate the efffect of heteroscedasticity. The algebraic expressions of the estimators are obtained for the model. It is seen that the algebraic expressions of the estimators depend mainly on the inverse of the variance-covariance matrix of the observation vector. So, the variance-covariance matrix is obtained and the formulae for the inversions are given. A Monte Carlo study is conducted. Five different variance patterns with different numbers of cells are considered in this study. For each variance pattern, 1000 Monte Carlo samples are drawn. Then the Monte Carlo biases and Monte Carlo MSE’s of the estimators of variance components are calculated. In respect of both bias and MSE, the Maximum Likelihood (ML) estimators of variance components are found to be sufficiently good.     

A family of admissible minimax estimators of the mean of a multivariate,normal distribution

Let X has a p-dimensional normal distribution with mean vector θ and identity covariance matrix I. In a compound decision problem consisting of squared-error estimation of θ, Strawderman (1971) placed a Beta (α, 1) prior distribution on a normal class of priors to produce a family of Bayes minimax estimators. We propose an incomplete Gamma(α, β) prior distribution on the same normal class of priors to produce a larger family of Bayes minimax estimators. We present the results of a Monte Carlo study to demonstrate the reduced risk of our estimators in comparison with the Strawderman estimators when θ is away from the zero vector.     

Local regression for vector responses

We explore a class of vector smoothers based on local polynomial regression for fitting nonparametric regression models which have a vector response. The asymptotic bias and variance for the class of estimators are derived for two different ways of representing the variance matrices within both a seemingly unrelated regression and a vector measurement error framework. We show that the asymptotic behaviour of the estimators is different in these four cases. In addition, the placement of the kernel weights in weighted least squares estimators is very important in the seeming unrelated regressions problem (to ensure that the estimator is asymptotically unbiased) but not in the vector measurement error model. It is shown that the component estimators are asymptotically uncorrelated in the seemingly unrelated regressions model but asymptotically correlated in the vector measurement error model. These new and interesting results extend our understanding of the problem of smoothing dependent data.     

Use of prior information in the form of interval constraints for the improved estimation of linear regression models with some missing responses

We consider the estimation of coefficients in a linear regression model when some responses on the study variable are missing and some prior information in the form of lower and upper bounds for the average values of missing responses is available. Employing the mixed regression framework, we present five estimators for the vector of regression coefficients. Their exact as well as asymptotic properties are discussed and superiority of one estimator over the other is examined.     

Nonparametric vector autoregression

We consider a vector conditional heteroscedastic autoregressive nonlinear (CHARN) model in which both the conditional mean and the conditional variance (volatility) matrix are unknown functions of the past. Nonparametric estimators of these functions are constructed based on local polynomial fitting. We examine the rates of convergence of these estimators and give a result on their asymptotic normality. These results are applied to estimation of volatility matrices in foreign exchange markets. Estimation of the conditional covariance surface for the Deutsche Mark/US Dollar (DEM/USD) and Deutsche Mark/British Pound (DEM/GBP) daily returns show negative correlation when the two series have opposite lagged values and positive correlation elsewhere. The relation of our findings to the capital asset pricing model is discussed.     

On second-order admissibilities of the estimators of gamma shape vector

Simultaneous estimation problem of gamma shape vector is considered.First, it is shown that the maximum likelihood estimator (MLE), the bias corrected MLE, and the conditional MLE of shape vector are second-order inadmissible. Second, these estimators are improved up to the second order. Finally, we identify whether these improved estimators are second-order admissible or not. Simulation studies are also given.     

Shrinkage and LASSO strategies in high-dimensional heteroscedastic models

ABSTRACT

In this paper, we consider the estimation problem of the parameter vector in the linear regression model with heteroscedastic errors. First, under heteroscedastic errors, we study the performance of shrinkage-type estimators and their performance as compared to theunrestricted and restricted least squares estimators. In order to accommodate the heteroscedastic structure, we generalize an identity which is useful in deriving the risk function. Thanks to the established identity, we prove that shrinkage estimators dominate the unrestricted estimator. Finally, we explore the performance of high-dimensional heteroscedastic regression estimator as compared to classical LASSO and shrinkage estimators.     

Mean squared error matrix comparisons between biased estimators — An overview of recent results

In the following we give a systematic report on mean squared error matrix comparisons of competing biased estimators. Our approach is quite general: The parameter vector to be estimated is assumed to belong to a subset of the p-dimensional Euclidean space. However, to illustrate our results, we shall pay attention to the linear regression model where biased estimation is very popular. Especially we are interested in generalized ridge and restricted least squares estimation. Support by the Deutsche Forschungsgemeinschaft (DFG, grant number TR 253/1-1 (s*) and grant number STA 284/1-1 (s**)) is gratefully acknowledged.