Prediction Error Property of the Lasso Estimator and its Generalization

The lasso procedure is an estimator‐shrinkage and variable selection method. This paper shows that there always exists an interval of tuning parameter values such that the corresponding mean squared prediction error for the lasso estimator is smaller than for the ordinary least squares estimator. For an estimator satisfying some condition such as unbiasedness, the paper defines the corresponding generalized lasso estimator. Its mean squared prediction error is shown to be smaller than that of the estimator for values of the tuning parameter in some interval. This implies that all unbiased estimators are not admissible. Simulation results for five models support the theoretical results.     

Laws of iterated logarithm for quasi-maximum likelihood estimator in generalized linear model

For the case that the expectation of the response variable Y   is correctly specified in the generalized linear model (GLM), under some regular assumptions, we obtain and prove the law of the iterated logarithm and Chung type law of the iterated logarithm for the quasi-maximum likelihood estimator (QMLE) _βn${\beta }_n$ in this model.     

Bond Risk Premia Forecasting: A Simple Approach for Extracting Macroeconomic Information from a Panel of Indicators

We propose a simple but effective estimation procedure to extract the level and the volatility dynamics of a latent macroeconomic factor from a panel of observable indicators. Our approach is based on a multivariate conditionally heteroskedastic exact factor model that can take into account the heteroskedasticity feature shown by most macroeconomic variables and relies on an iterated Kalman filter procedure. In simulations we show the unbiasedness of the proposed estimator and its superiority to different approaches introduced in the literature. Simulation results are confirmed in applications to real inflation data with the goal of forecasting long-term bond risk premia. Moreover, we find that the extracted level and conditional variance of the latent factor for inflation are strongly related to NBER business cycles.     

A Pointwise Estimator for the k-Fold Convolution of a Distribution Function

ABSTRACT

This article is concerned with some parametric and nonparametric estimators for the k-fold convolution of a distribution function. An alternative estimator is proposed and its unbiasedness, asymptotic unbiasedness, and consistency properties are investigated. The asymptotic normality of this estimator is established. Some applications of the estimator are given in renewal processes. Finally, the computational procedures are described and the relative performance of these estimators for small sample sizes is investigated by a simulation study.     

Iterated Feasible Generalized Least-Squares Estimation of Augmented Dynamic Panel Data Models

This article provides the large sample distribution of the iterated feasible generalized least-squares (IFGLS) estimator of an augmented dynamic panel data model. The regressors in the model include lagged values of the dependent variable and may include other explanatory variables that, while exogenous with respect to the time-varying error component, may be correlated with an unobserved time-invariant component. The article compares the finite sample properties of the IFGLS estimator to that of GMM estimators using both simulated and real data and finds that the IFGLS estimator compares favorably.     

Zero variance Markov chain Monte Carlo for Bayesian estimators

Interest is in evaluating, by Markov chain Monte Carlo (MCMC) simulation, the expected value of a function with respect to a, possibly unnormalized, probability distribution. A general purpose variance reduction technique for the MCMC estimator, based on the zero-variance principle introduced in the physics literature, is proposed. Conditions for asymptotic unbiasedness of the zero-variance estimator are derived. A central limit theorem is also proved under regularity conditions. The potential of the idea is illustrated with real applications to probit, logit and GARCH Bayesian models. For all these models, a central limit theorem and unbiasedness for the zero-variance estimator are proved (see the supplementary material available on-line).     

A note on estimating the common mean of k normal distributions and the stein problem

An unbiased estimator for the common mean of k normal distributions is suggested. A necessary and sufficient condition for the estimator Lo have a smaller variance than each sample mean is given. In the case of estimating the common mean vector of k p-variate (p ≤ 3) normal distributions a combined unbiased estimator may be used. We give a class of estimators which are better than the combined estimator when the loss is quadratic and the restriction of unbiasedness is removed.     

On two-stage shrinkage testtimation

The general procedure of two stage shrinkage testimation formulated in Adlce and Gokhale [Commun. Statist.- Theory Meth. 18, 633-627 (1989)] k further generalized and extended to the multiparameter case. Local optimal-ity result of that paper, in the restricted set up of univariate location families and scalar families, is generalized without any restriction on the parametric family. The local unbiasedness result of that paper is also generalized and in addition local risk- unbiasedness is considered. The optimality and risk-unbiasedness results are proved for the usual matrix loss and their validity for an arbitrary quadratic loss deduced as corollaries.     

Moment estimators for the beta-binomial distribution

A new moment estimator of the dispersion parameter of the beta-binomial distribution is proposed. It is derived by the method of moments which is constrained to satisfy the unbiasedness of the estimating equation. It gives a better performance than those of the usual moment estimators and the stabilized moment estimator proposed by Tamura & Young. The bias of the estimator is smaller than that of the maximum likelihood estimate in a wide range of parameter space.     

An extension of Horvitz–Thompson estimator used in adaptive cluster sampling to continuous universe

In this paper, an extension of Horvitz–Thompson estimator used in adaptive cluster sampling to continuous universe is developed. Main new results are presented in theorems. The primary notions of discrete population are transferred to continuous population. First and second order inclusion probabilities for networks are delivered. Horvitz–Thompson estimator for adaptive cluster sampling in continuous universe is constructed. The unbiasedness of the estimator is proven. Variance and unbiased variance estimator are delivered. Finally, the theory is illustrated with an example.     

Abelian and Tauberian Theorems on the Bias of the Hill Estimator

The bias of Hill's estimator for the positive extreme value index of a distribution is investigated in relation to the convergence rate in the regular variation property of the tail function of the common distribution of the sample and the corresponding tail quantile function. Based on the theory of generalized regular variation, natural second-order conditions are proposed which both imply and are implied by convergence of the expectation of Hill's estimator to the extreme value index at certain rates. A comparison with second-order conditions encountered in the literature is made.     

A study of regression calibration in a partially observed stratified Cox model

Regression calibration is a simple method for estimating regression models when covariate data are missing for some study subjects. It consists in replacing an unobserved covariate by an estimator of its conditional expectation given available covariates. Regression calibration has recently been investigated in various regression models such as the linear, generalized linear, and proportional hazards models. The aim of this paper is to investigate the appropriateness of this method for estimating the stratified Cox regression model with missing values of the covariate defining the strata. Despite its practical relevance, this problem has not yet been discussed in the literature. Asymptotic distribution theory is developed for the regression calibration estimator in this setting. A simulation study is also conducted to investigate the properties of this estimator.     

Ranked set sampling theory with selective probability vector

In this paper an estimator of the population mean is introduced by using the idea of selective probability vector and the optimization algorithm of linear programming to find the optimal solution of the selective probability vector under the condition of unbiasedness.     

Asymptotic Properties of the Maximum Quasi-Likelihood Estimator in Quasi-Likelihood Nonlinear Models

Quasi-likelihood nonlinear models (QLNM) are a further extension of generalized linear models by only specifying the expectation and variance functions of the response variable. In this article, some mild regularity conditions are proposed. These regularity conditions, respectively, assure the existence, strong consistency, and the asymptotic normality of the maximum quasi-likelihood estimator (MQLE) in QLNM.     

Laws of the Iterated Logarithm and a Moderate Deviation of MLE for the Proportional Hazards Model with Incomplete Information

In this paper, we obtain a law of iterated logarithm, a Chung-type law of iterated logarithm, and a moderate deviation result of the maximum likelihood estimator (MLE) for the unknown regression parameter vector in a proportional hazards model with incomplete information.     

Whither jackknifing in stein-rule estimation

In multi-parameter ( multivariate ) estimation, the Stein rule provides minimax and admissible estimators , compromising generally on their unbiasedness. On the other hand, the primary aim of jack-knifing is to reduce the bias of an estimator ( without necessarily compromising on its efficacy ), and, at the same time, jackknifing provides an estimator of the sampling variance of the estimator as well. In shrinkage estimation ( where minimization of a suitably defined risk function is the basic goal ), one may wonder how far the bias-reduction objective of jackknifing incorporates the dual objective of minimaxity ( or admissibility ) and estimating the risk of the estimator ? A critical appraisal of this basic role of jackknifing in shrinkage estimation is made here. Restricted, semi-restricted and the usual versions of jackknifed shrinkage estimates are considered and their performance characteristics are studied . It is shown that for Pitman-type ( local ) alternatives, usually, jackkntfing fails to provide a consistent estimator of the ( asymptotic ) risk of the shrinkage estimator, and a degenerate asymptotic situation arises for the usual fixed alternative case.     

A maximum-likelihood estimator with infinite error

The standard error of the maximum-likelihood estimator for 1/μ based on a random sample of size N from the normal distribution N(μ,σ²) is infinite. This could be considered to be a disadvantage.Another disadvantage is that the bias of the estimator is undefined if the integral is interpreted in the usual sense as a Lebesgue integral. It is shown here that the integral expression for the bias can be interpreted in the sense given by the Schwartz theory of generalized functions. Furthermore, an explicit closed form expression in terms of the complex error function is derived. It is also proven that unbiased estimation of 1/μ is impossible.Further results on the maximum-likelihood estimator are investigated, including closed form expressions for the generalized moments and corresponding complete asymptotic expansions. It is observed that the problem can be reduced to a one-parameter problem depending only on , and this holds also for more general location-scale problems. The parameter can be interpreted as a shape parameter for the distribution of the maximum-likelihood estimator.An alternative estimator is suggested motivated by the asymptotic expansion for the bias, and it is argued that the suggested estimator is an improvement. The method used for the construction of the estimator is simple and generalizes to other parametric families.The problem leads to a rediscovery of a generalized mathematical expectation introduced originally by Kolmogorov [1933. Foundations of the Theory of Probability, second ed. Chelsea Publishing Company (1956)]. A brief discussion of this, and some related integrals, is provided. It is in particular argued that the principal value expectation provides a reasonable location parameter in cases where it exists. This does not hold generally for expectations interpreted in the sense given by the Schwartz theory of generalized functions.     

Three modifications of the principle of the minque

Nonnegative estimators for the variance components of a linear model are obtained by ignoring the condition for unbiasedness in the principle of the MINQUE. An estimator is derived when the priori weights are proportional to the variance components. The ordinary sample variance is shown to be the nonnegative MINQUE. Efficiencies of the three estimators are examined for some special cases of the model.     

On the comparison of the pre-test and shrinkage estimators for the univariate normal mean

The estimation of the mean of an univariate normal population with unknown variance is considered when uncertain non-sample prior information is available. Alternative estimators are defined to incorporate both the sample as well as the non-sample information in the estimation process. Some of the important statistical properties of the restricted, preliminary test, and shrinkage estimators are investigated. The performances of the estimators are compared based on the criteria of unbiasedness and mean square error in order to search for a ‘best’ estimator. Both analytical and graphical methods are explored. There is no superior estimator that uniformly dominates the others. However, if the non-sample information regarding the value of the mean is close to its true value, the shrinkage estimator over performs the rest of the estimators. Received: June 19, 1999; revised version: March 23, 2000     

基于半参数方法的模型辅助抽样估计研究

在总结现有模型辅助估计方法的基础上,本文通过构造一种半参数超总体模型,同时结合广义差分估计思想提出一种新型的模型辅助估计量。该估计量比传统的非参数和半参数回归估计利用更少、更易得到的辅助信息,即只需利用和广义回归估计相同的辅助信息,但一般会比广义回归估计拥有更高的估计精度。理论证明了该估计量是渐近设计无偏和设计一致的,其渐近设计均方误差为广义差分估计量的方差。模拟结果显示：其至少与广义回归估计一样好;对于线性程度越低的超总体模型,其估计精度比广义回归估计有越明显的提高;就本文模拟而言,光滑参数在0.04～0.12间适当取值时其会取到相对较好的估计效果。