Stabilizing the Performance of Kurtosis Estimator of Multivariate Data

The estimation of the kurtosis parameter of the underlying distribution plays a central role in many statistical applications. The central theme of the article is to improve the estimation of the kurtosis parameter using a priori information. More specifically, we consider the problem of estimating kurtosis parameter of a multivariate population when some prior information regarding the the parameter is available. The rationale is that the sample estimator of the kurtosis parameter has a large estimation error. In this situation we consider shrinkage and pretest estimation methodologies and reappraise their statistical properties. The estimation based on these strategies yield relatively smaller estimation error in comparison with the sample estimator in the candidate subspace. A large sample theory of the suggested estimators are developed and compared. The results demonstrate that suggested estimators outperform the estimator based on the sample data only in the candidate subspace. In an effort to appreciate the relative behavior of the estimators in a finite sample scenario, a Monte-carlo simulation study is planned and performed. The result of simulation study strongly corroborates the asymptotic result. To illustrate the application of the estimators, some example are showcased based on recently published data.     

Statistical Analysis of Non Linear Least Squares Estimation for Harmonic Signals in Multiplicative and Additive Noise

Abstract

This article concerns the stochastically constrained linear model under a biased assumption. We propose a quasi-stochastically constrained least squares estimator. Furthermore, we provide the expectation of this estimator, demonstrate its consistency and asymptotic normality. In the end of the article, the simulation study of the new estimator shows that it is superior to the least squares estimator, ridge estimator, and the linear constrained estimators under certain conditions by comparing the mean squared errors of these estimators.     

On Local Polynomial Modelling of the Additive Risk Model

The additive risk model provides an alternative modelling technique for failure time data to the proportional hazards model. In this article, we consider the additive risk model with a nonparametric risk effect. We study estimation of the risk function and its derivatives with a parametric and an unspecified baseline hazard function respectively. The resulting estimators are the local likelihood and the local score estimators. We establish the asymptotic normality of the estimators and show that both methods have the same formula for asymptotic bias but different formula for variance. It is found that, in some special cases, the local score estimator is of the same efficiency as the local likelihood estimator though it does not use the information about the baseline hazard function. Another advantage of the local score estimator is that it has a closed form and is easy to implement. Some simulation studies are conducted to evaluate and compare the performance of the two estimators. A numerical example is used for illustration.     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

Asymptotic cumulants of the estimator of the canonical parameter in the exponential family

Asymptotic cumulants of the maximum likelihood estimator of the canonical parameter in the exponential family are obtained up to the fourth order with the added higher-order asymptotic variance. In the case of a scalar parameter, the corresponding results with and without studentization are given. These results are also obtained for the estimators by the weighted score, especially for those using the Jeffreys prior. The asymptotic cumulants are used for reducing bias and mean square error to improve a point estimator and for interval estimation to have higher-order accuracy. It is shown that the kurtosis to squared skewness ratio of the sufficient statistic plays a fundamental role.     

Estimation of partially specified spatial panel data models with fixed-effects

This article extends the spatial panel data regression with fixed-effects to the case where the regression function is partially linear and some regressors may be endogenous or predetermined. Under the assumption that the spatial weighting matrix is strictly exogenous, we propose a sieve two stage least squares (S2SLS) regression. Under some sufficient conditions, we show that the proposed estimator for the finite dimensional parameter is root-N consistent and asymptotically normally distributed and that the proposed estimator for the unknown function is consistent and also asymptotically normally distributed but at a rate slower than root-N. Consistent estimators for the asymptotic variances of the proposed estimators are provided. A small scale simulation study is conducted, and the simulation results show that the proposed procedure has good finite sample performance.     

Testing Hypotheses in Nonparametric Models of Production

Data envelopment analysis (DEA) and free disposal hull (FDH) estimators are widely used to estimate efficiency of production. Practitioners use DEA estimators far more frequently than FDH estimators, implicitly assuming that production sets are convex. Moreover, use of the constant returns to scale (CRS) version of the DEA estimator requires an assumption of CRS. Although bootstrap methods have been developed for making inference about the efficiencies of individual units, until now no methods exist for making consistent inference about differences in mean efficiency across groups of producers or for testing hypotheses about model structure such as returns to scale or convexity of the production set. We use central limit theorem results from our previous work to develop additional theoretical results permitting consistent tests of model structure and provide Monte Carlo evidence on the performance of the tests in terms of size and power. In addition, the variable returns to scale version of the DEA estimator is proved to attain the faster convergence rate of the CRS-DEA estimator under CRS. Using a sample of U.S. commercial banks, we test and reject convexity of the production set, calling into question results from numerous banking studies that have imposed convexity assumptions. Supplementary materials for this article are available online.     

A class of estimators using auxiliary information in sample surveys

A large class of estimators is considered for the mean of a finite population using information on an auxiliary variable. It is shown that members of this class of estimators are asymptotically no more efficient than the linear regression estimator.     

Efficient M-estimators with auxiliary information

This paper introduces a new class of M-estimators based on generalised empirical likelihood (GEL) estimation with some auxiliary information available in the sample. The resulting class of estimators is efficient in the sense that it achieves the same asymptotic lower bound as that of the efficient generalised method of moment (GMM) estimator with the same auxiliary information. The paper also shows that in case of smooth estimating equations the proposed estimators enjoy a small second order bias property compared to both efficient GMM and full GEL estimators. Analytical formulae to obtain bias corrected estimators are also provided. Simulations show that with correctly specified auxiliary information the proposed estimators and in particular those based on empirical likelihood outperform standard M and efficient GMM estimators both in terms of finite sample bias and efficiency. On the other hand with moderately misspecified auxiliary information estimators based on the nonparametric tilting method are typically characterised by the best finite sample properties.     

Generalized ratio-type and ratio-exponential-type estimators for population mean under modified Horvitz-Thompson estimator in adaptive cluster sampling

In the present article, we propose the generalized ratio-type and generalized ratio-exponential-type estimators for population mean in adaptive cluster sampling (ACS) under modified Horvitz-Thompson estimator. The proposed estimators utilize the auxiliary information in combination of conventional measures (coefficient of skewness, coefficient of variation, correlation coefficient, covariance, coefficient of kurtosis) and robust measures (tri-mean, Hodges-Lehmann, mid-range) to increase the efficiency of the estimators. Properties of the proposed estimators are discussed using the first order of approximation. The simulation study is conducted to evaluate the performances of the estimators. The results reveal that the proposed estimators are more efficient than competing estimators for population mean in ACS under both modified Hansen-Hurwitz and Horvitz-Thompson estimators.     

Asymptotic properties of a conditional maximum-likelihood estimator

Inference for a scalar interest parameter in the presence of nuisance parameters is considered in terms of the conditional maximum-likelihood estimator developed by Cox and Reid (1987). Parameter orthogonality is assumed throughout. The estimator is analyzed by means of stochastic asymptotic expansions in three cases: a scalar nuisance parameter, m nuisance parameters from m independent samples, and a vector nuisance parameter. In each case, the expansion for the conditional maximum-likelihood estimator is compared with that for the usual maximum-likelihood estimator. The means and variances are also compared. In each of the cases, the bias of the conditional maximum-likelihood estimator is unaffected by the nuisance parameter to first order. This is not so for the maximum-likelihood estimator. The assumption of parameter orthogonality is crucial in attaining this result. Regardless of parametrization, the difference in the two estimators is first-order and is deterministic to this order.     

The optimal Lp norm estimator in linear regression models

The least squares estimator is usually applied when estimating the parameters in linear regression models. As this estimator is sensitive to departures from normality in the residual distribution, several alternatives have been proposed. The L_p norm estimators is one class of such alternatives. It has been proposed that the kurtosis of the residual distribution be taken into account when a choice of estimator in the L_p norm class is made (i.e. the choice of p). In this paper, the asymtotic variance of the estimators is used as the criterion in the choice of p. It is shown that when this criterion is applied, other characteristics of the residual distribution than the kurtosis (namely moments of order p-2 and 2p-2) are important.     

Estimating and testing effect size from an arbitrary population

In this article, we develop inference tools for an effect size parameter in a paired experiment. A class of estimators is defined that includes natural, shrinkage and shrinkage preliminary test estimators. The shrinkage and preliminary test methods incorporate uncertain prior information on the parameter. This information may be available in the form of a realistic guess on the basis of the experimenter’s knowledge and experience, which can be incorporated into the estimation process to increase the efficiency of the estimator. Asymptotic properties of the proposed estimators are investigated both analytically and computationally. A simulation study is also conducted to assess the performance of the estimators for moderate and large samples. For illustration purposes, the method is applied to a data set.     

Asymptotics of cross-validated risk estimation in estimator selection and performance assessment

Risk estimation is an important statistical question for the purposes of selecting a good estimator (i.e., model selection) and assessing its performance (i.e., estimating generalization error). This article introduces a general framework for cross-validation and derives distributional properties of cross-validated risk estimators in the context of estimator selection and performance assessment. Arbitrary classes of estimators are considered, including density estimators and predictors for both continuous and polychotomous outcomes. Results are provided for general full data loss functions (e.g., absolute and squared error, indicator, negative log density). A broad definition of cross-validation is used in order to cover leave-one-out cross-validation, V-fold cross-validation, Monte Carlo cross-validation, and bootstrap procedures. For estimator selection, finite sample risk bounds are derived and applied to establish the asymptotic optimality of cross-validation, in the sense that a selector based on a cross-validated risk estimator performs asymptotically as well as an optimal oracle selector based on the risk under the true, unknown data generating distribution. The asymptotic results are derived under the assumption that the size of the validation sets converges to infinity and hence do not cover leave-one-out cross-validation. For performance assessment, cross-validated risk estimators are shown to be consistent and asymptotically linear for the risk under the true data generating distribution and confidence intervals are derived for this unknown risk. Unlike previously published results, the theorems derived in this and our related articles apply to general data generating distributions, loss functions (i.e., parameters), estimators, and cross-validation procedures.     

Visualization and Bandwidth Matrix Choice

In this study we compare three estimators of the extreme value index: Pickands estimator, the moment estimator and a maximum likelihood estimator. The estimators are explored both theoretically and by Monte Carlo simulation. We obtain two estimators for large quantiles using Pickands and the maximum likelihood estimators. The latter and one based on the moment estimator are then compared through simulation.     

A comparison of robust estimators based on two types of trimming

The least trimmed squares (LTS) estimator and the trimmed mean (TM) are two well-known trimming-based estimators of the location parameter. Both estimates are used in practice, and they are implemented in standard statistical software (e.g., S-PLUS, R, Matlab, SAS). The breakdown point of each of these estimators increases as the trimming proportion increases, while the efficiency decreases. Here we have shown that for a wide range of distributions with exponential and polynomial tails, TM is asymptotically more efficient than LTS as an estimator of the location parameter, when they have equal breakdown points.     

Pooling reliability functions

In this article large sample pooling procedures for reliability functions of an exponential life testing model is considered. Asymptotic properties of shrinkage estimation procedure subsequent to preliminary tests are developed. It is shown that the proposed estimator possesses substantially snakker asymptotic mean squared error than the usual estimator in a region of the lparameter space. Relative efficiencies of the purposed estimators to the usual estimators are obtained and recommendations of the level of the preliminary tests are provided. Relative dominance picture of the estimators is presented. It is shown that the proposed estimator provides a wider dominance range over usual estimator than the usual preliminary test estimator. More importantly, the size of the preliminary test is meaningful. Simulation studies is also carried out to appraise the performance of the estimators when samples are small.     

Minimum disparity estimation based on combined disparities: Asymptotic results

The maximum likelihood estimator (MLE) is asymptotically efficient for most parametric models under standard regularity conditions, but it has very poor robustness properties. On the other hand some of the minimum disparity estimators like the minimum Hellinger distance estimator (MHDE) have strong robustness features but their small sample efficiency at the model turns out to be very poor compared to the MLE. Methods based on the minimization of some combined disparities can substantially improve their small sample performances without affecting their robustness properties (Park et al., 1995). All studies involving the combined disparity have so far been empirical, and there are no results on the asymptotic properties of these estimators. In view of the usefulness of these procedures this is a major gap in theory, which we try to fill through the present work. Some illustrations of the performance of the estimators and the corresponding tests are also provided.     

Effect of Violation of the Normal Assumption on MI and ML Estimators in the Analysis of Incomplete Data

Asymptotic distributions of normal-theory-based ML/MI estimators are studied in a simple regression model under general distributions with MAR missing data. The asymptotic variance of the ML/MI estimator of residuals’ variance is explicitly derived, from which it follows that the kurtosis of the error distribution primarily affects the asymptotic variance. Results of numerical simulations conducted to study finite sample properties of the estimators, conformed largely to the asymptotic results, and they also indicated interesting findings particularly for small samples, which do not follow from the asymptotic property. It is concluded that the ML estimators perform best in the situation studied here.     

Robust weighted one-way ANOVA: Improved approximation and efficiency

A robust test for the one-way ANOVA model under heteroscedasticity is developed in this paper. The data are assumed to be symmetrically distributed, apart from some outliers, although the assumption of normality may be violated. The test statistic to be used is a weighted sum of squares similar to the Welch [1951. On the comparison of several mean values: an alternative approach. Biometrika 38, 330-336.] test statistic, but any of a variety of robust measures of location and scale for the populations of interest may be used instead of the usual mean and standard deviation. Under the commonly occurring condition that the robust measures of location and scale are asymptotically normal, we derive approximations to the distribution of the test statistic under the null hypothesis and to its distribution under alternative hypotheses. An expression for relative efficiency is derived, thus allowing comparison of the efficiency of the test as a function of the choice of the location and scale estimators used in the test statistic. As an illustration of the theory presented here, we apply it to three commonly used robust location–scale estimator pairs: the trimmed mean with the Winsorized standard deviation; the Huber Proposal 2 estimator pair; and the Hampel robust location estimator with the median absolute deviation.