Small area estimation under random regression coefficient models

Statistical agencies are interested to report precise estimates of linear parameters from small areas. This goal can be achieved by using model-based inference. In this sense, random regression coefficient models provide a flexible way of modelling the relationship between the target and the auxiliary variables. Because of this, empirical best linear unbiased predictor (EBLUP) estimates based on these models are introduced. A closed-formula procedure to estimate the mean-squared error of the EBLUP estimators is also given and empirically studied. Results of several simulation studies are reported as well as an application to the estimation of household normalized net annual incomes in the Spanish Living Conditions Survey.     

Bias-corrected confidence bands in nonparametric regression

Bias-corrected confidence bands for general nonparametric regression models are considered. We use local polynomial fitting to construct the confidence bands and combine the cross-validation method and the plug-in method to select the bandwidths. Related asymptotic results are obtained. Our simulations show that confidence bands constructed by local polynomial fitting have much better coverage than those constructed by using the Nadaraya–Watson estimator. The results are also applicable to nonparametric autoregressive time series models.     

A two-parameter estimator in the negative binomial regression model

In this article, a two-parameter estimator is proposed to combat multicollinearity in the negative binomial regression model. The proposed two-parameter estimator is a general estimator which includes the maximum likelihood (ML) estimator, the ridge estimator (RE) and the Liu estimator as special cases. Some properties on the asymptotic mean-squared error (MSE) are derived and necessary and sufficient conditions for the superiority of the two-parameter estimator over the ML estimator and sufficient conditions for the superiority of the two-parameter estimator over the RE and the Liu estimator in the asymptotic mean-squared error (MSE) matrix sense are obtained. Furthermore, several methods and three rules for choosing appropriate shrinkage parameters are proposed. Finally, a Monte Carlo simulation study is given to illustrate some of the theoretical results.     

Interpenetrating subsampling regression estimation with or without jackknifing

In this paper four regression estimators are considered for a finite population total based on interpenetrating subsamples, two of which are with jackknifing and the other two are without jackknifing. Both theoretical and empirical comparisons of the four proposed estimators are done with respect to bias, variance and mean square error.     

Empirical likelihood-based inference in nonlinear regression models with missing responses at random

This paper investigates the estimations of regression parameters and response mean in nonlinear regression models in the presence of missing response variables that are missing with missingness probabilities depending on covariates. We propose four empirical likelihood (EL)-based estimators for the regression parameters and the response mean. The resulting estimators are shown to be consistent and asymptotically normal under some general assumptions. To construct the confidence regions for the regression parameters as well as the response mean, we develop four EL ratio statistics, which are proven to have the χ² distribution asymptotically. Simulation studies and an artificial data set are used to illustrate the proposed methodologies. Empirical results show that the EL method behaves better than the normal approximation method and that the coverage probabilities and average lengths depend on the selection probability function.     

A nonparametric estimator of the number of classes based on a stratified random sample

Under stratified random sampling, we develop a k^th-order bootstrap bias-corrected estimator of the number of classes θ which exist in a study region. This research extends Smith and van Belle’s (1984) first-order bootstrap bias-corrected estimator under simple random sampling. Our estimator has applicability for many settings including: estimating the number of animals when there are stratified capture periods, estimating the number of species based on stratified random sampling of subunits (say, quadrats) from the region, and estimating the number of errors/defects in a product based on observations from two or more types of inspectors. When the differences between the strata are large, utilizing stratified random sampling and our estimator often results in superior performance versus the use of simple random sampling and its bootstrap or jackknife [Burnham and Overton (1978)] estimator. The superior performance is often associated with more observed classes, and we provide insights into optimal designation of the strata and optimal allocation of sample sectors to strata.     

Quantile regression for interval censored data

As direct generalization of the quantile regression for complete observed data, an estimation method for quantile regression models with interval censored data is proposed, and the property of consistency is obtained. The property of asymptotic normality is also established with a bias converging to zero, and to reduce the bias, two bias correction methods are proposed. Methods proposed in this paper do not require the censoring vectors to be identically distributed, and can be applied to models with various covariates. Simulation results show that the proposed methods work well.     

Improved maximum-likelihood estimation in a regression model with general parametrization

We analyse the finite-sample behaviour of two second-order bias-corrected alternatives to the maximum-likelihood estimator of the parameters in a multivariate normal regression model with general parametrization proposed by Patriota and Lemonte [A.G. Patriota and A.J. Lemonte, Bias correction in a multivariate regression model with genereal parameterization, Stat. Prob. Lett. 79 (2009), pp. 1655–1662]. The two finite-sample corrections we consider are the conventional second-order bias-corrected estimator and the bootstrap bias correction. We present the numerical results comparing the performance of these estimators. Our results reveal that analytical bias correction outperforms numerical bias corrections obtained from bootstrapping schemes.     

Sparse alternatives to ridge regression: a random effects approach

In a calibration of near-infrared (NIR) instrument, we regress some chemical compositions of interest as a function of their NIR spectra. In this process, we have two immediate challenges: first, the number of variables exceeds the number of observations and, second, the multicollinearity between variables are extremely high. To deal with the challenges, prediction models that produce sparse solutions have recently been proposed. The term ‘sparse’ means that some model parameters are zero estimated and the other parameters are estimated naturally away from zero. In effect, a variable selection is embedded in the model to potentially achieve a better prediction. Many studies have investigated sparse solutions for latent variable models, such as partial least squares and principal component regression, and for direct regression models such as ridge regression (RR). However, in the latter, it mainly involves an L₁ norm penalty to the objective function such as lasso regression. In this study, we investigate new sparse alternative models for RR within a random effects model framework, where we consider Cauchy and mixture-of-normals distributions on the random effects. The results indicate that the mixture-of-normals model produces a sparse solution with good prediction and better interpretation. We illustrate the methods using NIR spectra datasets from milk and corn specimens.     

Double-smoothing for bias reduction in local linear regression

Local linear regression involves fitting a straight line segment over a small region whose midpoint is the target point x, and the local linear estimate at x   is the estimated intercept of that straight line segment, with an asymptotic bias of order h²$h^2$ and variance of order (nh)^-1$(\mathit{nh}{)}^{-1}$ (h is the bandwidth). In this paper, we propose a new estimator, the double-smoothing local linear estimator, which is constructed by integrally combining all fitted values at x   of local lines in its neighborhood with another round of smoothing. The proposed estimator attempts to make use of all information obtained from fitting local lines. Without changing the order of variance, the new estimator can reduce the bias to an order of h⁴$h^4$. The proposed estimator has better performance than local linear regression in situations with considerable bias effects; it also has less variability and more easily overcomes the sparse data problem than local cubic regression. At boundary points, the proposed estimator is comparable to local linear regression. Simulation studies are conducted and an ethanol example is used to compare the new approach with other competitive methods.     

Post-hoc analyses in multiple regression based on prediction error

A well-known problem in multiple regression is that it is possible to reject the hypothesis that all slope parameters are equal to zero, yet when applying the usual Student's T-test to the individual parameters, no significant differences are found. An alternative strategy is to estimate prediction error via the 0.632 bootstrap method for all models of interest and declare the parameters associated with the model that yields the smallest prediction error to differ from zero. The main results in this paper are that this latter strategy can have practical value versus Student's T; replacing squared error with absolute error can be beneficial in some situations and replacing least squares with an extension of the Theil-Sen estimator can substantially increase the probability of identifying the correct model under circumstances that are described.     

Modified regression estimators using robust regression methods and covariance matrices in stratified random sampling

Abstract

This article proposes new regression-type estimators by considering Tukey-M, Hampel M, Huber MM, LTS, LMS and LAD robust methods and MCD and MVE robust covariance matrices in stratified sampling. Theoretically, we obtain the mean square error (MSE) for these estimators. We compare the efficiencies based on MSE equations, between the proposed estimators and the traditional combined and separate regression estimators. As a result of these comparisons, we observed that our proposed estimators give more efficient results than traditional approaches. And, these theoretical results are supported with the aid of numerical examples and simulation based on data sets that include outliers.     

Efficiency of an almost unbiased two-parameter estimator in linear regression model

This paper deals with parameter estimation in the linear regression model and an almost unbiased two-parameter estimator is introduced. The performance of this new estimator over the ordinary least-squares estimator and the two-parameter estimator [M.R. Özkale and S. Kaçiranlar, The restricted and unrestricted two-parameter estimator, Comm. Statist. Theory Methods 36 (2007), pp. 2707–2725] in terms of scalar mean-squared error criterion is investigated and a simulation study is done.     

Evaluation of the bias of the isotonic regression

The systematic error (bias) of the isotonic regression analysis of temporal spacings between failure events is investigated by means of numerical simulation. Spacings that are sampled from an exponential distribution with a constant failure rate (CFR) arc subjected to an isotonic regression search for a declining failure rate (DFR). The results indicate a considerable declining trend (bias) that is imposed upon these CFR-data by isotonic regression analysis. The corresponding results for an increasing trend can be readily obtained through transformation. For practical applications, the results of 100,000 simulations have been approximated by simple analytical expressions. For the evaluation of a trend in a specific set of isotonized spacings (or rates) the results of the latter analysis can be compared with the isotonic bias of a set of CFR data for the same number of events. Alternatively, the specific set of isotonized spacings can be suitably related to the corresponding isotonized CFR data to reduce the bias by largely eliminating the CFR contribution.     

Mean square error behavior for prediction in linear regression models

For the problem of individual prediction in linear regression models, that is, estimation of a linear combination of regression coefficients, mean square error behavior of a general class of adaptive predictors is examined.     

Quantile regression for panel data models with fixed effects under random censoring

Abstract

The locally weighted censored quantile regression approach is proposed for panel data models with fixed effects, which allows for random censoring. The resulting estimators are obtained by employing the fixed effects quantile regression method. The weights are selected either parametrically, semi-parametrically or non-parametrically. The large panel data asymptotics are used in an attempt to cope with the incidental parameter problem. The consistency and limiting distribution of the proposed estimator are also derived. The finite sample performance of the proposed estimators are examined via Monte Carlo simulations.     

Random gradient boosting for predicting conditional quantiles

Gradient Boosting (GB) was introduced to address both classification and regression problems with great power. People have studied the boosting with L2 loss intensively both in theory and practice. However, the L2 loss is not proper for learning distributional functionals beyond the conditional mean such as conditional quantiles. There are huge amount of literatures studying conditional quantile prediction with various methods including machine learning techniques such like random forests and boosting. Simulation studies reveal that the weakness of random forests lies in predicting centre quantiles and that of GB lies in predicting extremes. Is there an algorithm that enjoys the advantages of both random forests and boosting so that it can perform well over all quantiles? In this article, we propose such a boosting algorithm called random GB which embraces the merits of both random forests and GB. Empirical results will be presented to support the superiority of this algorithm in predicting conditional quantiles.     

Estimation of population mean in presence of random non response in two-stage cluster sampling

The present investigation addresses the problem of estimating a finite population mean in two-phase cluster sampling in presence of random non response situations. Utilizing information on an auxiliary variable, regression type estimators has been proposed. Effective imputation techniques have been suggested to deal with the random non response situations. The properties of the proposed estimation strategies have been studied for different cases of random non response situations in practical surveys. The superiority of the suggested methodology over the natural sample mean estimator of population mean has been established through empirical studies carried over the data sets of natural population and artificially generated population.     

Large sample inference in random coefficient regression models

Random coefficient regression models have been used t odescribe repeated measures on members of a sample of n in dividuals . Previous researchers have proposed methods of estimating the mean parameters of such models. Their methods require that eachindividual be observed under the same settings of independent variablesor , lesss stringently , that the number of observations ,r , on each individual be the same. Under the latter restriction ,estimators of mean regression parameters exist which are consist ent as both r^→∞and n^→∞ and efficient as r^→∞, and large sample ( r large ) tests of mean parameters are available . These results are easily extended to the case where not a11 individuals are observed an equal number of times provided limit are taken as min(r) → ∞. Existing methods of inference , however, are not justified by the current literature when n is large and r is small, as is the case i n many bio-medical applications . The primary con tribution of the current paper is a derivation of the asymptotic properties of modifications of existing estimators as n alone tends to infinity, r fixed. From these properties it is shown that existing methods of inference, which are currently justified only when min(r) is large, are also justifiable when n is large and min(r) is small. A secondary contribution is the definition of a positive definite estimator of the covariance matrix for the random coefficients in these models. Use of this estimator avoids computational problems that can otherwise arise.     

On adaptive linear regression

Ordinary least squares (OLS) is omnipresent in regression modeling. Occasionally, least absolute deviations (LAD) or other methods are used as an alternative when there are outliers. Although some data adaptive estimators have been proposed, they are typically difficult to implement. In this paper, we propose an easy to compute adaptive estimator which is simply a linear combination of OLS and LAD. We demonstrate large sample normality of our estimator and show that its performance is close to best for both light-tailed (e.g. normal and uniform) and heavy-tailed (e.g. double exponential and t ₃) error distributions. We demonstrate this through three simulation studies and illustrate our method on state public expenditures and lutenizing hormone data sets. We conclude that our method is general and easy to use, which gives good efficiency across a wide range of error distributions.