On empirical Bayes estimation of variance components in random effects model

In this paper, Bayes estimators of variance components are derived for the one-way random effects model, and empirical Bayes (EB) estimators are constructed by the kernel estimation method of a multivariate density and its mixed partial derivatives. It is shown that the EB estimators are asymptotically optimal and convergence rates are established. Finally, an example concerning the main results is given.     

Empirical bayes estimation of reliability characteristics for an exponential family

This paper considers empirical Bayes (EB) squared-error-loss estimations of mean lifetime, variance and reliability function for failure-time distributions belonging to an exponential family, which includes gamma and Weibull distributions as special cases. EB estimators are proposed when the prior distribution of the lifetime parameter is completely unknown but has a compact (known or unknown) support. Asymptotic optimality and rates of convergence of these estimators are investigated. The rates established here under the compact support restriction are better than the polynomial rates of convergence obtained previously.     

Empirical Bayes estimation in continuous one-parameter exponential families under associated samples

In this paper, we study the empirical Bayes (EB) estimation in continuous one-parameter exponential families under negatively associated (NA) samples and positively associated (PA) samples. Under certain regularity conditions, it is shown that the convergence rates of proposed EB estimators under NA or PA samples are the same as those of EB estimators under independent observations, which significantly improve the existing results in EB estimation under associated samples.     

Empirical Bayes Estimation of Small Area Means under a Nested Error Linear Regression Model with Measurement Errors in the Covariates

Abstract.  Previously, small area estimation under a nested error linear regression model was studied with area level covariates subject to measurement error. However, the information on observed covariates was not used in finding the Bayes predictor of a small area mean. In this paper, we first derive the fully efficient Bayes predictor by utilizing all the available data. We then estimate the regression and variance component parameters in the model to get an empirical Bayes (EB) predictor and show that the EB predictor is asymptotically optimal. In addition, we employ the jackknife method to obtain an estimator of mean squared prediction error (MSPE) of the EB predictor. Finally, we report the results of a simulation study on the performance of our EB predictor and associated jackknife MSPE estimators. Our results show that the proposed EB predictor can lead to significant gain in efficiency over the previously proposed EB predictor.     

Variance Estimation Based on Invariance Principles

Consistent variance estimators for certain stochastic processes are suggested using the fact that (weak or strong) invariance principles may be available. Convergence rates are also derived, the latter being essentially determined by the approximation rates in the corresponding invariance principles. As an application, a change point test in a simple AMOC renewal model is briefly discussed, where variance estimators possessing good enough convergence rates are required.     

Partial deconvolution estimation in nonparametric regression

In this article, we propose a class of partial deconvolution kernel estimators for the nonparametric regression function when some covariates are measured with error and some are not. The estimation procedure combines the classical kernel methodology and the deconvolution kernel technique. According to whether the measurement error is ordinarily smooth or supersmooth, we establish the optimal local and global convergence rates for these proposed estimators, and the optimal bandwidths are also identified. Furthermore, lower bounds for the convergence rates of all possible estimators for the nonparametric regression functions are developed. It is shown that, in both the super and ordinarily smooth cases, the convergence rates of the proposed partial deconvolution kernel estimators attain the lower bound. The Canadian Journal of Statistics 48: 535–560; 2020 © 2020 Statistical Society of Canada     

Bayesian estimation for first-order autoregressive model with explanatory variables

In this article, we develop a Bayesian analysis in autoregressive model with explanatory variables. When σ² is known, we consider a normal prior and give the Bayesian estimator for the regression coefficients of the model. For the case σ² is unknown, another Bayesian estimator is given for all unknown parameters under a conjugate prior. Bayesian model selection problem is also being considered under the double-exponential priors. By the convergence of ρ-mixing sequence, the consistency and asymptotic normality of the Bayesian estimators of the regression coefficients are proved. Simulation results indicate that our Bayesian estimators are not strongly dependent on the priors, and are robust.     

Conditional variance estimation via nonparametric generalized additive models

Generalized additive models provide a way of circumventing curse of dimension in a wide range of nonparametric regression problem. In this paper, we present a multiplicative model for conditional variance functions where one can apply a generalized additive regression method. This approach extends Fan and Yao (1998) to multivariate cases with a multiplicative structure. In this approach, we use squared residuals instead of using log-transformed squared residuals. This idea gives a smaller variance than Yu (2017) when the variance of squared error is smaller than the variance of log-transformed squared error. We provide estimators based on quasi-likelihood and an iterative algorithm based on smooth backfitting for generalized additive models. We also provide some asymptotic properties of estimators and the convergence of proposed algorithm. A numerical study shows the empirical evidence of the theory.     

Probability density estimation with data missing at random when covariables are present

This paper addresses the problem of the probability density estimation in the presence of covariates when data are missing at random (MAR). The inverse probability weighted method is used to define a nonparametric and a semiparametric weighted probability density estimators. A regression calibration technique is also used to define an imputed estimator. It is shown that all the estimators are asymptotically normal with the same asymptotic variance as that of the inverse probability weighted estimator with known selection probability function and weights. Also, we establish the mean squared error (MSE) bounds and obtain the MSE convergence rates. A simulation is carried out to assess the proposed estimators in terms of the bias and standard error.     

On the stability of the jackknife variance estimator in ratio estimation

The stability of a slightly modified version of the usual jackknife variance estimator is evaluated exactly in small samples under a suitable linear regression model and compared with that of two different linearization variance estimators. Depending on the degree of heteroscedasticity of the error variance in the model, the stability of the jackknife variance estimator is found to be somewhat comparable to that of one or the other of the linearization variance estimators under conditions especially favorable to ratio estimation (i.e., regression approximately through the origin with a relatively small coefficient of variation in the x population). When these conditions do not hold, however, the jackknife variance estimator is found to be less stable than either of the linearization variance estimators.     

On Measuring Uncertainty of Benchmarked Predictors with Application to Disease Risk Estimate

Empirical Bayes (EB) estimates in general linear mixed models are useful for the small area estimation in the sense of increasing precision of estimation of small area means. However, one potential difficulty of EB is that the overall estimate for a larger geographical area based on a (weighted) sum of EB estimates is not necessarily identical to the corresponding direct estimate such as the overall sample mean. Another difficulty is that EB estimates yield over‐shrinking, which results in the sampling variance smaller than the posterior variance. One way to fix these problems is the benchmarking approach based on the constrained empirical Bayes (CEB) estimators, which satisfy the constraints that the aggregated mean and variance are identical to the requested values of mean and variance. In this paper, we treat the general mixed models, derive asymptotic approximations of the mean squared error (MSE) of CEB and provide second‐order unbiased estimators of MSE based on the parametric bootstrap method. These results are applied to natural exponential families with quadratic variance functions. As a specific example, the Poisson‐gamma model is dealt with, and it is illustrated that the CEB estimates and their MSE estimates work well through real mortality data.     

Nonparametric quasi-likelihood for right censored data

Quasi-likelihood was extended to right censored data to handle heteroscedasticity in the frame of the accelerated failure time (AFT) model. However, the assumption of known variance function in the quasi-likelihood for right censored data is usually unrealistic. In this paper, we propose a nonparametric quasi-likelihood by replacing the specified variance function with a nonparametric variance function estimator. This nonparametric variance function estimator is obtained by smoothing a function of squared residuals via local polynomial regression. The rate of convergence of the nonparametric variance function estimator and the asymptotic limiting distributions of the regression coefficient estimators are derived. It is demonstrated in simulations that for finite samples the proposed nonparametric quasi-likelihood method performs well. The new method is illustrated with one real dataset.     

Estimation in a linear regression model under the Kullback–Leibler loss and its application to model selection

This paper is concerned with the problem of constructing a good predictive distribution relative to the Kullback–Leibler information in a linear regression model. The problem is equivalent to the simultaneous estimation of regression coefficients and error variance in terms of a complicated risk, which yields a new challenging issue in a decision-theoretic framework. An estimator of the variance is incorporated here into a loss for estimating the regression coefficients. Several estimators of the variance and of the regression coefficients are proposed and shown to improve on usual benchmark estimators both analytically and numerically. Finally, the prediction problem of a distribution is noted to be related to an information criterion for model selection like the Akaike information criterion (AIC). Thus, several AIC variants are obtained based on proposed and improved estimators and are compared numerically with AIC as model selection procedures.     

Estimating the error variance after a pre-test for an inequality restriction on the coefficients

Estimation of the regression error variance after a preliminary test of an inequality constraint on the coefficient vector is considered. We derive the exact finite sample risk of several inequality restricted and pre-test estimators of σ². These estimators are associated with the maximum likelihood, least squares and minimum mean squared error component estimators. Optimal critical values for the pre-test according to a mini-max regret criterion are numerically calculated. Furthermore, we examine the robustness of the optimal choice of critical values and the risk properties of the estimators of σ² to model mis-specification through the exclusion of relevant regressors.     

Semiparametric regression under long-range dependent errors

In this paper, we consider a semiparametric regression model under long-range dependent errors. By approximating the nonparametric component by a finite series sum, we construct consistent estimators for both parametric and nonparametric components. Meanwhile, convergence rates for the consistent estimators are also investigated. Additionally, an optimal truncation parameter selection procedure is proposed.     

Wavelet-based estimation of regression function with strong mixing errors under fixed design

We consider wavelet-based non linear estimators, which are constructed by using the thresholding of the empirical wavelet coefficients, for the mean regression functions with strong mixing errors and investigate their asymptotic rates of convergence. We show that these estimators achieve nearly optimal convergence rates within a logarithmic term over a large range of Besov function classes B^s_{p, q}. The theory is illustrated with some numerical examples.

A new ingredient in our development is a Bernstein-type exponential inequality, for a sequence of random variables with certain mixing structure and are not necessarily bounded or sub-Gaussian. This moderate deviation inequality may be of independent interest.     

Improved estimation in Lognormal regression models

Lognormal regression model with unknown error variance is considered. We give a class of estimators of the regression coefficients vector improving upon traditional estimator when the number of independent variables is at least three. The relationship between these estimators on one hand and James-Stein type estimators of the normal mean and improved estimators of the normal variance on another hand is discussed.     

Constrained Bayes and Empirical Bayes Estimation with Balanced Loss Functions

This article develops constrained Bayes and empirical Bayes estimators under balanced loss functions. In the normal-normal example, estimators of the mean squared errors of the EB and constrained EB estimators are provided which are correct asymptotically up to O(m ^?1), m denoting the number of strata.     

Nonparametric regression estimates with censored data based on block thresholding method

Here we consider wavelet-based identification and estimation of a censored nonparametric regression model via block thresholding methods and investigate their asymptotic convergence rates. We show that these estimators, based on block thresholding of empirical wavelet coefficients, achieve optimal convergence rates over a large range of Besov function classes, and in particular enjoy those rates without the extraneous logarithmic penalties that are usually suffered by term-by-term thresholding methods. This work is extension of results in Li et al. (2008). The performance of proposed estimator is investigated by a numerical study.     

On histograms for linear processes

Sharp rates of convergence of histogram estimates of the marginal density of a linear process are obtained. Histograms can achieve optimal rates of convergence (n⁻¹ log n)^1·3 under general conditions. The assumptions involved are easily verifiable. Histograms appear to be very good estimators from the point of view of uniform convergence.