Variable selection in partial linear regression with functional covariate

The problem of variable selection is considered in high-dimensional partial linear regression under some model allowing for possibly functional variable. The procedure studied is that of nonconcave-penalized least squares. It is shown the existence of a √n/s_n-consistent estimator for the vector of p_n linear parameters in the model, even when p_n tends to ∞ as the sample size n increases (s_n denotes the number of influential variables). An oracle property is also obtained for the variable selection method, and the nonparametric rate of convergence is stated for the estimator of the nonlinear functional component of the model. Finally, a simulation study illustrates the finite sample size performance of our procedure.     

Some small-sample properties of some recently proposed multivariate outlier detection techniques

Recently, several new robust multivariate estimators of location and scatter have been proposed that provide new and improved methods for detecting multivariate outliers. But for small sample sizes, there are no results on how these new multivariate outlier detection techniques compare in terms of p _n , their outside rate per observation (the expected proportion of points declared outliers) under normality. And there are no results comparing their ability to detect truly unusual points based on the model that generated the data. Moreover, there are no results comparing these methods to two fairly new techniques that do not rely on some robust covariance matrix. It is found that for an approach based on the orthogonal Gnanadesikan–Kettenring estimator, p _n can be very unsatisfactory with small sample sizes, but a simple modification gives much more satisfactory results. Similar problems were found when using the median ball algorithm, but a modification proved to be unsatisfactory. The translated-biweights (TBS) estimator generally performs well with a sample size of n≥20 and when dealing with p-variate data where p≤5. But with p=8 it can be unsatisfactory, even with n=200. A projection method as well the minimum generalized variance method generally perform best, but with p≤5 conditions where the TBS method is preferable are described. In terms of detecting truly unusual points, the methods can differ substantially depending on where the outliers happen to be, the number of outliers present, and the correlations among the variables.     

Feasible robust estimator in restricted semiparametric regression models based on the LTS approach

Under some nonstochastic linear restrictions based on either additional information or prior knowledge in a semiparametric regression model, a family of feasible generalized robust estimators for the regression parameter is proposed. The least trimmed squares (LTS) method proposed by Rousseeuw as a highly robust regression estimator is a statistical technique for fitting a regression model based on the subset of h observations (out of n) whose least-square fit possesses the smallest sum of squared residuals. The coverage h may be set between n/2 and n. The LTS estimator involves computing the hyperplane that minimizes the sum of the smallest h squared residuals. For practical purpose, it is assumed that the covariance matrix of the error term is unknown and thus feasible estimators are replaced. Then, we develop an algorithm for the LTS estimator based on feasible methods. Through the Monte Carlo simulation studies and a real data example, performance of the feasible type of robust estimators is compared with the classical ones in restricted semiparametric regression models.     

Spatial Mallows model averaging for geostatistical models

Important progress has been made with model averaging methods over the past decades. For spatial data, however, the idea of model averaging has not been applied well. This article studies model averaging methods for the spatial geostatistical linear model. A spatial Mallows criterion is developed to choose weights for the model averaging estimator. The resulting estimator can achieve asymptotic optimality in terms of L₂ loss. Simulation experiments reveal that our proposed estimator is superior to the model averaging estimator by the Mallows criterion developed for ordinary linear models [Hansen, 2007] and the model selection estimator using the corrected Akaike's information criterion, developed for geostatistical linear models [Hoeting et al., 2006]. The Canadian Journal of Statistics 47: 336–351; 2019 © 2019 Statistical Society of Canada     

Estimation and Inference for Linear Panel Data Models Under Misspecification When Both n and T are Large

This article considers fixed effects (FE) estimation for linear panel data models under possible model misspecification when both the number of individuals, n, and the number of time periods, T, are large. We first clarify the probability limit of the FE estimator and argue that this probability limit can be regarded as a pseudo-true parameter. We then establish the asymptotic distributional properties of the FE estimator around the pseudo-true parameter when n and T jointly go to infinity. Notably, we show that the FE estimator suffers from the incidental parameters bias of which the top order is O(T^{? 1}), and even after the incidental parameters bias is completely removed, the rate of convergence of the FE estimator depends on the degree of model misspecification and is either (nT)^{? 1/2} or n^{? 1/2}. Second, we establish asymptotically valid inference on the (pseudo-true) parameter. Specifically, we derive the asymptotic properties of the clustered covariance matrix (CCM) estimator and the cross-section bootstrap, and show that they are robust to model misspecification. This establishes a rigorous theoretical ground for the use of the CCM estimator and the cross-section bootstrap when model misspecification and the incidental parameters bias (in the coefficient estimate) are present. We conduct Monte Carlo simulations to evaluate the finite sample performance of the estimators and inference methods, together with a simple application to the unemployment dynamics in the U.S.     

A consistent nonparametric density estimator for the deconvolution problem

The problem of nonparametric estimation of a probability density function when the sample observations are contaminated with random noise is studied. A particular estimator f?_n(x) is proposed which uses kernel-density and deconvolution techniques. The estimator f?_n(x) is shown to be uniformly consistent, and its appearance and properties are affected by constants M_n and h_n which the user may choose. The optimal choices of M_n and h_n depend on the sample size n, the noise distribution, and the true distribution which is being estimated. Particular selections for M_n and h_n which minimize upper-bound functions of the mean squared error for f?_n(x) are recommended.     

Whittle estimation in multivariate CCC-GARCH processes

In this paper, we explore some probabilistic properties and statistical analysis of multivariate constant conditional correlation GARCH (CCC-GARCH for short) model. So, in the first part we give the conditions for the model stationarity and its finite moments up to some orders. In the second part, the Whittle estimator is proposed for the parameters CCC-GARCH model based on a transformation. This Whittle estimator is shown to be consistent when the data have finite 4th moment, and its asymptotic normality is established when the data have finite 8th moment. Finite sample properties of this Whittle estimator are further examined through Monte-Carlo experiments.     

Efficiency of the generalized difference-based Liu estimators in semiparametric regression models with correlated errors

In this paper, a generalized difference-based estimator is introduced for the vector parameter β in the semiparametric regression model when the errors are correlated. A generalized difference-based Liu estimator is defined for the vector parameter β in the semiparametric regression model. Under the linear nonstochastic constraint Rβ=r, the generalized restricted difference-based Liu estimator is given. The risk function for the β?_GRD(η) associated with weighted balanced loss function is presented. The performance of the proposed estimators is evaluated by a simulated data set.     

On improving convergence rate of Bernstein polynomial density estimator

This paper is concerned with the Bernstein estimator [Vitale, R.A. (1975), ‘A Bernstein Polynomial Approach to Density Function Estimation’, in Statistical Inference and Related Topics, ed. M.L. Puri, 2, New York: Academic Press, pp. 87–99] to estimate a density with support [0, 1]. One of the major contributions of this paper is an application of a multiplicative bias correction [Terrell, G.R., and Scott, D.W. (1980), ‘On Improving Convergence Rates for Nonnegative Kernel Density Estimators’, The Annals of Statistics, 8, 1160–1163], which was originally developed for the standard kernel estimator. Moreover, the renormalised multiplicative bias corrected Bernstein estimator is studied rigorously. The mean squared error (MSE) in the interior and mean integrated squared error of the resulting bias corrected Bernstein estimators as well as the additive bias corrected Bernstein estimator [Leblanc, A. (2010), ‘A Bias-reduced Approach to Density Estimation Using Bernstein Polynomials’, Journal of Nonparametric Statistics, 22, 459–475] are shown to be O(n^?8/9) when the underlying density has a fourth-order derivative, where n is the sample size. The condition under which the MSE near the boundary is O(n^?8/9) is also discussed. Finally, numerical studies based on both simulated and real data sets are presented.     

Corrected p-values for tests based on estimated nuisance parameters

In some situations the asymptotic distribution of a random function T _n() that depends on a nuisance parameter is tractable when has known value. In that case it can be used as a test statistic, if suitably constructed, for some hypothesis. However, in practice, often needs to be replaced by an estimator S _n. In this paper general results are given concerning the asymptotic distribution of T _n(S _n) that include special cases previously dealt with. In particular, some situations are covered where the usual likelihood theory is nonregular and extreme values are employed to construct estimators and test statistics.     

On the asymptotic non‐equivalence of efficient‐GMM and MEL estimators in models with missing data

The generalized method of moments (GMM) and empirical likelihood (EL) are popular methods for combining sample and auxiliary information. These methods are used in very diverse fields of research, where competing theories often suggest variables satisfying different moment conditions. Results in the literature have shown that the efficient‐GMM (GMM_E) and maximum empirical likelihood (MEL) estimators have the same asymptotic distribution to order n^?1/2 and that both estimators are asymptotically semiparametric efficient. In this paper, we demonstrate that when data are missing at random from the sample, the utilization of some well‐known missing‐data handling approaches proposed in the literature can yield GMM_E and MEL estimators with nonidentical properties; in particular, it is shown that the GMM_E estimator is semiparametric efficient under all the missing‐data handling approaches considered but that the MEL estimator is not always efficient. A thorough examination of the reason for the nonequivalence of the two estimators is presented. A particularly strong feature of our analysis is that we do not assume smoothness in the underlying moment conditions. Our results are thus relevant to situations involving nonsmooth estimating functions, including quantile and rank regressions, robust estimation, the estimation of receiver operating characteristic (ROC) curves, and so on.     

L'effet de l'erreur d'arrondi sur l'estimateur d'une densité par la méthode du noyau

In a model for rounded data suppose that the random sample X₁,.,.,X_n,. i.i.d., is transformed into an observed random sample X_1Δ,.,.,X_nΔ, where X_iΔ = 2vΔ if X_i, ∈ (2vΔ - Δ, 2vΔ + Δ), for i = 1,.,.,n. We show that the precision Δ of the observations has an important effect on the shape of the kernel density estimator, and we identify important points for the graphical display of this estimator. We examine the IMSE criteria to find the optimal window under the rounded-data model.     

Correlation estimation in Downton's bivariate exponential distribution using incomplete samples

Suppose (X, Y) has a Downton's bivariate exponential distribution with correlation ρ. For a random sample of size n from (X, Y), let X _r:n be the rth X-order statistic and Y _[r:n] be its concomitant. We investigate estimators of ρ when all the parameters are unknown and the available data is an incomplete bivariate sample made up of (i) all the Y-values and the ranks of associated X-values, i.e. (i, Y _[i:n]), 1≤i≤n, and (ii) a Type II right-censored bivariate sample consisting of (X _i:n , Y _[i:n]), 1≤i≤r<n. In both setups, we use simulation to examine the bias and mean square errors of several estimators of ρ and obtain their estimated relative efficiencies. The preferred estimator under (i) is a function of the sample correlation of (Y _i:n , Y _[i:n]) values, and under (ii), a method of moments estimator involving the regression function is preferred.     

Estimation of the linear-plateau segmented regression model in the presence of measurement error

It is well known that when the true values of the independent variable are unobservable due to measurement error, the least squares estimator for a regression model is biased and inconsistent. When repeated observations on each x_i are taken, consistent estimators for the linear-plateau model can be formed. The repeated observations are required to classify each observation to the appropriate line segment. Two cases of repeated observations are treated in detail. First, when a single value of y_i is observed with the repeated observations of x_i the least squares estimator using the mean of the repeated x_i observations is consistent and asymptotically normal. Second, when repeated observations on the pair (x_i, y_i ) are taken the least squares estimator is inconsistent, but two consistent estimators are proposed: one that consistently estimates the bias of the least squares estimator and adjusts accordingly; the second is the least squares estimator using the mean of the repeated observations on each pair.     

Bias reduction of the maximum-likelihood estimator for a conditional Gaussian MA(1) model

In this paper, we consider an estimation for the unknown parameters of a conditional Gaussian MA(1) model. In the majority of cases, a maximum-likelihood estimator is chosen because the estimator is consistent. However, for small sample sizes the error is large, because the estimator has a bias of O(n^{? 1}). Therefore, we provide a bias of O(n^{? 1}) for the maximum-likelihood estimator for the conditional Gaussian MA(1) model. Moreover, we propose new estimators for the unknown parameters of the conditional Gaussian MA(1) model based on the bias of O(n^{? 1}). We investigate the properties of the bias, as well as the asymptotical variance of the maximum-likelihood estimators for the unknown parameters, by performing some simulations. Finally, we demonstrate the validity of the new estimators through this simulation study.     

Full likelihood inference in the Cox model with LTRC data when covariates are discrete

For the regression parameter β in the Cox model, there have been several estimates based on different types of approximated likelihood. For right-censored data, Ren and Zhou [Full likelihood inferences in the Cox model: an empirical approach. Ann Inst Statist Math. 2011;63:1005–1018] derive the full likelihood function for (β, F₀), where F₀ is the baseline distribution function in the Cox model. In this article, we extend their results to left-truncated and right-censored data with discrete covariates. Using the empirical likelihood parameterization, we obtain the full-profile likelihood function for β when covariates are discrete. Simulation results indicate that the maximum likelihood estimator outperforms Cox's partial likelihood estimator in finite samples.     

Mean likelihood estimators

The use of Mathematica in deriving mean likelihood estimators is discussed. Comparisons are made between the mean likelihood estimator, the maximum likelihood estimator, and the Bayes estimator based on a Jeffrey's noninformative prior. These estimators are compared using the mean-square error criterion and Pitman measure of closeness. In some cases it is possible, using Mathematica, to derive exact results for these criteria. Using Mathematica, simulation comparisons among the criteria can be made for any model for which we can readily obtain estimators.In the binomial and exponential distribution cases, these criteria are evaluated exactly. In the first-order moving-average model, analytical comparisons are possible only for n = 2. In general, we find that for the binomial distribution and the first-order moving-average time series model the mean likelihood estimator outperforms the maximum likelihood estimator and the Bayes estimator with a Jeffrey's noninformative prior. Mathematica was used for symbolic and numeric computations as well as for the graphical display of results. A Mathematica notebook which provides the Mathematica code used in this article is available: http://www.stats.uwo.ca/mcleod/epubs/mele. Our article concludes with our opinions and criticisms of the relative merits of some of the popular computing environments for statistics researchers.     

Nonparametric Estimation of a Regression Function: Limiting Distribution2

Consider the regression model Y_i= g(x_i) + e_i, i = 1,…, n, where g is an unknown function defined on [0, 1], 0 = x₀ < x₁ < … < x_n≤ 1 are chosen so that max_1≤i≤n(x_i-x_{i- 1}) = 0(n^-1), and where {e_i} are i.i.d. with Ee₁= 0 and Var e₁ - s?². In a previous paper, Cheng & Lin (1979) study three estimators of g, namely, g_1n of Cheng & Lin (1979), g_2n of Clark (1977), and g_3n of Priestley & Chao (1972). Consistency results are established and rates of strong uniform convergence are obtained. In the current investigation the limiting distribution of &_in, i = 1, 2, 3, and that of the isotonic estimator g**_n are considered.     

Optimal Nonparametric Quantile Estimators. Towards a General Theory. A Survey

“Nonparametric” in the title is used to say that observations X ₁,…,X _n come from an unknown distribution F ∈ ? with ? being the class of all continuous and strictly increasing distribution functions. The problem is to estimate the quantile of a given order q ∈ (0,1) of the distribution F. The class ? of distributions is very large; it is so large that even X _nq:n , where nq is an integer, may be very poor estimator of the qth quantile. To assess the performance of estimators no properties based on moments may be used: expected values of estimators should be replaced by their medians, their variances—by some characteristics of concentration of distributions around the median. If an estimator is median-biased for one of distributions, the bias of the estimator may be infinitely large for other distributions. In the note optimal estimators with respect to various criteria of optimality are presented. The pivotal function F(T) of the estimator T is introduced which enables us to apply the classical statistical approach.