Estimation of parameters of two-dimensional sinusoidal signal in heavy-tailed errors

In this paper, we consider a two-dimensional sinusoidal model observed in an additive random field. The proposed model has wide applications in statistical signal processing. The additive noise has mean zero but the variance may not be finite. We propose the least squares estimators to estimate the unknown parameters. It is observed that the least squares estimators are strongly consistent. We obtain the asymptotic distribution of the least squares estimators under the assumption that the additive errors are from a symmetric stable distribution. Some numerical experiments are performed to see how the results work for finite samples.     

Adaptive estimation of vector autoregressive models with time-varying variance: Application to testing linear causality in mean

Linear vector autoregressive (VAR) models where the innovations could be unconditionally heteroscedastic are considered. The volatility structure is deterministic and quite general, including breaks or trending variances as special cases. In this framework we propose ordinary least squares (OLS), generalized least squares (GLS) and adaptive least squares (ALS) procedures. The GLS estimator requires the knowledge of the time-varying variance structure while in the ALS approach the unknown variance is estimated by kernel smoothing with the outer product of the OLS residual vectors. Different bandwidths for the different cells of the time-varying variance matrix are also allowed. We derive the asymptotic distribution of the proposed estimators for the VAR model coefficients and compare their properties. In particular we show that the ALS estimator is asymptotically equivalent to the infeasible GLS estimator. This asymptotic equivalence is obtained uniformly with respect to the bandwidth(s) in a given range and hence justifies data-driven bandwidth rules. Using these results we build Wald tests for the linear Granger causality in mean which are adapted to VAR processes driven by errors with a nonstationary volatility. It is also shown that the commonly used standard Wald test for the linear Granger causality in mean is potentially unreliable in our framework (incorrect level and lower asymptotic power). Monte Carlo experiments illustrate the use of the different estimation approaches for the analysis of VAR models with time-varying variance innovations.     

Large sample behavior of the Bernstein copula estimator

Bernstein polynomial estimators have been used as smooth estimators for density functions and distribution functions. The idea of using them for copula estimation has been given in Sancetta and Satchell (2004). In the present paper we study the asymptotic properties of this estimator: almost sure consistency rates and asymptotic normality. We also obtain explicit expressions for the asymptotic bias and asymptotic variance and show the improvement of the asymptotic mean squared error compared to that of the classical empirical copula estimator. A small simulation study illustrates this superior behavior in small samples.     

Estimators based on ranks for arma models

In this paper we introduce a new family of robust estimators for ARMA models. These estimators are defined by replacing the residual sample autocovariances in the least squares equations by autocovariances based on ranks. The asymptotic normality of the proposed estimators is provided. The efficiency and robustness properties of these estimators are studied. An adequate choice of the score functions gives estimators which have high efficiency under normality and robustness in the presence of outliers. The score functions can also be chosen so that the resulting estimators are asymptotically as efficient as the maximum likelihood estimators for a given distribution.     

Local Linear Least Squares Kernel Regression for Linear and Circular Predictors

We extend nonparametric regression models with local linear least squares fitting using kernel weights to the case of linear and circular predictors. We derive the asymptotic properties of the conditional bias and variance of bivariate local linear least squares kernel estimators. A small simulation study and a real experiment are given.     

A Simple Approach to Inference in Random Coefficient Models

Random coefficient regression models have been used to analyze cross-sectional and longitudinal data in economics and growth-curve data from biological and agricultural experiments. In the literature several estimators, including the ordinary least squares and the estimated generalized least squares (EGLS), have been considered for estimating the parameters of the mean model. Based on the asymptotic properties of the EGLS estimators, test statistics have been proposed for testing linear hypotheses involving the parameters of the mean model. An alternative estimator, the simple mean of the individual regression coefficients, provides estimation and hypothesis-testing procedures that are simple to compute and teach. The large sample properties of this simple estimator are shown to be similar to that of the EGLS estimator. The performance of the proposed estimator is compared with that of the existing estimators by Monte Carlo simulation.     

Estimation for varying coefficient partially nonlinear models with distorted measurement errors

In this paper, we propose a new varying coefficient partially nonlinear model where both the response and predictors are not directly observed, but are observed by unknown distorting functions of a commonly observable covariate. Because of the complexity of the model, existing estimation methods cannot be directly employed. For this, we propose using an efficient nonparametric regression to estimate the unknown distortion functions concerning the covariates and response on the distorting variable, and further, we obtain the profile nonlinear least squares estimators for the parameters and the coefficient functions using the calibrated variables. Furthermore, we establish the asymptotic properties of the resulting estimators. To illustrate our proposed methodology, we carry out some simulated and real examples.     

Bounded influence nonlinear signed‐rank regression

In this paper we consider weighted generalized‐signed‐rank estimators of nonlinear regression coefficients. The generalization allows us to include popular estimators such as the least squares and least absolute deviations estimators but by itself does not give bounded influence estimators. Adding weights results in estimators with bounded influence function. We establish conditions needed for the consistency and asymptotic normality of the proposed estimator and discuss how weight functions can be chosen to achieve bounded influence function of the estimator. Real life examples and Monte Carlo simulation experiments demonstrate the robustness and efficiency of the proposed estimator. An example shows that the weighted signed‐rank estimator can be useful to detect outliers in nonlinear regression. The Canadian Journal of Statistics 40: 172–189; 2012 © 2012 Statistical Society of Canada     

Estimation of parameters in a generalized GMANOVA model based on an outer product analogy and least squares

This paper investigates estimation of parameters in a combination of the multivariate linear model and growth curve model, called a generalized GMANOVA model. Making analogy between the outer product of data vectors and covariance yields an approach to directly do least squares to covariance. An outer product least squares estimator of covariance (COPLS estimator) is obtained and its distribution is presented if a normal assumption is imposed on the error matrix. Based on the COPLS estimator, two-stage generalized least squares estimators of the regression coefficients are derived. In addition, asymptotic normalities of these estimators are investigated. Simulation studies have shown that the COPLS estimator and two-stage GLS estimators are alternative competitors with more efficiency in the sense of sample mean, standard deviations and mean of the variance estimates to the existing ML estimator in finite samples. An example of application is also illustrated.     

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.     

Kernel Estimation of Rate Function for Recurrent Event Data

Abstract.  Recurrent event data are largely characterized by the rate function but smoothing techniques for estimating the rate function have never been rigorously developed or studied in statistical literature. This paper considers the moment and least squares methods for estimating the rate function from recurrent event data. With an independent censoring assumption on the recurrent event process, we study statistical properties of the proposed estimators and propose bootstrap procedures for the bandwidth selection and for the approximation of confidence intervals in the estimation of the occurrence rate function. It is identified that the moment method without resmoothing via a smaller bandwidth will produce a curve with nicks occurring at the censoring times, whereas there is no such problem with the least squares method. Furthermore, the asymptotic variance of the least squares estimator is shown to be smaller under regularity conditions. However, in the implementation of the bootstrap procedures, the moment method is computationally more efficient than the least squares method because the former approach uses condensed bootstrap data. The performance of the proposed procedures is studied through Monte Carlo simulations and an epidemiological example on intravenous drug users.     

Error variance estimation via least squares for small sample nonparametric regression

In this paper we explore statistical properties of some difference-based approaches to estimate an error variance for small sample based on nonparametric regression which satisfies Lipschitz condition. Our study is motivated by Tong and Wang (2005), who estimated error variance using a least squares approach. They considered the error variance as the intercept in a simple linear regression which was obtained from the expectation of their lag-k Rice estimator. Their variance estimators are highly dependent on the setting of a regressor and weight of their simple linear regression. Although this regressor and weight can be varied based on the characteristic of an unknown nonparametric mean function, Tong and Wang (2005) have used a fixed regressor and weight in a large sample and gave no indication of how to determine the regressor and the weight. In this paper, we propose a new approach via local quadratic approximation to determine this regressor and weight. Using our proposed regressor and weight, we estimate the error variance as the intercept of simple linear regression using both ordinary least squares and weighted least squares. Our approach applies to both small and large samples, while most existing difference-based methods are appropriate solely for large samples. We compare the performance of our approach with other existing approaches using extensive simulation study. The advantage of our approach is demonstrated using a real data set.     

Variable Selection for Partially Linear Models with Randomly Censored Data

This article proposes a variable selection procedure for partially linear models with right-censored data via penalized least squares. We apply the SCAD penalty to select significant variables and estimate unknown parameters simultaneously. The sampling properties for the proposed procedure are investigated. The rate of convergence and the asymptotic normality of the proposed estimators are established. Furthermore, the SCAD-penalized estimators of the nonzero coefficients are shown to have the asymptotic oracle property. In addition, an iterative algorithm is proposed to find the solution of the penalized least squares. Simulation studies are conducted to examine the finite sample performance of the proposed method.     

Nonparametric Shrinkage Estimation for Aalen's Additive Hazards Model

Aalen's nonparametric additive model in which the regression coefficients are assumed to be unspecified functions of time is a flexible alternative to Cox's proportional hazards model when the proportionality assumption is in doubt. In this paper, we incorporate a general linear hypothesis into the estimation of the time‐varying regression coefficients. We combine unrestricted least squares estimators and estimators that are restricted by the linear hypothesis and produce James‐Stein‐type shrinkage estimators of the regression coefficients. We develop the asymptotic joint distribution of such restricted and unrestricted estimators and use this to study the relative performance of the proposed estimators via their integrated asymptotic distributional risks. We conduct Monte Carlo simulations to examine the relative performance of the estimators in terms of their integrated mean square errors. We also compare the performance of the proposed estimators with a recently devised LASSO estimator as well as with ridge‐type estimators both via simulations and data on the survival of primary billiary cirhosis patients.     

On approximate least squares estimators of parameters of one-dimensional chirp signal

Chirp signals are quite common in many natural and man-made systems such as audio signals, sonar, and radar. Estimation of the unknown parameters of a signal is a fundamental problem in statistical signal processing. Recently, Kundu and Nandi [Parameter estimation of chirp signals in presence of stationary noise. Stat Sin. 2008;75:187–201] studied the asymptotic properties of least squares estimators (LSEs) of the unknown parameters of a simple chirp signal model under the assumption of stationary noise. In this paper, we propose periodogram-type estimators called the approximate least squares estimators (ALSEs) to estimate the unknown parameters and study the asymptotic properties of these estimators under the same error assumptions. It is observed that the ALSEs are strongly consistent and asymptotically equivalent to the LSEs. Similar to the periodogram estimators, these estimators can also be used as initial guesses to find the LSEs of the unknown parameters. We perform some numerical simulations to see the performance of the proposed estimators and compare them with the LSEs and the estimators proposed by Lahiri et al. [Efficient algorithm for estimating the parameters of two dimensional chirp signal. Sankhya B. 2013;75(1):65–89]. We have analysed two real data sets for illustrative purposes.     

Wavelet estimation in time-varying coefficient time series models with measurement errors

The article studies a time-varying coefficient time series model in which some of the covariates are measured with additive errors. In order to overcome the bias of estimator of the coefficient functions when measurement errors are ignored, we propose a modified least squares estimator based on wavelet procedures. The advantage of the wavelet method is to avoid the restrictive smoothness requirement for varying-coefficient functions of the traditional smoothing approaches, such as kernel and local polynomial methods. The asymptotic properties of the proposed wavelet estimators are established under the α-mixing conditions and without specifying the error distribution. These results can be used to make asymptotically valid statistical inference.     

Relative efficiency of three estimators in a polynomial regression with measurement errors

In a polynomial regression with measurement errors in the covariate, the latter being supposed to be normally distributed, one has (at least) three ways to estimate the unknown regression parameters: one can apply ordinary least squares (OLS) to the model without regard to the measurement error or one can correct for the measurement error, either by correcting the estimating equation (ALS) or by correcting the mean and variance functions of the dependent variable, which is done by conditioning on the observable, error ridden, counter part of the covariate (SLS). While OLS is biased, the other two estimators are consistent. Their asymptotic covariance matrices and thus their relative efficiencies can be compared to each other, in particular for the case of a small measurement error variance. In this case, it appears that ALS and SLS become almost equally efficient, even when they differ noticeably from OLS.     

Super efficient frequency estimation

In this paper we propose a modified Newton-Raphson method to obtain super efficient estimators of the frequencies of a sinusoidal signal in presence of stationary noise. It is observed that if we start from an initial estimator with convergence rate O_p(n⁻¹) and use Newton-Raphson algorithm with proper step factor modification, then it produces super efficient frequency estimator in the sense that its asymptotic variance is lower than the asymptotic variance of the corresponding least squares estimator. The proposed frequency estimator is consistent and it has the same rate of convergence, namely O_p(n^−3/2), as the least squares estimator. Monte Carlo simulations are performed to observe the performance of the proposed estimator for different sample sizes and for different models. The results are quite satisfactory. One real data set has been analyzed for illustrative purpose.     

SHRINKAGE, PRETEST AND ABSOLUTE PENALTY ESTIMATORS IN PARTIALLY LINEAR MODELS

We consider a partially linear model in which the vector of coefficients β in the linear part can be partitioned as ( β ₁, β ₂) , where β ₁ is the coefficient vector for main effects (e.g. treatment effect, genetic effects) and β ₂ is a vector for ‘nuisance’ effects (e.g. age, laboratory). In this situation, inference about β ₁ may benefit from moving the least squares estimate for the full model in the direction of the least squares estimate without the nuisance variables (Steinian shrinkage), or from dropping the nuisance variables if there is evidence that they do not provide useful information (pretesting). We investigate the asymptotic properties of Stein‐type and pretest semiparametric estimators under quadratic loss and show that, under general conditions, a Stein‐type semiparametric estimator improves on the full model conventional semiparametric least squares estimator. The relative performance of the estimators is examined using asymptotic analysis of quadratic risk functions and it is found that the Stein‐type estimator outperforms the full model estimator uniformly. By contrast, the pretest estimator dominates the least squares estimator only in a small part of the parameter space, which is consistent with the theory. We also consider an absolute penalty‐type estimator for partially linear models and give a Monte Carlo simulation comparison of shrinkage, pretest and the absolute penalty‐type estimators. The comparison shows that the shrinkage method performs better than the absolute penalty‐type estimation method when the dimension of the β ₂ parameter space is large.