Penalized Spline Varying-Coefficient Single-Index Model

In this article, the varying-coefficient single-index model (VCSIM) is discussed based on penalized spline estimation method. All the coefficient functions are fitted by P-spline and all parameters in P-spline varying-coefficient model can be estimated simultaneously by penalized nonlinear least squares. The detailed algorithm is given, including choosing smoothing parameters and knots. The approach is rapid and computationally stable. √n consistency and asymptotic normality of the estimators of all the parameters are showed. Both simulated and real data examples are given to illustrate the proposed estimation methodology.     

Penalized least squares for single index models

The single index model is a useful regression model. In this paper, we propose a nonconcave penalized least squares method to estimate both the parameters and the link function of the single index model. Compared to other variable selection and estimation methods, the proposed method can estimate parameters and select variables simultaneously. When the dimension of parameters in the single index model is a fixed constant, under some regularity conditions, we demonstrate that the proposed estimators for parameters have the so-called oracle property, and furthermore we establish the asymptotic normality and develop a sandwich formula to estimate the standard deviations of the proposed estimators. Simulation studies and a real data analysis are presented to illustrate the proposed methods.     

The influence function of penalized regression estimators

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore, it can be used to compute the asymptotic variance and the mean-squared error (MSE). In this paper we compute the influence function, the asymptotic variance and the MSE for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non-standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function.     

Variable selection and weighted composite quantile estimation of regression parameters with left-truncated data

In this paper, we consider the weighted composite quantile regression for linear model with left-truncated data. The adaptive penalized procedure for variable selection is proposed. The asymptotic normality and oracle property of the resulting estimators are also established. Simulation studies are conducted to illustrate the finite sample performance of the proposed methods.     

Generalized least squares estimation of multivariate nonlinear models with missing data

The method of estimated generalized least squares estimation of multiple response models is extended to the randomly missing date case. This estimation procedure is computationally simply when there are many missing data but the number of distinct patterns of missing data for the response vectors is small. The consistency and asymptotic normality of the proposed estimators are established.     

Variable Selection for Semiparametric Partially Linear Covariate-Adjusted Regression Models

In this article, the partially linear covariate-adjusted regression models are considered, and the penalized least-squares procedure is proposed to simultaneously select variables and estimate the parametric components. The rate of convergence and the asymptotic normality of the resulting estimators are established under some regularization conditions. With the proper choices of the penalty functions and tuning parameters, it is shown that the proposed procedure can be as efficient as the oracle estimators. Some Monte Carlo simulation studies and a real data application are carried out to assess the finite sample performances for the proposed method.     

Estimation of parameters of two-dimensional sinusoidal signal in heavy-tailed errors

In this paper, we consider a two-dimensional sinusoidal model observed in an additive random field. The proposed model has wide applications in statistical signal processing. The additive noise has mean zero but the variance may not be finite. We propose the least squares estimators to estimate the unknown parameters. It is observed that the least squares estimators are strongly consistent. We obtain the asymptotic distribution of the least squares estimators under the assumption that the additive errors are from a symmetric stable distribution. Some numerical experiments are performed to see how the results work for finite samples.     

Nonconcave penalized estimation for partially linear models with longitudinal data

A nonconcave penalized estimation method is proposed for partially linear models with longitudinal data when the number of parameters diverges with the sample size. The proposed procedure can simultaneously estimate the parameters and select the important variables. Under some regularity conditions, the rate of convergence and asymptotic normality of the resulting estimators are established. In addition, an iterative algorithm is proposed to implement the proposed estimators. To improve efficiency for regression coefficients, the estimation of the covariance function is integrated in the iterative algorithm. Simulation studies are carried out to demonstrate that the proposed method performs well, and a real data example is analysed to illustrate the proposed procedure.     

Change points in nonparametric regresion functions

Estimators of location and size of jumps or discontinuities in a regression function and/or its derivatives are proposed. The estimators are based on the analysis of residuals obtained from the locally weighted least squares regression. The proposed estimators adapt to both fixed and random designs. The asymptotic properties of the estimators are investigated. The method is illustrated through simulation studies.     

Statistical Analysis of Parameter Estimation for 2-D Harmonics in Multiplicative and Additive Noise

This article considers the problem of parameter estimation for two dimensional (2-D) multi-component harmonics in non zero-mean multiplicative and additive noise. The least squares estimators (LSEs) are proposed to estimate the coherent model parameters, and some statistical results of the LSEs are obtained, including strong consistency, strong convergence rate, and asymptotic normality. Furthermore, the LSEs-based estimators are proposed to estimate the noncoherent model parameters, and the strong consistency and the asymptotic normality are also proved. Finally, some numerical experiments are performed to see how the asymptotic results work for finite sample sizes.     

Minimax Linear Smoothers

We consider the problem of estimating the mean of a multivariate distribution. As a general alternative to penalized least squares estimators, we consider minimax estimators for squared error over a restricted parameter space where the restriction is determined by the penalization term. For a quadratic penalty term, the minimax estimator among linear estimators can be found explicitly. It is shown that all symmetric linear smoothers with eigenvalues in the unit interval can be characterized as minimax linear estimators over a certain parameter space where the bias is bounded. The minimax linear estimator depends on smoothing parameters that must be estimated in practice. Using results in Kneip (1994), this can be done using Mallows' C _L -statistic and the resulting adaptive estimator is now asymptotically minimax linear. The minimax estimator is compared to the penalized least squares estimator both in finite samples and asymptotically.     

Model averaging procedure for partially linear single-index models

This paper is concerned with model selection and model averaging procedures for partially linear single-index models. The profile least squares procedure is employed to estimate regression coefficients for the full model and submodels. We show that the estimators for submodels are asymptotically normal. Based on the asymptotic distribution of the estimators, we derive the focused information criterion (FIC), formulate the frequentist model average (FMA) estimators and construct proper confidence intervals for FMA estimators and FIC estimator, a special case of FMA estimators. Monte Carlo studies are performed to demonstrate the superiority of the proposed method over the full model, and over models chosen by AIC or BIC in terms of coverage probability and mean squared error. Our approach is further applied to real data from a male fertility study to explore potential factors related to sperm concentration and estimate the relationship between sperm concentration and monobutyl phthalate.     

An efficient algorithm for estimating the parameters of superimposed exponential signals

An efficient computational algorithm is proposed for estimating the parameters of undamped exponential signals, when the parameters are complex valued. Such data arise in several areas of applications including telecommunications, radio location of objects, seismic signal processing and computer assisted medical diagnostics. It is observed that the proposed estimators are consistent and the dispersion matrix of these estimators is asymptotically the same as that of the least squares estimators. Moreover, the asymptotic variances of the proposed estimators attain the Cramer–Rao lower bounds, when the errors are Gaussian.     

Adaptation over parametric families of symmetric linear estimators

This paper treats an abstract parametric family of symmetric linear estimators for the mean vector of a standard linear model. The estimator in this family that has smallest estimated quadratic risk is shown to attain, asymptotically, the smallest risk achievable over all candidate estimators in the family. The asymptotic analysis is carried out under a strong Gauss–Markov form of the linear model in which the dimension of the regression space tends to infinity. Leading examples to which the results apply include: (a) penalized least squares fits constrained by multiple, weighted, quadratic penalties; and (b) running, symmetrically weighted, means. In both instances, the weights define a parameter vector whose natural domain is a continuum.     

A Simple Approach to Inference in Random Coefficient Models

Random coefficient regression models have been used to analyze cross-sectional and longitudinal data in economics and growth-curve data from biological and agricultural experiments. In the literature several estimators, including the ordinary least squares and the estimated generalized least squares (EGLS), have been considered for estimating the parameters of the mean model. Based on the asymptotic properties of the EGLS estimators, test statistics have been proposed for testing linear hypotheses involving the parameters of the mean model. An alternative estimator, the simple mean of the individual regression coefficients, provides estimation and hypothesis-testing procedures that are simple to compute and teach. The large sample properties of this simple estimator are shown to be similar to that of the EGLS estimator. The performance of the proposed estimator is compared with that of the existing estimators by Monte Carlo simulation.     

Quantile regression for robust inference on varying coefficient partially nonlinear models

In this paper, we propose a robust statistical inference approach for the varying coefficient partially nonlinear models based on quantile regression. A three-stage estimation procedure is developed to estimate the parameter and coefficient functions involved in the model. Under some mild regularity conditions, the asymptotic properties of the resulted estimators are established. Some simulation studies are conducted to evaluate the finite performance as well as the robustness of our proposed quantile regression method versus the well known profile least squares estimation procedure. Moreover, the Boston housing price data is given to further illustrate the application of the new method.     

Estimation of partially specified spatial panel data models with fixed-effects

This article extends the spatial panel data regression with fixed-effects to the case where the regression function is partially linear and some regressors may be endogenous or predetermined. Under the assumption that the spatial weighting matrix is strictly exogenous, we propose a sieve two stage least squares (S2SLS) regression. Under some sufficient conditions, we show that the proposed estimator for the finite dimensional parameter is root-N consistent and asymptotically normally distributed and that the proposed estimator for the unknown function is consistent and also asymptotically normally distributed but at a rate slower than root-N. Consistent estimators for the asymptotic variances of the proposed estimators are provided. A small scale simulation study is conducted, and the simulation results show that the proposed procedure has good finite sample performance.     

On approximate least squares estimators of parameters of one-dimensional chirp signal

Chirp signals are quite common in many natural and man-made systems such as audio signals, sonar, and radar. Estimation of the unknown parameters of a signal is a fundamental problem in statistical signal processing. Recently, Kundu and Nandi [Parameter estimation of chirp signals in presence of stationary noise. Stat Sin. 2008;75:187–201] studied the asymptotic properties of least squares estimators (LSEs) of the unknown parameters of a simple chirp signal model under the assumption of stationary noise. In this paper, we propose periodogram-type estimators called the approximate least squares estimators (ALSEs) to estimate the unknown parameters and study the asymptotic properties of these estimators under the same error assumptions. It is observed that the ALSEs are strongly consistent and asymptotically equivalent to the LSEs. Similar to the periodogram estimators, these estimators can also be used as initial guesses to find the LSEs of the unknown parameters. We perform some numerical simulations to see the performance of the proposed estimators and compare them with the LSEs and the estimators proposed by Lahiri et al. [Efficient algorithm for estimating the parameters of two dimensional chirp signal. Sankhya B. 2013;75(1):65–89]. We have analysed two real data sets for illustrative purposes.     

Estimators based on ranks for arma models

In this paper we introduce a new family of robust estimators for ARMA models. These estimators are defined by replacing the residual sample autocovariances in the least squares equations by autocovariances based on ranks. The asymptotic normality of the proposed estimators is provided. The efficiency and robustness properties of these estimators are studied. An adequate choice of the score functions gives estimators which have high efficiency under normality and robustness in the presence of outliers. The score functions can also be chosen so that the resulting estimators are asymptotically as efficient as the maximum likelihood estimators for a given distribution.     

On shrinkage least squares estimation in a parallelism problem

In a multi-sample simple regression model, generally, homogeneity of the regression slopes leads to improved estimation of the intercepts. Analogous to the preliminary test estimators, (smooth) shrinkage least squares estimators of Intercepts based on the James-Stein rule on regression slopes are considered. Relative pictures on the (asymptotic) risk of the classical, preliminary test and the shrinkage least squares estimators are also presented. None of the preliminary test and shrinkage least squares estimators may dominate over the other, though each of them fares well relative to the other estimators.