Penalized weighted composite quantile regression in the linear regression model with heavy-tailed autocorrelated errors

In this paper, a penalized weighted composite quantile regression estimation procedure is proposed to estimate unknown regression parameters and autoregression coefficients in the linear regression model with heavy-tailed autoregressive errors. Under some conditions, we show that the proposed estimator possesses the oracle properties. In addition, we introduce an iterative algorithm to achieve the proposed optimization problem, and use a data-driven method to choose the tuning parameters. Simulation studies demonstrate that the proposed new estimation method is robust and works much better than the least squares based method when there are outliers in the dataset or the autoregressive error distribution follows heavy-tailed distributions. Moreover, the proposed estimator works comparably to the least squares based estimator when there are no outliers and the error is normal. Finally, we apply the proposed methodology to analyze the electricity demand dataset.     

Simultaneous Inference For The Mean Function Based on Dense Functional Data

A polynomial spline estimator is proposed for the mean function of dense functional data together with a simultaneous confidence band which is asymptotically correct. In addition, the spline estimator and its accompanying confidence band enjoy oracle efficiency in the sense that they are asymptotically the same as if all random trajectories are observed entirely and without errors. The confidence band is also extended to the difference of mean functions of two populations of functional data. Simulation experiments provide strong evidence that corroborates the asymptotic theory while computing is efficient. The confidence band procedure is illustrated by analyzing the near infrared spectroscopy data.     

Efficient Inference for Longitudinal Data Varying‐coefficient Regression Models

Informative identification of the within‐subject correlation is essential in longitudinal studies in order to forecast the trajectory of each subject and improve the validity of inferences. In this paper, we fit this correlation structure by employing a time adaptive autoregressive error process. Such a process can automatically accommodate irregular and possibly subject‐specific observations. Based on the fitted correlation structure, we propose an efficient two‐stage estimator of the unknown coefficient functions by using a local polynomial approximation. This procedure does not involve within‐subject covariance matrices and hence circumvents the instability of calculating their inverses. The asymptotic normality of resulting estimators is established. Numerical experiments were conducted to check the finite sample performance of our method and an example of an application involving a set of medical data is also illustrated.     

Partially linear models and polynomial spline approximations for the analysis of unbalanced panel data

In this paper, we study the estimation of the unbalanced panel data partially linear models with a one-way error components structure. A weighted semiparametric least squares estimator (WSLSE) is developed using polynomial spline approximation and least squares. We show that the WSLSE is asymptotically more efficient than the corresponding unweighted estimator for both parametric and nonparametric components of the model. This is a significant improvement over previous results in the literature which showed that the simply weighting technique can only improve the estimation of the parametric component. The asymptotic normalities of the proposed WSLSE are also established.     

Quantile regression estimation for distortion measurement error data

We study the quantile estimation methods for the distortion measurement error data when variables are unobserved and distorted with additive errors by some unknown functions of an observable confounding variable. After calibrating the error-prone variables, we propose the quantile regression estimation procedure and composite quantile estimation procedure. Asymptotic properties of the proposed estimators are established, and we also investigate the asymptotic relative efficiency compared with the least-squares estimator. Simulation studies are conducted to evaluate the performance of the proposed methods, and a real dataset is analyzed as an illustration.     

Efficient inference for parameters of unobservable periodic autoregressive time series

This paper considers estimating the model coefficients when the observed periodic autoregressive time series is contaminated by a trend. The proposed Yule–Walker estimators are obtained by a two-step procedure. In the first step, the trend is estimated by a weighted local polynomial, and the residuals are obtained by subtracting the trend estimates from the observations; in the second step, the model coefficients are estimated by the well-known Yule–Walker method via the residuals. It is shown that under certain conditions such Yule–Walker estimators are oracally efficient, i.e., they are asymptotically equivalent to those obtained from periodic autoregressive time series without a trend. An easy-to-use implementation procedure is provided. The performance of the estimators is illustrated by simulation studies and real data analysis. In particular, the simulation studies show that the proposed estimator outperforms that obtained from the residuals when the trend is estimated by kernel smoothing without taking the heteroscedasticity into consideration.     

Risk efficient estimation of fully dependent random coefficient autoregressive models of general order

We consider a stochastic dynamic model with autoregressive progression. The drift coefficients of the autoregressive model are random where the randomness in the coefficients can have any dependence structure. We propose a two-step sequential estimator and study the asymptotic behavior of few important properties. Paradigm of sequential estimation has its own advantage in reducing sample size and plugging estimates of nuisance parameters while inferring about the main parameters. Our proposed estimator is asymptotically optimal as the predictive risk of the proposed estimator attains the risk of the oracle that assumes known nuisance parameters. Extensive simulation confirms our results.     

A semiparametric approach for modelling multivariate nonlinear time series

In this article, a semiparametric time‐varying nonlinear vector autoregressive (NVAR) model is proposed to model nonlinear vector time series data. We consider a combination of parametric and nonparametric estimation approaches to estimate the NVAR function for both independent and dependent errors. We use the multivariate Taylor series expansion of the link function up to the second order which has a parametric framework as a representation of the nonlinear vector regression function. After the unknown parameters are estimated by the maximum likelihood estimation procedure, the obtained NVAR function is adjusted by a nonparametric diagonal matrix, where the proposed adjusted matrix is estimated by the nonparametric kernel estimator. The asymptotic consistency properties of the proposed estimators are established. Simulation studies are conducted to evaluate the performance of the proposed semiparametric method. A real data example on short‐run interest rates and long‐run interest rates of United States Treasury securities is analyzed to demonstrate the application of the proposed approach. The Canadian Journal of Statistics 47: 668–687; 2019 © 2019 Statistical Society of Canada     

Asymptotic and Bootstrap Inference for AR(∞) Processes with Conditional Heteroskedasticity

The main contribution of this paper is a proof of the asymptotic validity of the application of the bootstrap to AR(∞) processes with unmodelled conditional heteroskedasticity. We first derive the asymptotic properties of the least-squares estimator of the autoregressive sieve parameters when the data are generated by a stationary linear process with martingale difference errors that are possibly subject to conditional heteroskedasticity of unknown form. These results are then used in establishing that a suitably constructed bootstrap estimator will have the same limit distribution as the least-squares estimator. Our results provide theoretical justification for the use of either the conventional asymptotic approximation based on robust standard errors or the bootstrap approximation of the distribution of autoregressive parameters. A simulation study suggests that the bootstrap approach tends to be more accurate in small samples.     

A plug-in the number of knots selector for polynomial spline regression

A plug-in the number of interior knots (NIKs) selector is proposed for polynomial spline estimation in nonparametric regression. The existence and properties of the optimal NIKs for spline regression are established by minimising the weighted mean integrated squared error. We obtain plug-in formulae for the optimal NIKs based on the theoretical results of asymptotic optimality, and develop strategies for choosing the NIKs of the spline estimator. The proposed NIKs selection method is tested on our simulated data with quite satisfactory performance, and is illustrated by analysing a fossil data set.     

Asymptotic and Bootstrap Inference for AR(∞) Processes with Conditional Heteroskedasticity

The main contribution of this paper is a proof of the asymptotic validity of the application of the bootstrap to AR(∞) processes with unmodelled conditional heteroskedasticity. We first derive the asymptotic properties of the least-squares estimator of the autoregressive sieve parameters when the data are generated by a stationary linear process with martingale difference errors that are possibly subject to conditional heteroskedasticity of unknown form. These results are then used in establishing that a suitably constructed bootstrap estimator will have the same limit distribution as the least-squares estimator. Our results provide theoretical justification for the use of either the conventional asymptotic approximation based on robust standard errors or the bootstrap approximation of the distribution of autoregressive parameters. A simulation study suggests that the bootstrap approach tends to be more accurate in small samples.     

Functional coefficient time series models with trending regressors

This paper studies a functional coe?cient time series model with trending regressors, where the coe?cients are unknown functions of time and random variables. We propose a local linear estimation method to estimate the unknown coe?cient functions, and establish the corresponding asymptotic theory under mild conditions. We also develop a test procedure to see if the functional coe?cients take particular parametric forms. For practical use, we further propose a Bayesian approach to select the bandwidths, and conduct several numerical experiments to examine the finite sample performance of our proposed local linear estimator and the test procedure. The results show that the local linear estimator works well and the proposed test has satisfactory size and power. In addition, our simulation studies show that the Bayesian bandwidth selection method performs better than the cross-validation method. Furthermore, we use the functional coe?cient model to study the relationship between consumption per capita and income per capita in United States, and it was shown that the functional coe?cient model with our proposed local linear estimator and Bayesian bandwidth selection method performs well in both in-sample fitting and out-of-sample forecasting.     

A Spline‐Based Semiparametric Maximum Likelihood Estimation Method for the Cox Model with Interval‐Censored Data

Abstract. We propose a spline‐based semiparametric maximum likelihood approach to analysing the Cox model with interval‐censored data. With this approach, the baseline cumulative hazard function is approximated by a monotone B‐spline function. We extend the generalized Rosen algorithm to compute the maximum likelihood estimate. We show that the estimator of the regression parameter is asymptotically normal and semiparametrically efficient, although the estimator of the baseline cumulative hazard function converges at a rate slower than root‐n. We also develop an easy‐to‐implement method for consistently estimating the standard error of the estimated regression parameter, which facilitates the proposed inference procedure for the Cox model with interval‐censored data. The proposed method is evaluated by simulation studies regarding its finite sample performance and is illustrated using data from a breast cosmesis study.     

Spline Confidence Bands for Functional Derivatives

We develop in this paper a new procedure to construct simultaneous confidence bands for derivatives of mean curves in functional data analysis. The technique involves polynomial splines that provide an approximation to the derivatives of the mean functions, the covariance functions and the associated eigenfunctions. We show that the proposed procedure has desirable statistical properties. In particular, we first show that the proposed estimators of derivatives of the mean curves are semiparametrically efficient. Second, we establish consistency results for derivatives of covariance functions and their eigenfunctions. Most importantly, we show that the proposed spline confidence bands are asymptotically efficient as if all random trajectories were observed with no error. Finally, the confidence band procedure is illustrated through numerical simulation studies and a real life example.     

Estimation in autoregressive models with surrogate data and validation data

Time-series data are often subject to measurement error, usually the result of needing to estimate the variable of interest. Generally, however, the relationship between the surrogate variables and the true variables can be rather complicated compared to the classical additive error structure usually assumed. In this article, we address the estimation of the parameters in autoregressive models in the presence of function measurement errors. We first develop a parameter estimation method with the help of validation data; this estimation method does not depend on functional form and the distribution of the measurement error. The proposed estimator is proved to be consistent. Moreover, the asymptotic representation and the asymptotic normality of the estimator are also derived, respectively. Simulation results indicate that the proposed method works well for practical situation.     

Improved local polynomial estimation in time series regression

We propose a modification of local polynomial estimation which improves the efficiency of the conventional method when the observation errors are correlated. The procedure is based on a pre-transformation of the data as a generalization of the pre-whitening procedure introduced by Xiao et al. [(2003), ‘More Efficient Local Polynomial Estimation in Nonparametric Regression with Autocorrelated Errors’, Journal of the American Statistical Association, 98, 980–992]. While these authors assumed a linear process representation for the error process, we avoid any structural assumption. We further allow the regressors and the errors to be dependent. More importantly, we show that the inclusion of both leading and lagged variables in the approximation of the error terms outperforms the best approximation based on lagged variables only. Establishing its asymptotic distribution, we show that the proposed estimator is more efficient than the standard local polynomial estimator. As a by-product we prove a suitable version of a central limit theorem which allows us to improve the asymptotic normality result for local polynomial estimators by Masry and Fan [(1997), ‘Local Polynomial Estimation of Regression Functions for Mixing Processes’, Scandinavian Journal of Statistics, 24, 165–179]. A simulation study confirms the efficiency of our estimator on finite samples. An application to climate data also shows that our new method leads to an estimator with decreased variability.     

Nonlinear semiparametric autoregressive model with finite mixtures of scale mixtures of skew normal innovations

We propose data generating structures which can be represented as the nonlinear autoregressive models with single and finite mixtures of scale mixtures of skew normal innovations. This class of models covers symmetric/asymmetric and light/heavy-tailed distributions, so provide a useful generalization of the symmetrical nonlinear autoregressive models. As semiparametric and nonparametric curve estimation are the approaches for exploring the structure of a nonlinear time series data set, in this article the semiparametric estimator for estimating the nonlinear function of the model is investigated based on the conditional least square method and nonparametric kernel approach. Also, an Expectation–Maximization-type algorithm to perform the maximum likelihood (ML) inference of unknown parameters of the model is proposed. Furthermore, some strong and weak consistency of the semiparametric estimator in this class of models are presented. Finally, to illustrate the usefulness of the proposed model, some simulation studies and an application to real data set are considered.     

AN EXAMINATION OF ESTIMATED RESIDUALS IN A REGRESSION WITH AN INFINITE ORDER PARAMETRIC MODEL

We consider a linear regression with the error term that obeys an autoregressive model of infinite order and estimate parameters of the models. The parameters of the autoregressive model should be estimated based on estimated residuals obtained by means of the method of ordinary least squares, because the errors are unobservable. The consistency of the coefficients, variance and spectral density of the model obeyed by the error term is shown. Further, we estimate the coefficients of the linear regression by means of the method of estimated generalized least squares. We also show the consistency of the estimator.

     

Bias-corrected confidence bands in nonparametric regression

Bias-corrected confidence bands for general nonparametric regression models are considered. We use local polynomial fitting to construct the confidence bands and combine the cross-validation method and the plug-in method to select the bandwidths. Related asymptotic results are obtained. Our simulations show that confidence bands constructed by local polynomial fitting have much better coverage than those constructed by using the Nadaraya–Watson estimator. The results are also applicable to nonparametric autoregressive time series models.     

Estimation of the Spectral Density with Assigned Risk

It is well known that adaptive sequential nonparametric estimation of differentiable functions with assigned mean integrated squared error and minimax expected stopping time is impossible. In other words, no sequential estimator can compete with an oracle estimator that knows how many derivatives an estimated curve has. Differentiable functions are typical in probability density and regression models but not in spectral density models, where considered functions are typically smoother. This paper shows that for a large class of spectral densities, which includes spectral densities of classical autoregressive moving average processes, an adaptive minimax sequential estimation with assigned mean integrated squared error is possible. Furthermore, a two‐stage sequential procedure is proposed, which is minimax and adaptive to smoothness of an underlying spectral density.