LASSO order selection for sparse autoregression: a bootstrap approach

Autoregressive models are widely employed for predictions and other inferences in many scientific fields. While the determination of their order is in general a difficult and critical step, this task becomes more complicated and crucial when the time series under investigation is realization of a stochastic process characterized by sparsity. In this paper we present a method for order determination of a stationary AR model with a sparse structure, given a set of observations, based upon a bootstrapped version of MAICE procedure [Akaike H. Prediction and entropy. Springer; 1998], in conjunction with a LASSO-type constraining procedure for lag suppression of insignificant lags. Empirical results will be obtained via Monte Carlo simulations. The quality of our method is assessed by comparison with the commonly adopted cross-validation approach and the non bootstrap counterpart of the presented procedure.     

Bootstrap-based ARMA order selection

Modelling the underlying stochastic process is one of the main goals in the study of many dynamic phenomena, such as signal processing, system identification and time series. The issue is often addressed within the framework of ARMA (Autoregressive Moving Average) paradigm, so that the related task of identification of the ‘true’ order is crucial. As it is well known, the effectiveness of such an approach may be seriously compromised by misspecification errors since they may affect model capabilities in capturing dynamic structures of the process. As a result, inference and empirical outcomes may be heavily misleading. Despite the big number of available approaches aimed at determining the order of an ARMA model, the issue is still open. In this paper, we bring the problem in the framework of bootstrap theory in conjunction with the information-based criterion of Akaike (AIC), and a new method for ARMA model selection will be presented. A theoretical justification for the proposed approach as well as an evaluation of its small sample performances, via simulation study, are given.     

Improved model selection criteria for SETAR time series models

The purpose of this paper is threefold. First, we obtain the asymptotic properties of the modified model selection criteria proposed by Hurvich et al. (1990. Improved estimators of Kullback-Leibler information for autoregressive model selection in small samples. Biometrika 77, 709–719) for autoregressive models. Second, we provide some highlights on the better performance of this modified criteria. Third, we extend the modification introduced by these authors to model selection criteria commonly used in the class of self-exciting threshold autoregressive (SETAR) time series models. We show the improvements of the modified criteria in their finite sample performance. In particular, for small and medium sample size the frequency of selecting the true model improves for the consistent criteria and the root mean square error (RMSE) of prediction improves for the efficient criteria. These results are illustrated via simulation with SETAR models in which we assume that the threshold and the parameters are unknown.     

Bootstrap bandwidth selection method for local linear estimator in exponential family models

Many biological experiments involve data whose distribution belongs to the exponential family. Such data are often analysed using generalised linear models but this method requires specification of the link function which can have strong influence on the resulting estimate. Instead a local method based on quasi-likelihood can be used, but the choice of the smoothing parameter is crucial for its performance. A bootstrap bandwidth selection method is proposed and shown to be consistent. Examples of application to data from biological and psychometric experiments are given.     

Moving Block Bootstrap for Analyzing Longitudinal Data

In a longitudinal study subjects are followed over time. I focus on a case where the number of replications over time is large relative to the number of subjects in the study. I investigate the use of moving block bootstrap methods for analyzing such data. Asymptotic properties of the bootstrap methods in this setting are derived. The effectiveness of these resampling methods is also demonstrated through a simulation study.     

Adaptive order selection for autoregressive models

Autoregressive model is a popular method for analysing the time dependent data, where selection of order parameter is imperative. Two commonly used selection criteria are the Akaike information criterion (AIC) and the Bayesian information criterion (BIC), which are known to suffer the potential problems regarding overfit and underfit, respectively. To our knowledge, there does not exist a criterion in the literature that can satisfactorily perform under various situations. Therefore, in this paper, we focus on forecasting the future values of an observed time series and propose an adaptive idea to combine the advantages of AIC and BIC but to mitigate their weaknesses based on the concept of generalized degrees of freedom. Instead of applying a fixed criterion to select the order parameter, we propose an approximately unbiased estimator of mean squared prediction errors based on a data perturbation technique for fairly comparing between AIC and BIC. Then use the selected criterion to determine the final order parameter. Some numerical experiments are performed to show the superiority of the proposed method and a real data set of the retail price index of China from 1952 to 2008 is also applied for illustration.     

Using the Bootstrap to Test for Symmetry Under Unknown Dependence

This article considers tests for symmetry of the one-dimensional marginal distribution of fractionally integrated processes. The tests are implemented by using an autoregressive sieve bootstrap approximation to the null sampling distribution of the relevant test statistics. The sieve bootstrap allows inference on symmetry to be carried out without knowledge of either the memory parameter of the data or of the appropriate norming factor for the test statistic and its asymptotic distribution. The small-sample properties of the proposed method are examined by means of Monte Carlo experiments, and applications to real-world data are also presented.     

Test of Significance in order selection

To select a proper order is a critical step in fitting a model to a data set. It is well known that Mallows' C_p,as well as other asymptotically equivalent order selectors, often selects an excessively large order. To reduce the chance of overestimating the optimal order using a heavier penalty term has been suggested. In this paper, the issue about setting the penalty term is considered from a perspective similar to hypothesis testing. It is suggested setting the penalty according to the probabiiity of overestimating the true or optimal order. A test procedure based on the order selecters is proposed to assess the significance of the order selected. Some simulation results are presented to illustrate the properties of the order selectors and the test procedure. The application of the procedures to two real examples are also presented.     

Some efficient computational procedures for high order ARMA models

Recursive methods are commonly used to solve Yule—Walker equations for autoregrsssive parameters given an autocovariance function. The reverse procedure can be extended to the efficient solution of various sets of equations which arise in time series analysis. Those presented in this paper include computation of the autocovariance function of an ARMA model, and the Cramer—Wold factorization.     

Bootstrap entropy test for general location-scale time series models with heteroscedasticity

This study considers a goodness-of-fit test for location-scale time series models with heteroscedasticity, including a broad class of generalized autoregressive conditional heteroscedastic-type models. In financial time series analysis, the correct identification of model innovations is crucial for further inferences in diverse applications such as risk management analysis. To implement a goodness-of-fit test, we employ the residual-based entropy test generated from the residual empirical process. Since this test often shows size distortions and is affected by parameter estimation, its bootstrap version is considered. It is shown that the bootstrap entropy test is weakly consistent, and thereby its usage is justified. A simulation study and data analysis are conducted by way of an illustration.     

Bootstrap test of hypothesis for the multi-state models in survival analysis

Abstract

In the present paper we develop bootstrap tests of hypothesis, based on simulation, for the transition probability matrix arising in the context of a multi-state model. The bootstrap test statistic is based on the paper of Tattar and Vaman (2008 Tattar, P. N., Vaman, H. J. (2008). Testing transition probability matrix of a multi-state model with censored data. Lifetime Data Anal. 14(2):216–230.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar]), which develops a statistic for the testing problems concerning the transition probability matrix of the non homogeneous Markov process.     

Model selection of generalized partially linear models with missing covariates

In this paper, a generalized partially linear model (GPLM) with missing covariates is studied and a Monte Carlo EM (MCEM) algorithm with penalized-spline (P-spline) technique is developed to estimate the regression coefficients and nonparametric function, respectively. As classical model selection procedures such as Akaike's information criterion become invalid for our considered models with incomplete data, some new model selection criterions for GPLMs with missing covariates are proposed under two different missingness mechanism, say, missing at random (MAR) and missing not at random (MNAR). The most attractive point of our method is that it is rather general and can be extended to various situations with missing observations based on EM algorithm, especially when no missing data involved, our new model selection criterions are reduced to classical AIC. Therefore, we can not only compare models with missing observations under MAR/MNAR settings, but also can compare missing data models with complete-data models simultaneously. Theoretical properties of the proposed estimator, including consistency of the model selection criterions are investigated. A simulation study and a real example are used to illustrate the proposed methodology.     

A new model selection procedure for finite mixture regression models

Abstract

In this article, we propose a new penalized-likelihood method to conduct model selection for finite mixture of regression models. The penalties are imposed on mixing proportions and regression coefficients, and hence order selection of the mixture and the variable selection in each component can be simultaneously conducted. The consistency of order selection and the consistency of variable selection are investigated. A modified EM algorithm is proposed to maximize the penalized log-likelihood function. Numerical simulations are conducted to demonstrate the finite sample performance of the estimation procedure. The proposed methodology is further illustrated via real data analysis.     

Bootstrap methods for heteroskedastic regression models: evidence on estimation and testing

This paper uses Monte Carlo simulation analysis to study the finite-sample behavior of bootstrap estimators and tests in the linear heteroskedastic model. We consider four different bootstrapping schemes, three of them specifically tailored to handle heteroskedasticity. Our results show that weighted bootstrap methods can be successfully used to estimate the variances of the least squares estimators of the linear parameters both under normality and under nonnormality. Simulation results are also given comparing the size and power of the bootstrapped Breusch-Pagan test with that of the original test and of Bartlett and Edgeworth-corrected tests. The bootstrap test was found to be robust against unfavorable regression designs.     

Estimating the order of moving average models: the max X2 method

The max X² technique for estimating rhe order of autoregressive processes (McClave (1976)) is extended to moving average models. The autöregressive-moving average duality is exploited by using the inverse autocorrelation function and the subset autoregression algorithm. The technique is demonstrated via simulations, and is applied to Box and Jenkins (1970) Series A.     

Variable selection strategies in survival models with multiple imputations

In this paper, the variable selection strategies (criteria) are thoroughly discussed and their use in various survival models is investigated. The asymptotic efficiency property, in the sense of Shibata Ann Stat 8: 147-164, 1980, of a class of variable selection strategies which includes the AIC and all criteria equivalent to it, is established for a general class of survival models, such as parametric frailty or transformation models and accelerated failure time models, under minimum conditions. Furthermore, a multiple imputations method is proposed which is found to successfully handle censored observations and constitutes a competitor to existing methods in the literature. A number of real and simulated data are used for illustrative purposes.     

Computational Examples of a New Method for Distribution Selection in the Pearson System

A considerable problem in statistics and risk management is finding distributions that capture the complex behaviour exhibited by financial data. The importance of higher order moments in decision making has been well recognized and there is increasing interest in modelling with distributions that are able to account for these effects. The Pearson system can be used to model a wide scale of distributions with various skewness and kurtosis. This paper provides computational examples of a new easily implemented method for selecting probability density functions from the Pearson family of distributions. We apply this method to daily, monthly, and annual series using a range of data from commodity markets to macroeconomic variables.     

Random Weighted Bootstrap Method for Recurrent Events with Informative Censoring

Using the data from the AIDS Link to Intravenous Experiences cohort study as an example, an informative censoring model was used to characterize the repeated hospitalization process of a group of patients. Under the informative censoring assumption, the estimators of the baseline rate function and the regression parameters were shown to be related to a latent variable. Hence, it becomes impractical to directly estimate the unknown quantities in the moments of the estimators for the bandwidth selection of a smoothing estimator and the construction of confidence intervals, which are respectively based on the asymptotic mean squared errors and the asymptotic distributions of the estimators. To overcome these difficulties, we develop a random weighted bootstrap procedure to select appropriate bandwidths and to construct approximated confidence intervals. One can see that our method is simple and faster to implement from a practical point of view, and is at least as accurate as other bootstrap methods. In this article, it is shown that the proposed method is useful through the performance of a Monte Carlo simulation. An application of our procedure is also illustrated by a recurrent event sample of intravenous drug users for inpatient cares over time.     

Component selection in additive quantile regression models

Nonparametric additive models are powerful techniques for multivariate data analysis. Although many procedures have been developed for estimating additive components both in mean regression and quantile regression, the problem of selecting relevant components has not been addressed much especially in quantile regression. We present a doubly-penalized estimation procedure for component selection in additive quantile regression models that combines basis function approximation with a ridge-type penalty and a variant of the smoothly clipped absolute deviation penalty. We show that the proposed estimator identifies relevant and irrelevant components consistently and achieves the nonparametric optimal rate of convergence for the relevant components. We also provide an accurate and efficient computation algorithm to implement the estimator and demonstrate its performance through simulation studies. Finally, we illustrate our method via a real data example to identify important body measurements to predict percentage of body fat of an individual.