Improved model selection criteria for SETAR time series models

The purpose of this paper is threefold. First, we obtain the asymptotic properties of the modified model selection criteria proposed by Hurvich et al. (1990. Improved estimators of Kullback-Leibler information for autoregressive model selection in small samples. Biometrika 77, 709–719) for autoregressive models. Second, we provide some highlights on the better performance of this modified criteria. Third, we extend the modification introduced by these authors to model selection criteria commonly used in the class of self-exciting threshold autoregressive (SETAR) time series models. We show the improvements of the modified criteria in their finite sample performance. In particular, for small and medium sample size the frequency of selecting the true model improves for the consistent criteria and the root mean square error (RMSE) of prediction improves for the efficient criteria. These results are illustrated via simulation with SETAR models in which we assume that the threshold and the parameters are unknown.     

Empirical information criteria for time series forecasting model selection

In this article, we propose a new empirical information criterion (EIC) for model selection which penalizes the likelihood of the data by a non-linear function of the number of parameters in the model. It is designed to be used where there are a large number of time series to be forecast. However, a bootstrap version of the EIC can be used where there is a single time series to be forecast. The EIC provides a data-driven model selection tool that can be tuned to the particular forecasting task.

We compare the EIC with other model selection criteria including Akaike’s information criterion (AIC) and Schwarz’s Bayesian information criterion (BIC). The comparisons show that for the M3 forecasting competition data, the EIC outperforms both the AIC and BIC, particularly for longer forecast horizons. We also compare the criteria on simulated data and find that the EIC does better than existing criteria in that case also.     

Information criteria in identifying regression models

This paper proposes to use information criteria to discriminate the standard regression model from error components models, heteroskedastic models, or models with autocorrelated errors.     

Laplace approximations and Bayesian information criteria in possibly misspecified models

We provide general conditions to ensure the valid Laplace approximations to the marginal likelihoods under model misspecification, and derive the Bayesian information criteria including all terms of order O_p(1). Under conditions in theorem 1 of Lv and Liu [J. R. Statist. Soc. B, 76, (2014), 141–167] and a continuity condition for prior densities, asymptotic expansions with error terms of order o_p(1) are derived for the log-marginal likelihoods of possibly misspecified generalized linear models. We present some numerical examples to illustrate the finite sample performance of the proposed information criteria in misspecified models.     

A comparison of model selection criteria

There has been significant new work published recently on the subject of model selection. Notably Rissanen (1986, 1987, 1988) has introduced new criteria based on the notion of stochastic complexity and Hurvich and Tsai(1989) have introduced a bias corrected version of Akaike's information criterion. In this paper, a Monte Carlo study is conducted to evaluate the relative performance of these new model selection criteria against the commonly used alternatives. In addition, we compare the performance of all the criteria in a number of situations not considered in earlier studies: robustness to distributional assumptions, collinearity among regressors, and non-stationarity in a time series. The evaluation is based on the number of times the correct model is chosen and the out of sample prediction error. The results of this study suggest that Rissanen's criteria are sensitive to the assumptions and choices that need to made in their application, and so are sometimes unreliable. While many of the criteria often perform satisfactorily, across experiments the Schwartz Bayesian Information Criterion (and the related Bayesian Estimation Criterion of Geweke-Meese) seem to consistently outperfom the other alternatives considered.     

A comparison of model selection criteria

There has been significant new work published recently on the subject of model selection. Notably Rissanen (1986, 1987, 1988) has introduced new criteria based on the notion of stochastic complexity and Hurvich and Tsai(1989) have introduced a bias corrected version of Akaike's information criterion. In this paper, a Monte Carlo study is conducted to evaluate the relative performance of these new model selection criteria against the commonly used alternatives. In addition, we compare the performance of all the criteria in a number of situations not considered in earlier studies: robustness to distributional assumptions, collinearity among regressors, and non-stationarity in a time series. The evaluation is based on the number of times the correct model is chosen and the out of sample prediction error. The results of this study suggest that Rissanen's criteria are sensitive to the assumptions and choices that need to made in their application, and so are sometimes unreliable. While many of the criteria often perform satisfactorily, across experiments the Schwartz Bayesian Information Criterion (and the related Bayesian Estimation Criterion of Geweke-Meese) seem to consistently outperfom the other alternatives considered.     

Profile information matrix for nonlinear transformation models

For semiparametric models, interval estimation and hypothesis testing based on the information matrix for the full model is a challenge because of potentially unlimited dimension. Use of the profile information matrix for a small set of parameters of interest is an appealing alternative. Existing approaches for the estimation of the profile information matrix are either subject to the curse of dimensionality, or are ad-hoc and approximate and can be unstable and numerically inefficient. We propose a numerically stable and efficient algorithm that delivers an exact observed profile information matrix for regression coefficients for the class of Nonlinear Transformation Models [A. Tsodikov (2003) J R Statist Soc Ser B 65:759-774]. The algorithm deals with the curse of dimensionality and requires neither large matrix inverses nor explicit expressions for the profile surface.     

Variable selection in linear regression analysis with alternative Bayesian information criteria using differential evaluation algorithm

In statistical analysis, one of the most important subjects is to select relevant exploratory variables that perfectly explain the dependent variable. Variable selection methods are usually performed within regression analysis. Variable selection is implemented so as to minimize the information criteria (IC) in regression models. Information criteria directly affect the power of prediction and the estimation of selected models. There are numerous information criteria in literature such as Akaike Information Criteria (AIC) and Bayesian Information Criteria (BIC). These criteria are modified for to improve the performance of the selected models. BIC is extended with alternative modifications towards the usage of prior and information matrix. Information matrix-based BIC (IBIC) and scaled unit information prior BIC (SPBIC) are efficient criteria for this modification. In this article, we proposed a combination to perform variable selection via differential evolution (DE) algorithm for minimizing IBIC and SPBIC in linear regression analysis. We concluded that these alternative criteria are very useful for variable selection. We also illustrated the efficiency of this combination with various simulation and application studies.     

Model selection information criteria in latent class models with missing data and contingency question

Latent class analysis (LCA) has been found to have important applications in social and behavioural sciences for modelling categorical response variables, and non-response is typical when collecting data. In this study, the non-response mainly included ‘contingency questions’ and real ‘missing data’. The primary objective of this study was to evaluate the effects of some potential factors on model selection indices in LCA with non-response data. We simulated missing data with contingency question and evaluated the accuracy rates of eight information criteria for selecting the correct models. The results showed that the main factors are latent class proportions, conditional probabilities, sample size, the number of items, the missing data rate and the contingency data rate. Interactions of the conditional probabilities with class proportions, sample size and the number of items are also significant. From our simulation results, the impact of missing data and contingency questions can be amended by increasing the sample size or the number of items.     

Focused information criterion for locally misspecified vector autoregressive models

This paper investigates the focused information criterion and plug-in average for vector autoregressive models with local-to-zero misspecification. These methods have the advantage of focusing on a quantity of interest rather than aiming at overall model fit. Any (su?ciently regular) function of the parameters can be used as a quantity of interest. We determine the asymptotic properties and elaborate on the role of the locally misspecified parameters. In particular, we show that the inability to consistently estimate locally misspecified parameters translates into suboptimal selection and averaging. We apply this framework to impulse response analysis. A Monte Carlo simulation study supports our claims.     

Consistency of information criteria for model selection with missing data

ABSTRACT

In this paper, we investigate the consistency of the Expectation Maximization (EM) algorithm-based information criteria for model selection with missing data. The criteria correspond to a penalization of the conditional expectation of the complete data log-likelihood given the observed data and with respect to the missing data conditional density. We present asymptotic properties related to maximum likelihood estimation in the presence of incomplete data and we provide sufficient conditions for the consistency of model selection by minimizing the information criteria. Their finite sample performance is illustrated through simulation and real data studies.     

Variable selection and estimation for high-dimensional spatial autoregressive models

Spatial regression models are important tools for many scientific disciplines including economics, business, and social science. In this article, we investigate postmodel selection estimators that apply least squares estimation to the model selected by penalized estimation in high-dimensional regression models with spatial autoregressive errors. We show that by separating the model selection and estimation process, the postmodel selection estimator performs at least as well as the simultaneous variable selection and estimation method in terms of the rate of convergence. Moreover, under perfect model selection, the ℓ₂ rate of convergence is the oracle rate of $\sqrt{s/n}$, compared with the convergence rate of $\text{◂√▸}\sqrt{s\log p/n}$ in the general case. Here, n is the sample size and p, s are the model dimension and number of significant covariates, respectively. We further provide the convergence rate of the estimation error in the form of $\sup$ norm, and ideally the rate can reach as fast as $\text{◂√▸}\sqrt{\log s/n}$.     

On the use of area-wide models in the Euro-zone

This paper investigates the legitimacy of using area-wide models in predicting aggregate variables in the Euro-area. We aim to compare the performance of area-wide versus national specific models for modeling money demand when using different aggregation schemes. A generalized Grunfeld and Griliches criterion and the Vuong test are used to discriminate between competitive models. Results show that the use of different aggregation methods is not irrelevant. In fact, due to the volatility of the exchange rates, the aggregate models fit better than the disaggregate whenever we employ ECU exchange rates. However, for fixed exchange rates expressed in Euro, the disaggregate models outperform the aggregate ones. This paper was written during my visiting research period at the Department of Economics, University of Southampton. I wish to thank John Aldrich, Jan Podivinsky, Grayham Mizon and Akos Valentinyi. Financial support of the Universita degli Studi “Roma Tre” and the Marie Curie fellowship (HPMT-CT-2001-00353) are gratefully acknowledged.     

Model selection using information criteria under a new estimation method: least squares ratio

In this study, we evaluate several forms of both Akaike-type and Information Complexity (ICOMP)-type information criteria, in the context of selecting an optimal subset least squares ratio (LSR) regression model. Our simulation studies are designed to mimic many characteristics present in real data – heavy tails, multicollinearity, redundant variables, and completely unnecessary variables. Our findings are that LSR in conjunction with one of the ICOMP criteria is very good at selecting the true model. Finally, we apply these methods to the familiar body fat data set.     

Model Selection and Regression t-Statistics

It is shown that dropping quantitative variables from a linear regression, based on t-statistics, is mathematically equivalent to dropping variables based on commonly used information criteria.     

Consistency of a class of information criteria for model selection in non-linear regression

In this paper we prove the consistency in probability of a class of generalized BIC criteria for model selection in non-linear regression, by using asymptotic results of Gallant. This extends a result obtained by Nishii for model selection in linear regression.     

Model selection using union-intersection principle for non nested models

In this article, we have extended the Vuong’s (1989 Vuong, Q.H. (1989). Likelihood ratio tests for model selection and non-nested hypothesis. Econometrica. 57:307–333.[Crossref], [Web of Science ®] , [Google Scholar]) model selection test to three models in accordance to union-intersection principle. Using the Kullback–Leibler criterion to measure the closeness of a model to the truth, we propose a simple likelihood ratio-based statistics for testing the null hypothesis that the competing models are equally close to the true data-generating process against the alternative hypothesis that at least one model is closer. We show that the distribution of the test statistic is asymptotically equal to the distribution of the maximum of dependent random variables with bivariate folded standard normal distribution. The density function of the maximum of dependent random variables with elliptically contoured distributions has been obtained by other researchers, but, not for distributions which do not belong to the elliptically contoured distributions family. In this article, the exact distribution of the maximum of dependent random variables with bivariate folded standard normal distribution is calculated as an asymptotic distribution of the proposed test statistic. The test is directional and is derived successively for the cases where the competing models are non nested and whether three, two, one, or none of them are misspecified.     

Model selection criteria in beta regression with varying dispersion

We address the issue of model selection in beta regressions with varying dispersion. The model consists of two submodels, namely: for the mean and for the dispersion. Our focus is on the selection of the covariates for each submodel. Our Monte Carlo evidence reveals that the joint selection of covariates for the two submodels is not accurate in finite samples. We introduce two new model selection criteria that explicitly account for varying dispersion and propose a fast two step model selection scheme which is considerably more accurate and is computationally less costly than usual joint model selection. Monte Carlo evidence is presented and discussed. We also present the results of an empirical application.     

Tracking interval for selecting between non-nested models: An investigation for Type II right censored data

In this paper, we consider the setting where the observed data is incomplete. For the general situation where the number of gaps as well as the number of unobserved values in some gaps go to infinity, the asymptotic behavior of maximum likelihood estimator is not clear. We derive and investigate the asymptotic properties of maximum likelihood estimator under censorship and drive a statistic for testing the null hypothesis that the proposed non-nested models are equally close to the true model against the alternative hypothesis that one model is closer when we are faced with a life-time situation. Furthermore rewrite a normalization of a difference of Akaike criterion for estimating the difference of expected Kullback–Leibler risk between the distributions in two different models.     

Maximum likelihood estimation for generalized conditionally autoregressive models of spatial data

Conditionally autoregressive (CAR) models are often used to analyze a spatial process observed over a lattice or a set of irregular regions. The neighborhoods within a CAR model are generally formed deterministically using the inter-distances or boundaries between the regions. To accommodate directional and inherent anisotropy variation, a new class of spatial models is proposed that adaptively determines neighbors based on a bivariate kernel using the distances and angles between the centroid of the regions. The newly proposed model generalizes the usual CAR model in a sense of accounting for adaptively determined weights. Maximum likelihood estimators are derived and simulation studies are presented for the sampling properties of the estimates on the new model, which is compared to the CAR model. Finally the method is illustrated using a data set on the elevated blood lead levels of children under the age of 72 months observed in Virginia in the year of 2000.