GENERALIZED INVERSES USED IN RECURSIVE ESTIMATION OF THE GENERAL LINEAR MODEL1

Recursive methods in regression have proved useful in providing diagnostic tools for checking the model as well as checking the stability of the model over time. Such methods are now extended to deal with the problems of singularity that arise when one variable is completely confounded with previously fitted variables up to a particular time point. The problem is solved by setting it in the framework of the general linear model with dependent errors.     

A SIMPLE VARIABLE SELECTION TECHNIQUE FOR NONLINEAR MODELS

Applying nonparametric variable selection criteria in nonlinear regression models generally requires a substantial computational effort if the data set is large. In this paper we present a selection technique that is computationally much less demanding and performs well in comparison with methods currently available. It is based on a polynomial approximation of the nonlinear model. Performing the selection only requires repeated least squares estimation of models that are linear in parameters. The main limitation of the method is that the number of variables among which to select cannot be very large if the sample is small and the order of an adequate polynomial at the same time is high. Large samples can be handled without problems.     

PREDICTION-FOCUSED MODEL SELECTION FOR AUTOREGRESSIVE MODELS

In order to make predictions of future values of a time series, one needs to specify a forecasting model. A popular choice is an autoregressive time‐series model, for which the order of the model is chosen by an information criterion. We propose an extension of the focused information criterion (FIC) for model‐order selection, with emphasis on a high predictive accuracy (i.e. the mean squared forecast error is low). We obtain theoretical results and illustrate by means of a simulation study and some real data examples that the FIC is a valid alternative to the Akaike information criterion (AIC) and the Bayesian information criterion (BIC) for selection of a prediction model. We also illustrate the possibility of using the FIC for purposes other than forecasting, and explore its use in an extended model.     

ROBUST ESTIMATION FOR EPIDEMIC MODELS

A robust approach to the analysis of epidemic data is suggested. This method is based on a natural extension of M-estimation for i.i.d. observations where the distribution may be asymmetric. It is discussed initially in the context of a general discrete time stochastic process before being applied to previously studied epidemic models. In particular we consider a class of chain binomial models and models based on time dependent branching processes. Robustness and efficiency properties are studied through simulation and some previously analysed data sets are considered.     

ORDER SELECTION IN ARMA MODELS USING THE FOCUSED INFORMATION CRITERION

This paper develops a new approach for order selection in autoregressive moving average models using the focused information criterion. This criterion minimizes the asymptotic mean squared error of the estimator of a parameter of interest. Simulation studies indicate that the suggested criterion is quite effective and comparable to the Akaike information criterion, the corrected Akaike information criterion and the Bayesian information criterion in autoregressive moving average order selection. The use of the focused information criterion for the simultaneous selection of regression variables and order of the error process in a linear regression model with autoregressive moving average errors is also considered.     

MODEL SELECTION CRITERIA FOR LOGLINEAR MODELS

Considerable work has been devoted to developing model selection criteria for normal theory regression models. Less attention has been paid to discrete data. We develop two loglinear model selection criteria for Poisson counts. These criteria are based on an estimated bias adjustment of the Akaike information criterion. We observe in a simulation study that the corrected statistics provide good model choices and relatively accurate estimates of the mean structure.     

PARAMETER ESTIMATION FOR LOW-ORDER AUTO-REGRESSIVE MODELS WITH MISSING VALUES

Longitudinal data analysis in epidemiological settings is complicated by large multiplicities of short time series and the occurrence of missing observations. To handle such difficulties Rosner & Muñoz (1988) developed a weighted non-linear least squares algorithm for estimating parameters for first-order autoregressive (AR1) processes with time-varying covariates. This method proved efficient when compared to complete case procedures. Here that work is extended by (1) introducing a different estimation procedure based on the EM algorithm, and (2) formulating estimation techniques for second-order autoregressive models. The second development is important because some of the intended areas of application (adult pulmonary function decline, childhood blood pressure) have autocorrelation functions which decay more slowly than the geometric rate imposed by an AR1 model. Simulation studies are used to compare the three methodologies (non-linear, EM based and complete case) with respect to bias, efficiency and coverage both in the presence and in the absence of time-varying covariates. Differing degrees and mechanisms of missingness are examined. Preliminary results indicate the non-linear approach to be the method of choice: it has high efficiency and is easily implemented. An illustrative example concerning pulmonary function decline in the Netherlands is analyzed using this method.     

A MORE GENERAL CRITERION FOR SUBSET SELECTION IN MULTIPLE LINEAR REGRESSION

ABSTRACT

In this article, we propose a more general criterion called S_p -criterion, for subset selection in the multiple linear regression Model. Many subset selection methods are based on the Least Squares (LS) estimator of β, but whenever the data contain an influential observation or the distribution of the error variable deviates from normality, the LS estimator performs ‘poorly’ and hence a method based on this estimator (for example, Mallows’ C_p -criterion) tends to select a ‘wrong’ subset. The proposed method overcomes this drawback and its main feature is that it can be used with any type of estimator (either the LS estimator or any robust estimator) of β without any need for modification of the proposed criterion. Moreover, this technique is operationally simple to implement as compared to other existing criteria. The method is illustrated with examples.     

NEW EFFICIENT ESTIMATION AND VARIABLE SELECTION METHODS FOR SEMIPARAMETRIC VARYING-COEFFICIENT PARTIALLY LINEAR MODELS

The complexity of semiparametric models poses new challenges to statistical inference and model selection that frequently arise from real applications. In this work, we propose new estimation and variable selection procedures for the semiparametric varying-coefficient partially linear model. We first study quantile regression estimates for the nonparametric varying-coefficient functions and the parametric regression coefficients. To achieve nice efficiency properties, we further develop a semiparametric composite quantile regression procedure. We establish the asymptotic normality of proposed estimators for both the parametric and nonparametric parts and show that the estimators achieve the best convergence rate. Moreover, we show that the proposed method is much more efficient than the least-squares-based method for many non-normal errors and that it only loses a small amount of efficiency for normal errors. In addition, it is shown that the loss in efficiency is at most 11.1% for estimating varying coefficient functions and is no greater than 13.6% for estimating parametric components. To achieve sparsity with high-dimensional covariates, we propose adaptive penalization methods for variable selection in the semiparametric varying-coefficient partially linear model and prove that the methods possess the oracle property. Extensive Monte Carlo simulation studies are conducted to examine the finite-sample performance of the proposed procedures. Finally, we apply the new methods to analyze the plasma beta-carotene level data.     

CORRECTING FOR KURTOSIS IN DENSITY ESTIMATION

Using a global window width kernel estimator to estimate an approximately symmetric probability density with high kurtosis usually leads to poor estimation because good estimation of the peak of the distribution leads to unsatisfactory estimation of the tails and vice versa. The technique proposed corrects for kurtosis via a transformation of the data before using a global window width kernel estimator. The transformation depends on a “generalised smoothing parameter” consisting of two real-valued parameters and a window width parameter which can be selected either by a simple graphical method or, for a completely data-driven implementation, by minimising an estimate of mean integrated squared error. Examples of real and simulated data demonstrate the effectiveness of this approach, which appears suitable for a wide range of symmetric, unimodal densities. Its performance is similar to ordinary kernel estimation in situations where the latter is effective, e.g. Gaussian densities. For densities like the Cauchy where ordinary kernel estimation is not satisfactory, our methodology offers a substantial improvement.     

AN EXACT CONDITIONAL TEST FOR COVARIANCE SELECTION MODELS

An exact conditional test is developed for testing the absence of an edge in a graphical covariance selection model and is shown to be equivalent to a test based on the partial correlation coefficient. An example is given.     

PARTIALLY LINEAR MODEL SELECTION BY THE BOOTSTRAP

We propose a new approach to the selection of partially linear models based on the conditional expected prediction square loss function, which is estimated using the bootstrap. Because of the different speeds of convergence of the linear and the nonlinear parts, a key idea is to select each part separately. In the first step, we select the nonlinear components using an ' m -out-of- n ' residual bootstrap that ensures good properties for the nonparametric bootstrap estimator. The second step selects the linear components from the remaining explanatory variables, and the non-zero parameters are selected based on a two-level residual bootstrap. We show that the model selection procedure is consistent under some conditions, and our simulations suggest that it selects the true model most often than the other selection procedures considered.     

ON QUASI-LIKELIHOOD METHODS AND ESTIMATION FOR BUNCHING PROCESSES AND HETEROSCEDASTIC REGRESSION MODELS

This paper is concerned with quasi-likelihood estimation methods which provide benchmarks in efficiency against which other methods may be as sessed. We discuss general principles and illustrate them by the estimation of parameters for branching processes with immigration and heteroscedastic regression models.     

MULTIPLE COMPARISONS OF LOG-LIKELIHOODS AND COMBINING NONNESTED MODELS WITH APPLICATIONS TO PHYLOGENETIC TREE SELECTION

We consider multiple comparisons of log-likelihood's to take account of the multiplicity of testings in selection of nonnested models. A resampling version of the Gupta procedure for the selection problem is used to obtain a set of good models, which are not significantly worse than the maximum likelihood model; i.e., a confidence set of models. Our method is to test which model is better than the other, while the object of the classical testing methods is to find the correct model. Thus the null hypotheses behind these two approaches are very different. Our method and the other commonly used approaches, such as the approximate Bayesian posterior, the bootstrap selection probability, and the LR test against the full model, are applied to the selection of molecular phylogenetic tree of mammal species. Tree selection is a version of the model-based clustering, which is an example of nonnested model selection. It is shown that the structure of the tree selection problem is equivalent to that of the variable selection problem of the multiple regression with some constraints on the combinations of the variables. It turns out that the LR test rejects all the possible trees because of the misspecification of the models, whereas our method gives a reasonable confidence set. For a better understanding of the uncertainty in the selection, we combine the maximum likelihood estimates (MLE's) of the trees to obtain the full model that includes the trees as the submodels by using a linear approximation of the parametric models. The MLE of the phylogeny is then represented as a network of species rather than a tree. A geometrical interpretation of the problem is also discussed.     

ROBUST VARIABLE SELECTION IN REGRESSION IN THE PRESENCE OF OUTLIERS AND LEVERAGE POINTS

In linear regression, outliers and leverage points often have large influence in the model selection process. Such cases are downweighted with Mallows-type weights here, during estimation of submodel parameters by generalised M-estimation. A robust version of Mallows's Cp (Ronchetti &. Staudte, 1994) is then used to select a variety of submodels which are as informative as the full model. The methodology is illustrated on a new dataset concerning the agglomeration of alumina in Bayer precipitation.     

LARGE-SAMPLE INFERENCE FOR THE GENERAL HALF-NORMAL DISTRIBUTION

ABSTRACT

The paper considers the development of inferential techniques for the general half-normal distribution based on the maximum likelihood method. Point estimates and large-sample confidence sets are derived for the distribution's parameters, and the use of the confidence sets as a basis for hypothesis testing is discussed. Data on the percentage body fat of elite male athletes are used to illustrate the application of the new methods.     

NON-PARAMETRIC ESTIMATION OF DIRECTION IN SINGLE-INDEX MODELS WITH CATEGORICAL PREDICTORS

This paper proposes a general dimension‐reduction method targeting the partial central subspace recently introduced by Chiaromonte, Cook & Li. The dependence need not be confined to particular conditional moments, nor are restrictions placed on the predictors that are necessary for methods like partial sliced inverse regression. The paper focuses on a partially linear single‐index model. However, the underlying idea is applicable more generally. Illustrative examples are presented.     

ROBUST ESTIMATION AND HYPOTHESIS TESTS FOR FIRST-ORDER THRESHOLD AUTOREGRESSIVE MODELS

Results of Petrucelli & Woolford (1984) for a first-order threshold autoregressive model are considered from a robust point of view. Robust estimators of the threshold parameters of the model are obtained and their asymptotic normality is proved. Testing the equality of the threshold parameters is considered using the robust analogues of Wald and score test statistics. Limiting distributions of these statistics are given under both null and alternative hypotheses.     

ON THE HILDRETH-HOUCK ESTIMATOR FOR RANDOM COEFFICIENT REGRESSION MODELS

This paper is concerned with the linear regression model in which the coefficients are random variables. The Hildreth-Houck method is considered for estimating the model. Sufficient conditions for the consistency of the Hildreth-Houck estimator are discussed and its asymptotic normality is established.