PARTIALLY LINEAR MODEL SELECTION BY THE BOOTSTRAP

We propose a new approach to the selection of partially linear models based on the conditional expected prediction square loss function, which is estimated using the bootstrap. Because of the different speeds of convergence of the linear and the nonlinear parts, a key idea is to select each part separately. In the first step, we select the nonlinear components using an ' m -out-of- n ' residual bootstrap that ensures good properties for the nonparametric bootstrap estimator. The second step selects the linear components from the remaining explanatory variables, and the non-zero parameters are selected based on a two-level residual bootstrap. We show that the model selection procedure is consistent under some conditions, and our simulations suggest that it selects the true model most often than the other selection procedures considered.     

SADDLEPOINT METHODS FOR BOOTSTRAP CONFIDENCE BANDS IN NONPARAMETRIC REGRESSION

Bootstrap techniques have been used to construct confidence bands in nonparametric regression problems (Härdle & Bowman, 1988). Yet the required simulation is generally computationally intensive and therefore makes it difficult to conduct further investigations. In this paper, two saddlepoint methods are considered as alternatives to the naive simulation procedure. Some improvements to Härdle & Bowman's bootstrap method are suggested. The improvements are numerically verified using these efficient and accurate analytic methods.     

JAMES-STEIN RULE ESTIMATORS IN LINEAR REGRESSION MODELS WITH MULTIVARIATE-t DISTRIBUTED ERROR

This paper considers estimation of β in the regression model y =Xβ+μ, where the error components in μ have the jointly multivariate Student-t distribution. A family of James-Stein type estimators (characterised by nonstochastic scalars) is presented. Sufficient conditions involving only X are given, under which these estimators are better (with respect to the risk under a general quadratic loss function) than the usual minimum variance unbiased estimator (MVUE) of β. Approximate expressions for the bias, the risk, the mean square error matrix and the variance-covariance matrix for the estimators in this family are obtained. A necessary and sufficient condition for the dominance of this family over MVUE is also given.     

IMPROVED SCORE TESTS FOR VON MISES REGRESSION MODELS

We present, in matrix notation, a finite-sample correction formula to improve score tests in von Mises regression models with concentration covariates. The formula only requires simple operations on matrices and can be used to obtain analytically closed-form corrections for score test statistics in a variety of special von Mises models. The paper also provides a numerical comparison of the size of two score test statistics with bootstrap-based critical values.     

ESTIMATING THE ROOT OF A NONPARAMETRIC REGRESSION FUNCTION IN A ROBUST FASHION

For a nonparametric regression model y = m(x)+e with n independent observations, we analyze a robust method of finding the root of m(x) based on an M-estimation first discussed by Härdle & Gasser (1984). It is shown here that the robustness properties (minimaxity and breakdown function) of such an estimate are quite analogous to those of an M -estimator in the simple location model, but the rate of convergence is somewhat limited due to the nonparametric nature of the problem.     

THE EXAMINATION AND ANALYSIS OF RESIDUALS FOR SOME BIASED ESTIMATORS IN LINEAR REGRESSION

In the presence of collinearity certain biased estimation procedures like ridge regression, generalized inverse estimator, principal component regression, Liu estimator, or improved ridge and Liu estimators are used to improve the ordinary least squares (OLS) estimates in the linear regression model. In this paper new biased estimator (Liu estimator), almost unbiased (improved) Liu estimator and their residuals will be analyzed and compared with OLS residuals in terms of mean-squared error.     

NONPARAMETRIC REGRESSION ON FUNCTIONAL DATA: INFERENCE AND PRACTICAL ASPECTS

We consider the problem of predicting a real random variable from a functional explanatory variable. The problem is tackled using a nonparametric kernel approach, which has been recently adapted to this functional context. We derive theoretical results from a deep asymptotic analysis of the behaviour of the estimate, including mean squared convergence (with rates and precise evaluation of the constant terms) as well as asymptotic distribution. Practical use of these results relies on the ability to estimate these constants. Some perspectives in this direction are discussed. In particular, a functional version of wild bootstrapping ideas is proposed and used both on simulated and real functional datasets.     

SECOND-ORDER ASYMPTOTICS FOR SCORE TESTS IN HETEROSKEDASTIC t REGRESSION MODELS

ABSTRACT

This paper develops corrected score tests for heteroskedastic t regression models, thus generalizing results by Cordeiro, Ferrari and Paula[1] Cordeiro, G.M., Ferrari, S.L.P. and Paula, G.A. 1993. Improved Score Tests for Generalized Linear Models. Journal of the Royal Statistical Society B, 55: 661–674.  [Google Scholar] and Cribari-Neto and Ferrari[2] Cribari-Neto, F. and Ferrari, S.L.P. 1995. Second-order Asymptotics for Score Tests in Generalised Linear Models. Biometrika, 82: 426–432. [Crossref], [Web of Science ®] , [Google Scholar] for normal regression models and by Ferrari and Arellano-Valle[3] Ferrari, S.L.P. and Arellano-Valle, R. 1996. Modified Likelihood Ratio and Score Tests in Linear Regression Models Using the t Distribution. Brazilian Journal of Probability and Statistics, 10: 15–33.  [Google Scholar] for homoskedastic t regression models. We present, in matrix notation, Bartlett-type correction formulae to improve score tests in this class of models. The corrected score statistics have a chi-squared distribution to order n ^?1, where n is the sample size. We apply our main result to a few special models and present simulation results comparing the performance of the usual score tests and their corrected versions.     

ROBUST ESTIMATION AND HYPOTHESIS TESTS FOR FIRST-ORDER THRESHOLD AUTOREGRESSIVE MODELS

Results of Petrucelli & Woolford (1984) for a first-order threshold autoregressive model are considered from a robust point of view. Robust estimators of the threshold parameters of the model are obtained and their asymptotic normality is proved. Testing the equality of the threshold parameters is considered using the robust analogues of Wald and score test statistics. Limiting distributions of these statistics are given under both null and alternative hypotheses.     

THE POWER AND SIZE OF NONPARAMETRIC TESTS FOR COMMON DISTRIBUTIONAL CHARACTERISTICS

This paper considers the power and size properties of some well known nonparametric linear rank tests for location and scale as well as the Kolmogorov-Smirnov omnibus test and proposed alternatives to it. Independence between some classes of linear rank tests is established facilitating their joint application. Monte Carlo study confirms the asymptotic power properties of the linear rank tests but raises concerns about their application in more general and practically relevant circumstances. It also indicates that the new omnibus tests constitute viable alternatives with superior properties to the Kolmogorov-Smirnov test in certain circumstances.     

ON TESTING THE GOODNESS-OF-FIT OF NONLINEAR HETEROSCEDASTIC REGRESSION MODELS

For testing the adequacy of a parametric model in regression, various test statistics can be constructed on the basis of a marked empirical process of residuals. By using a discretized version of the decomposition of the corresponding Gaussian limiting process into its principal components, we obtain a test statistic with an asymptotic chi-squared distribution under the null hypothesis. We investigate the consistency of this test statistic and of the estimators needed to compute it. Numerical experiments indicate that the distributional approximations already work for small to moderate sample sizes and reveal that the test has good power properties against a variety of alternatives. The test has a simple implementation. We present an application to a real-data example for testing the adequacy of a possible heteroscedastic exponential model.     

SKEWNESS FOR PARAMETERS IN GENERALIZED LINEAR MODELS

In this paper we give an asymptotic formula of order n ^?1/2, where n is the sample size, for the skewness of the distribution of the maximum likelihood estimates of the linear parameters in generalized linear models. The formula is given in matrix notation and is very suitable for computer implementation. Several special cases are discussed. We also give asymptotic formulae for the skewness of the distribution of the maximum likelihood estimates of the dispersion and precision parameters.     

THE CONSTRUCTION OF EQUILEVERAGE DESIGNS FOR MULTIPLE LINEAR REGRESSION

The hat matrix is widely used as a diagnostic tool in linear regression because it contains the leverages which the independent variables exert on the fitted values. In some experiments, cases with high leverage may be avoided by judicious choice of design for the independent variables. A variety of methods for constructing equileverage designs for linear regression are discussed. Such designs remove one of the factors, namely large leverage points, which can lead to nonrobust estimators and tests. In addition, a method is given for combining equileverage designs to test for lack of fit of the linear model.     

ROBUST ESTIMATION IN PARAMETRIC TIME SERIES MODELS UNDER LONG- AND SHORT-RANGE-DEPENDENT STRUCTURES

This paper studies the asymptotic behaviour of an M-estimator of regression parameters in the linear model when the design variables are either stationary short-range dependent (SRD), α-mixing or long-range dependent (LRD), and the errors are LRD. The weak consistency and the asymptotic distributions of the M-estimator are established. We present some simulated examples to illustrate the efficiency of the proposed M-estimation method.     

A MORE GENERAL CRITERION FOR SUBSET SELECTION IN MULTIPLE LINEAR REGRESSION

ABSTRACT

In this article, we propose a more general criterion called S_p -criterion, for subset selection in the multiple linear regression Model. Many subset selection methods are based on the Least Squares (LS) estimator of β, but whenever the data contain an influential observation or the distribution of the error variable deviates from normality, the LS estimator performs ‘poorly’ and hence a method based on this estimator (for example, Mallows’ C_p -criterion) tends to select a ‘wrong’ subset. The proposed method overcomes this drawback and its main feature is that it can be used with any type of estimator (either the LS estimator or any robust estimator) of β without any need for modification of the proposed criterion. Moreover, this technique is operationally simple to implement as compared to other existing criteria. The method is illustrated with examples.     

A NOTE ON VARIANCE ESTIMATION FOR THE GENERALIZED REGRESSION PREDICTOR

The generalized regression (GREG) predictor is used for estimating a finite population total when the study variable is well‐related to the auxiliary variable. In 1997, Chaudhuri & Roy provided an optimal estimator for the variance of the GREG predictor within a class of non‐homogeneous quadratic estimators (H) under a certain superpopulation model M. They also found an inequality concerning the expected variances of the estimators of the variance of the GREG predictor belonging to the class H under the model M. This paper shows that the derivation of the optimal estimator and relevant inequality, presented by Chaudhuri & Roy, are incorrect.     

SHRINKAGE, PRETEST AND ABSOLUTE PENALTY ESTIMATORS IN PARTIALLY LINEAR MODELS

We consider a partially linear model in which the vector of coefficients β in the linear part can be partitioned as ( β ₁, β ₂) , where β ₁ is the coefficient vector for main effects (e.g. treatment effect, genetic effects) and β ₂ is a vector for ‘nuisance’ effects (e.g. age, laboratory). In this situation, inference about β ₁ may benefit from moving the least squares estimate for the full model in the direction of the least squares estimate without the nuisance variables (Steinian shrinkage), or from dropping the nuisance variables if there is evidence that they do not provide useful information (pretesting). We investigate the asymptotic properties of Stein‐type and pretest semiparametric estimators under quadratic loss and show that, under general conditions, a Stein‐type semiparametric estimator improves on the full model conventional semiparametric least squares estimator. The relative performance of the estimators is examined using asymptotic analysis of quadratic risk functions and it is found that the Stein‐type estimator outperforms the full model estimator uniformly. By contrast, the pretest estimator dominates the least squares estimator only in a small part of the parameter space, which is consistent with the theory. We also consider an absolute penalty‐type estimator for partially linear models and give a Monte Carlo simulation comparison of shrinkage, pretest and the absolute penalty‐type estimators. The comparison shows that the shrinkage method performs better than the absolute penalty‐type estimation method when the dimension of the β ₂ parameter space is large.     

GENERALIZED INVERSES USED IN RECURSIVE ESTIMATION OF THE GENERAL LINEAR MODEL1

Recursive methods in regression have proved useful in providing diagnostic tools for checking the model as well as checking the stability of the model over time. Such methods are now extended to deal with the problems of singularity that arise when one variable is completely confounded with previously fitted variables up to a particular time point. The problem is solved by setting it in the framework of the general linear model with dependent errors.     

GOODNESS-OF-FIT AND DISCORDANCY TESTS FOR SAMPLES FROM THE WATSON DISTRIBUTION ON THE SPHERE

The only parametric model in current use for axial data from a rotationally symmetric bipolar or girdle distribution on the sphere is the Watson distribution. This paper develops methods for evaluating the model as a fit to data using graphical and formal goodness-of-fit tests, and tests of discordancy.