Testing Monotonicity of Regression Functions – An Empirical Process Approach

We propose several new tests for monotonicity of regression functions based on different empirical processes of residuals and pseudo‐residuals. The residuals are obtained from an unconstrained kernel regression estimator whereas the pseudo‐residuals are obtained from an increasing regression estimator. Here, in particular, we consider a recently developed simple kernel‐based estimator for increasing regression functions based on increasing rearrangements of unconstrained non‐parametric estimators. The test statistics are estimated distance measures between the regression function and its increasing rearrangement. We discuss the asymptotic distributions, consistency and small sample performances of the tests.     

Partial residuals in cumulative regression models for ordinal data

We are concerned with cumulative regression models for an ordered categorical response variable Y. We propose two methods to build partial residuals from regression on a subset Z₁ of covariates Z., which take into regard the ordinal character of the response. The first method makes use of a multivariate GLM-representation of the model and produces residual measures for diagnostic purposes. The second uses a latent continuous variable model and yields new (adjusted) ordinal data Y^*. Both methods are illustrated by a data set from forestry.     

Linear and nonlinear causality tests in an LSTAR model: wavelet decomposition in a nonlinear environment

In this paper, we use simulated data to investigate the power of different causality tests in a two-dimensional vector autoregressive (VAR) model. The data are presented in a nonlinear environment that is modelled using a logistic smooth transition autoregressive function. We use both linear and nonlinear causality tests to investigate the unidirection causality relationship and compare the power of these tests. The linear test is the commonly used Granger causality F test. The nonlinear test is a non-parametric test based on Baek and Brock [A general test for non-linear Granger causality: Bivariate model. Tech. Rep., Iowa State University and University of Wisconsin, Madison, WI, 1992] and Hiemstra and Jones [Testing for linear and non-linear Granger causality in the stock price–volume relation, J. Finance 49(5) (1994), pp. 1639–1664]. When implementing the nonlinear test, we use separately the original data, the linear VAR filtered residuals, and the wavelet decomposed series based on wavelet multiresolution analysis. The VAR filtered residuals and the wavelet decomposition series are used to extract the nonlinear structure of the original data. The simulation results show that the non-parametric test based on the wavelet decomposition series (which is a model-free approach) has the highest power to explore the causality relationship in nonlinear models.     

Adjusted Pearson residuals in exponential family nonlinear models

In this paper, we give matrix formulae of order 𝒪(n ^?1), where n is the sample size, for the first two moments of Pearson residuals in exponential family nonlinear regression models [G.M. Cordeiro and G.A. Paula, Improved likelihood ratio statistic for exponential family nonlinear models, Biometrika 76 (1989), pp. 93–100.]. The formulae are applicable to many regression models in common use and generalize the results by Cordeiro [G.M. Cordeiro, On Pearson's residuals in generalized linear models, Statist. Prob. Lett. 66 (2004), pp. 213–219.] and Cook and Tsai [R.D. Cook and C.L. Tsai, Residuals in nonlinear regression, Biometrika 72(1985), pp. 23–29.]. We suggest adjusted Pearson residuals for these models having, to this order, the expected value zero and variance one. We show that the adjusted Pearson residuals can be easily computed by weighted linear regressions. Some numerical results from simulations indicate that the adjusted Pearson residuals are better approximated by the standard normal distribution than the Pearson residuals.     

Change-Point Detection in Two-Phase Regression with Inequality Constraints on the Regression Parameters

Two-phase regression models with inequality constraints on the regression coefficients and with a small number of measurements is considered. A new test based on the likelihood ratio in linear model with inequality constraints for the presence of a change-point is proposed. Numerical approximations to the powers against various alternatives are given and compared with the powers of the likelihood ratio test in the two-phase regression models without inequality constraints, the backwards CUSUM test, and the k-linear-r-ahead recursive residuals tests. Performance of related likelihood based estimators of the change-point is briefly studied in a Monte Carlo experiment.     

A finite mixture model for multivariate counts under endogenous selectivity

We describe a selection model for multivariate counts, where association between the primary outcomes and the endogenous selection source is modeled through outcome-specific latent effects which are assumed to be dependent across equations. Parametric specifications of this model already exist in the literature; in this paper, we show how model parameters can be estimated in a finite mixture context. This approach helps us to consider overdispersed counts, while allowing for multivariate association and endogeneity of the selection variable. In this context, attention is focused both on bias in estimated effects when exogeneity of selection (treatment) variable is assumed, as well as on consistent estimation of the association between the random effects in the primary and in the treatment effect models, when the latter is assumed endogeneous. The model behavior is investigated through a large scale simulation experiment. An empirical example on health care utilization data is provided.     

Improved Score Tests in Symmetric Linear Regression Models

The class of symmetric linear regression models has the normal linear regression model as a special case and includes several models that assume that the errors follow a symmetric distribution with longer-than-normal tails. An important member of this class is the t linear regression model, which is commonly used as an alternative to the usual normal regression model when the data contain extreme or outlying observations. In this article, we develop second-order asymptotic theory for score tests in this class of models. We obtain Bartlett-corrected score statistics for testing hypotheses on the regression and the dispersion parameters. The corrected statistics have chi-squared distributions with errors of order O(n ^?3/2), n being the sample size. The corrections represent an improvement over the corresponding original Rao's score statistics, which are chi-squared distributed up to errors of order O(n ^?1). Simulation results show that the corrected score tests perform much better than their uncorrected counterparts in samples of small or moderate size.     

Diagnostic checks for discrete data regression models using posterior predictive simulations

Model checking with discrete data regressions can be difficult because the usual methods such as residual plots have complicated reference distributions that depend on the parameters in the model. Posterior predictive checks have been proposed as a Bayesian way to average the results of goodness-of-fit tests in the presence of uncertainty in estimation of the parameters. We try this approach using a variety of discrepancy variables for generalized linear models fitted to a historical data set on behavioural learning. We then discuss the general applicability of our findings in the context of a recent applied example on which we have worked. We find that the following discrepancy variables work well, in the sense of being easy to interpret and sensitive to important model failures: structured displays of the entire data set, general discrepancy variables based on plots of binned or smoothed residuals versus predictors and specific discrepancy variables created on the basis of the particular concerns arising in an application. Plots of binned residuals are especially easy to use because their predictive distributions under the model are sufficiently simple that model checks can often be made implicitly. The following discrepancy variables did not work well: scatterplots of latent residuals defined from an underlying continuous model and quantile–quantile plots of these residuals.     

A Coefficient of Determination for Generalized Linear Models

The coefficient of determination, a.k.a. R², is well-defined in linear regression models, and measures the proportion of variation in the dependent variable explained by the predictors included in the model. To extend it for generalized linear models, we use the variance function to define the total variation of the dependent variable, as well as the remaining variation of the dependent variable after modeling the predictive effects of the independent variables. Unlike other definitions that demand complete specification of the likelihood function, our definition of R² only needs to know the mean and variance functions, so applicable to more general quasi-models. It is consistent with the classical measure of uncertainty using variance, and reduces to the classical definition of the coefficient of determination when linear regression models are considered.     

Nonnull asymptotic distributions of the LR,Wald, score and gradient statistics in generalized linear models with dispersion covariates

The class of generalized linear models with dispersion covariates, which allows us to jointly model the mean and dispersion parameters, is a natural extension to the classical generalized linear models. In this paper, we derive the asymptotic expansions under a sequence of Pitman alternatives (up to order n ^?1/2) for the nonnull distribution functions of the likelihood ratio, Wald, Rao score and gradient statistics in this class of models. The asymptotic distributions of these statistics are obtained for testing a subset of regression parameters and for testing a subset of dispersion parameters. Based on these nonnull asymptotic expansions, the power of all four tests, which are equivalent to first order, are compared. Furthermore, we consider Monte Carlo simulations in order to compare the finite-sample performance of these tests in this class of models. We present two empirical applications to two real data sets for illustrative purposes.     

Variable selection and importance in presence of high collinearity: an application to the prediction of lean body mass from multi-frequency bioelectrical impedance

In prediction problems both response and covariates may have high correlation with a second group of influential regressors, that can be considered as background variables. An important challenge is to perform variable selection and importance assessment among the covariates in the presence of these variables. A clinical example is the prediction of the lean body mass (response) from bioimpedance (covariates), where anthropometric measures play the role of background variables. We introduce a reduced dataset in which the variables are defined as the residuals with respect to the background, and perform variable selection and importance assessment both in linear and random forest models. Using a clinical dataset of multi-frequency bioimpedance, we show the effectiveness of this method to select the most relevant predictors of the lean body mass beyond anthropometry.     

TWO-STAGE SUPPORT ESTIMATION

This paper presents a two‐stage procedure for estimating the conditional support curve of a random variable X, given the information of a random vector X. Quantile estimation is followed by an extremal analysis on the residuals for problems which can be written as regression models. The technique is applied to data from the National Bureau of Economic Research and US Census Bureau's Center for Economic Studies which contain all four‐digit manufacturing industries. Simulation results show that in linear regression models the proposed estimation procedure is more efficient than the extreme linear regression quantile.     

Pseudo Panel Data Models With Cohort Interactive Effects

When genuine panel data samples are not available, repeated cross-sectional surveys can be used to form so-called pseudo panels. In this article, we investigate the properties of linear pseudo panel data estimators with fixed number of cohorts and time observations. We extend standard linear pseudo panel data setup to models with factor residuals by adapting the quasi-differencing approach developed for genuine panels. In a Monte Carlo study, we find that the proposed procedure has good finite sample properties in situations with endogeneity, cohort interactive effects, and near nonidentification. Finally, as an illustration the proposed method is applied to data from Ecuador to study labor supply elasticity. Supplementary materials for this article are available online.     

SIMULATIONS AND DERIVED APPROXIMATIONS FOR THE MEANS AND STANDARD DEVIATIONS OF THE CHARACTERISTIC ROOTS OF A WISHART MATRIX

Monte Carlo simulations were done to estimate the means and standard deviations of the characteristic roots of a Wishart matrix which can be used in computing tests of hypotheses concerning multiplicative terms in balanced linear-bilinear (multiplicative) models for an m × n table of data. In this report we extend the previous results (Mandel, 1971; Cornelius, 1980) to r ≤ 199, c ≤ 149 or r ≤ 149, c ≤ 199, where r and c are row and column degrees of freedom, respectively, of the two-way array of residuals (with total degrees of freedom rc) after fitting the linear effects. For 187 combinations of r and c at intervals over this domain, we used 5000 simulations to estimate expectations and standard deviations of the Wishart roots. Using weighted linear regression variable selection techniques, symmetric functions of r and c were obtained for approximating the simulated means and standard deviations. Use of these approximating functions will avoid the need for reference to tables for input to computer programs which require these values for tests of significance of sequentially fitted terms in the analyses of balanced linear-bilinear models.     

Partial linear single-index models with additive distortion measurement errors

We study partial linear single-index models (PLSiMs) when the response and the covariates in the parametric part are measured with additive distortion measurement errors. These distortions are modeled by unknown functions of a commonly observable confounding variable. We use the semiparametric profile least-squares method to estimate the parameters in the PLSiMs based on the residuals obtained from the distorted variables and confounding variable. We also employ the smoothly clipped absolute deviation penalty (SCAD) to select the relevant variables in the PLSiMs. We show that the resulting SCAD estimators are consistent and possess the oracle property. For the non parametric link function, we construct the simultaneous confidence bands and obtain the asymptotic distribution of the maximum absolute deviation between the estimated link function and the true link function. A simulation study is conducted to evaluate the performance of the proposed methods and a real dataset is analyzed for illustration.     

The extreme residuals in logistic regression models

Goodness-of-fit tests for logistic regression models using extreme residuals are considered. Approximations to the moments of the Pearson residuals are given for model fits made by maximum likelihood, minimum chi-square and weighted least squares and used to define modified residuals. Approximations to the critical values of the extreme statistics based on the ordinary and modified Pearson residuals are developed and assessed for the case of a single explanatory variable.     

Distributional aspects in latent variable models

For observable indicators with ordered categories one can assume underlying latent variables following certain marginal distributions. Transforming the latent variables changes its marginal distributions but not the observable qualitative indicators. The joint distribution of the latent variables can be constructed from the marginal distributions. There is a broad class of multivariate distributions for which the observable indicators are equivalent. By choosing the multivariate normal distribution from this class we can analyse a linear relationship between the transformed latent variables. This leads to latent structural equation models. Estimation of these latter models is therefore more general than the distributional assumption might initially suggest. Robustness of the estimation procedure is also discussed for deviations from this distribution family. Using ordinal business survey data of the German Ifo-institute we test the efficiency of firms' price expectations implied by the rational expectation hypothesis.     

Orthogonalized Residuals for Estimation of Marginally Specified Association Parameters in Multivariate Binary Data

Abstract. This paper focuses on marginal regression models for correlated binary responses when estimation of the association structure is of primary interest. A new estimating function approach based on orthogonalized residuals is proposed. A special case of the proposed procedure allows a new representation of the alternating logistic regressions method through marginal residuals. The connections between second‐order generalized estimating equations, alternating logistic regressions, pseudo‐likelihood and other methods are explored. Efficiency comparisons are presented, with emphasis on variable cluster size and on the role of higher‐order assumptions. The new method is illustrated with an analysis of data on impaired pulmonary function.     

Adjusted Pearson residuals in beta regression models

In this paper, matrix formulae of order n^?1, where n is the sample size, for the first two moments of Pearson residuals are obtained in beta regression models. Adjusted Pearson residuals are also obtained, having, to this order, expected value zero and variance one. Monte Carlo simulation results are presented illustrating the behaviour of both adjusted and unadjusted residuals.     

Semiparametric Estimators for Limited Dependent Variable (LDV) Models with Endogenous Regressors

This article reviews semiparametric estimators for limited dependent variable (LDV) models with endogenous regressors, where nonlinearity and nonseparability pose difficulties. We first introduce six main approaches in the linear equation system literature to handle endogenous regressors with linear projections: (i) ‘substitution’ replacing the endogenous regressors with their projected versions on the system exogenous regressors x, (ii) instrumental variable estimator (IVE) based on E{(error) × x} = 0, (iii) ‘model-projection’ turning the original model into a model in terms of only x-projected variables, (iv) ‘system reduced form (RF)’ finding RF parameters first and then the structural form (SF) parameters, (v) ‘artificial instrumental regressor’ using instruments as artificial regressors with zero coefficients, and (vi) ‘control function’ adding an extra term as a regressor to control for the endogeneity source. We then check if these approaches are applicable to LDV models using conditional mean/quantiles instead of linear projection. The six approaches provide a convenient forum on which semiparametric estimators in the literature can be categorized, although there are a few exceptions. The pros and cons of the approaches are discussed, and a small-scale simulation study is provided for some reviewed estimators.