Critical values for testing for a single outlier in a nonlinear regression model

The maximum absolute studentized residual is commonly used for testing for a single outlier in a linear regression model. This test statistic, however, is seldom discussed in a nonlinear regression setting. We simulate the critical values for the tests under various nonlinear models. The associated critical values are found to be very close to one another. Moreover, they are very well approximated using the critical values obtained from F-distributions based on the Bonferroni equations in linear models. The results are promising even in samples of size 6.     

Outlier detection in linear models: a comparative study in simple linear regression

Five widely used test statistics for detecting outliers and influential observations were studied using Monte Carlo method . The test statistic based on Studentized residuals, with critical values given by Tietjen, Moore and Beckman (1973), appears to be the best procedure for detecting a single outlier in simple linear regression.     

A weighted bootstrap approximation for comparing the error distributions in nonparametric regression

Several procedures have been proposed for testing the equality of error distributions in two or more nonparametric regression models. Here we deal with methods based on comparing estimators of the cumulative distribution function (CDF) of the errors in each population to an estimator of the common CDF under the null hypothesis. The null distribution of the associated test statistics has been approximated by means of a smooth bootstrap (SB) estimator. This paper proposes to approximate their null distribution through a weighted bootstrap. It is shown that it produces a consistent estimator. The finite sample performance of this approximation is assessed by means of a simulation study, where it is also compared to the SB. This study reveals that, from a computational point of view, the proposed approximation is more efficient than the one provided by the SB.     

An asymptotic theory for semiparametric generalized least squares estimation in partially linear regression models

Consider a partially linear regression model with an unknown vector parameter β, an unknown functiong(·), and unknown heteroscedastic error variances. In this paper we develop an asymptotic semiparametric generalized least squares estimation theory under some weak moment conditions. These moment conditions are satisfied by many of the error distributions encountered in practice, and our theory does not require the number of replications to go to infinity.     

Detection of frailty in weibull lifetime data using outlier tests

Heterogeneity in lifetime data may be modelled by multiplying an individual's hazard by an unobserved frailty. We test for the presence of frailty of this kind in univariate and bivariate data with Weibull distributed lifetimes, using statistics based on the ordered Cox–Snell residuals from the null model of no frailty. The form of the statistics is suggested by outlier testing in the gamma distribution. We find through simulation that the sum of the k largest or k smallest order statistics, for suitably chosen k, provides a powerful test when the frailty distribution is assumed to be gamma or positive stable, respectively. We provide recommended values of k for sample sizes up to 100 and simple formulae for estimated critical values for tests at the 5% level.     

A comparison between bootstrap methods and generalized estimating equations for correlated outcomes in generalized linear models

We discuss and evaluate bootstrap algorithms for obtaining confidence intervals for parameters in Generalized Linear Models when the data are correlated. The methods are based on a stratified bootstrap and are suited to correlation occurring within “blocks” of data (e.g., individuals within a family, teeth within a mouth, etc.). Application of the intervals to data from a Dutch follow-up study on preterm infants shows the corroborative usefulness of the intervals, while the intervals are seen to be a powerful diagnostic in studying annual measles data. In a simulation study, we compare the coverage rates of the proposed intervals with existing methods (e.g., via Generalized Estimating Equations). In most cases, the bootstrap intervals are seen to perform better than current methods, and are produced in an automatic fashion, so that the user need not know (or have to guess) the dependence structure within a block.     

A new estimator combining the ridge regression and the restricted least squares methods of estimation

It is well-known in the literature on multicollinearity that one of the major consequences of multicollinearity on the ordinary least squares estimator is that the estimator produces large sampling variances, which in turn might inappropriately lead to exclusion of otherwise significant coefficients from the model. To circumvent this problem, two accepted estimation procedures which are often suggested are the restricted least squares method and the ridge regression method. While the former leads to a reduction in the sampling variance of the estimator, the later ensures a smaller mean square error value for the estimator. In this paper we have proposed a new estimator which is based on a criterion that combines the ideas underlying these two estimators. The standard properties of this new estimator have been studied in the paper. It has also been shown that this estimator is superior to both the restricted least squares as well as the ordinary ridge regression estimators by the criterion of mean sauare error of the estimator of the regression coefficients when the restrictions are indeed correct. The conditions for superiority of this estimator over the other two have also been derived for the situation when the restrictions are not correct.     

Comparison of bootstrap and generalized bootstrap methods for estimating high quantiles

The generalized bootstrap is a parametric bootstrap method in which the underlying distribution function is estimated by fitting a generalized lambda distribution to the observed data. In this study, the generalized bootstrap is compared with the traditional parametric and non-parametric bootstrap methods in estimating the quantiles at different levels, especially for high quantiles. The performances of the three methods are evaluated in terms of cover rate, average interval width and standard deviation of width of the 95% bootstrap confidence intervals. Simulation results showed that the generalized bootstrap has overall better performance than the non-parametric bootstrap in high quantile estimation.     

Small-sample behavior of weighted least squares in experimental design applications

In experimental design applications unbiased estimators s_i ² of the variances σ_i ² are possible. These estimators may be used in Weighted Least Squares (WLS) when estimating the parameters β. The resulting small-sample behavior is investigated in a Monte Carlo experiment. This experiment shows that an asymptotically valid covariance formula can be used if s_i ² is based on, say, at least 5 observations. The WLS estimator based on estimators s_i ² gives more accurate estimators of β, provided the σ_i ² differ by a factor, say, 10.     

A note on determining the number of outliers in an exponential sample by least squares procedure

In this paper, we suggest a least squares procedure for the determination of the number of upper outliers in an exponential sample by minimizing sample mean squared error. Moreover, the method can reduce the masking or “swamping” effects. In addition, we have also found that the least squares procedure is easy and simple to compute than test test procedure T _k suggested by Zhang (1998) for determining the number of upper outliers, since Zhang (1998) need to use the complicated null distribution of T _k . Moreover, we give three practical examples and a simulated example to illustrate the procedures. Further, simulation studies are given to show the advantages of the proposed method. Finally, the proposed least squares procedure can also determine the number of upper outliers in other continuous univariate distributions (for example, Pareto, Gumbel, Weibull, etc.). Received: May 10, 1999; revised version: June 5, 2000     

Improper use of the ordinary least squares estimator in the switching regression model

The paper considers the consequences of incorrectly using the ordinary least squares estimator, when the true but unknown model is a switching regression. Bias and mean square error express ons are given for slope and residual variance estimators. Except for in very specialized cases the estimators are biased. A numerical exarnple illustrates some of the issues raised and provides a conpelison between the ordinary least squares and maximum likelihood estimators.     

Analysis of least squares regression estimates in case of additional errors in the variables

Estimation of a regression function from independent and identical distributed data is considered. The L₂ error with integration with respect to the design measure is used as error criterion. Upper bounds on the L₂ error of least squares regression estimates are presented, which bound the error of the estimate in case that in the sample given to the estimate the values of the independent and the dependent variables are pertubated by some arbitrary procedure. The bounds are applied to analyze regression-based Monte Carlo methods for pricing American options in case of errors in modelling the price process.     

On necessary and sufficient condition for superiority of ridge estimator over least squares estimator

The necessary and sufficient condition is obtained such that ridge estimator is better than the least squares estimator relative to the matrix mean square error.     

Estimation in inverse Gaussian regression: comparison of asymptotic and bootstrap distributions

This paper presents a brief review of the asymptotic properties of the pseudo-maximum likelihood estimator in the regression model where the reciprocal of the mean of the dependent variable is considered to be a linear function of the regressor variables, and the observations on the dependent variable are assumed to have an inverse Gaussian distribution. The large sample theory for the pseudo-maximum likelihood estimator presented in Babu and Chaubey (1996) is highlighted and a simulation study is carried out to compare the approximation yielded by the bootstrap distribution to that of the asymptotic distribution.     

Deletion residuals in the detection of heterogeneity of variances in linear regression

The heterogeneity of error variance often causes a huge interpretive problem in linear regression analysis. Before taking any remedial measures we first need to detect this problem. A large number of diagnostic plots are now available in the literature for detecting heteroscedasticity of error variances. Among them the ‘residuals’ and ‘fits’ (R–F) plot is very popular and commonly used. In the R–F plot residuals are plotted against the fitted responses, where both these components are obtained using the ordinary least squares (OLS) method. It is now evident that the OLS fits and residuals suffer a huge setback in the presence of unusual observations and hence the R–F plot may not exhibit the real scenario. The deletion residuals based on a data set free from all unusual cases should estimate the true errors in a better way than the OLS residuals. In this paper we propose ‘deletion residuals’ and the ‘deletion fits’ (DR–DF) plot for the detection of the heterogeneity of error variances in a linear regression model to get a more convincing and reliable graphical display. Examples show that this plot locates unusual observations more clearly than the R–F plot. The advantage of using deletion residuals in the detection of heteroscedasticity of error variance is investigated through Monte Carlo simulations under a variety of situations.     

A new approach to statistical efficiency of weighted least squares fitting algorithms for reparameterization of nonlinear regression models

We study nonlinear least-squares problem that can be transformed to linear problem by change of variables. We derive a general formula for the statistically optimal weights and prove that the resulting linear regression gives an optimal estimate (which satisfies an analogue of the Rao-Cramer lower bound) in the limit of small noise.     

The total bootstrap median: a robust and efficient estimator of location and scale for small samples

We propose the total bootstrap median (TBM) as a robust and efficient estimator of location and scale for small samples. We demonstrate its performance by estimating the mean and variance of a variety of distributions. We also show that, if the underlying distribution is unknown and there is either no contamination or low to moderate contamination, the TBM provides a better estimate of the mean, in mean square terms, than the sample mean or the sample median. In addition, the TBM is a better estimator of the variance of the underlying distribution than the sample variance or the square of the bias-corrected median absolute deviation from the median estimator. We also show that the TBM is an explicit L-estimator, which allows a direct study of its properties.     

Numerical computation of exact moments of the least squares estimator in a first-order stationary autoregressive model

This paper examines the least squares estimator of the autoregressive coefficient in a first-order sta¬tionary autoregressive model. Exact lower-order moments are computed by numerical integrations. From the study of moment values, it is found that the exact distribution of the least squares estimator may be well approximated by a beta distribution.     

Examination and analysis of residuals: A test for detecting a monotonic relation between mean and variance in regression through the origin

This paper presents a simple and exact test for detecting a monotonic relation between the mean and variance in linear regression through the origin. This test resulted from utilizing uncorrelated Theil-residuals and the Goldfeld-Quandt peak test. A numerical example is provided to elucidate the method. A simulation experiment was performed to compare the empirical power of this test with those of the existing tests.