A simple test for comparing regression curves versus one-sided alternatives

In this article we present a simple procedure to test for the null hypothesis of equality of two regression curves versus one-sided alternatives in a general nonparametric and heteroscedastic setup. The test is based on the comparison of the sample averages of the estimated residuals in each regression model under the null hypothesis. The test statistic has asymptotic normal distribution and can detect any local alternative of rate n^-1/2. Some simulations and an application to a data set are included.     

Diagnosis of Multivariate Control Chart Signal Based on Dummy Variable Regression Technique

Abstract

It is common to monitor several correlated quality characteristics using the Hotelling's T ² statistic. However, T ² confounds the location shift with scale shift and consequently it is often difficult to determine the factors responsible for out of control signal in terms of the process mean vector and/or process covariance matrix. In this paper, we propose a diagnostic procedure called ‘D-technique’ to detect the nature of shift. For this purpose, two sets of regression equations, each consisting of regression of a variable on the remaining variables, are used to characterize the ‘structure’ of the ‘in control’ process and that of ‘current’ process. To determine the sources responsible for an out of control state, it is shown that it is enough to compare these two structures using the dummy variable multiple regression equation. The proposed method is operationally simpler and computationally advantageous over existing diagnostic tools. The technique is illustrated with various examples.     

Another Cautionary Note about R 2: Its Use in Weighted Least-Squares Regression Analysis

A recent article in this journal presented a variety of expressions for the coefficient of determination (R ²) and demonstrated that these expressions were generally not equivalent. The article discussed potential pitfalls in interpreting the R ² statistic in ordinary least-squares regression analysis. The current article extends this discussion to the case in which regression models are fit by weighted least squares and points out an additional pitfall that awaits the unwary data analyst. We show that unthinking reliance on the R ² statistic can lead to an overly optimistic interpretation of the proportion of variance accounted for in the regression. We propose a modification of the estimator and demonstrate its utility by example.     

Comparison of Goodness-of-Fit Measures in Probit Regression Model

This article examines several goodness-of-fit measures in the binary probit regression model. Existing pseudo-R ² measures are reviewed, two modified and one new pseudo-R ² measure are proposed. For the probit regression model, empirical comparisons are made for different goodness-of-fit measures with the squared sample correlation coefficient of the observed response and the predicted probabilities. As an illustration, the goodness-of-fit measures are applied to a “paid labor force” data set.     

Bootstrap-Based T 2 Multivariate Control Charts

Control charts have been used effectively for years to monitor processes and detect abnormal behaviors. However, most control charts require a specific distribution to establish their control limits. The bootstrap method is a nonparametric technique that does not rely on the assumption of a parametric distribution of the observed data. Although the bootstrap technique has been used to develop univariate control charts to monitor a single process, no effort has been made to integrate the effectiveness of the bootstrap technique with multivariate control charts. In the present study, we propose a bootstrap-based multivariate T ² control chart that can efficiently monitor a process when the distribution of observed data is nonnormal or unknown. A simulation study was conducted to evaluate the performance of the proposed control chart and compare it with a traditional Hotelling's T ² control chart and the kernel density estimation (KDE)-based T ² control chart. The results showed that the proposed chart performed better than the traditional T ² control chart and performed comparably with the KDE-based T ² control chart. Furthermore, we present a case study to demonstrate the applicability of the proposed control chart to real situations.     

A Coefficient of Determination for Generalized Linear Models

The coefficient of determination, a.k.a. R², is well-defined in linear regression models, and measures the proportion of variation in the dependent variable explained by the predictors included in the model. To extend it for generalized linear models, we use the variance function to define the total variation of the dependent variable, as well as the remaining variation of the dependent variable after modeling the predictive effects of the independent variables. Unlike other definitions that demand complete specification of the likelihood function, our definition of R² only needs to know the mean and variance functions, so applicable to more general quasi-models. It is consistent with the classical measure of uncertainty using variance, and reduces to the classical definition of the coefficient of determination when linear regression models are considered.     

The Prediction Sum of Squares as a General Measure for Regression Diagnostics

Statistics that usually accompany the regression model do not provide insight into the quality of the data or the potential influence of the individual observations on the estimates. In this study, the Q² statistic is used as a criterion for detecting influential observations or outliers. The statistic is derived from the jackknifed residuals, the squared sum of which is generally known as the prediction sum of squares or PRESS. This article compares R ² with Q² and suggests that the latter be used as part of the data-quality check. It is shown, for two separate data sets obtained from regional cost of living and U.S. food industry studies, that in the presence of outliers the Q² statistic can be negative, because it is sensitive to the choice of regressors and the inclusion of influential observations. Once the outliers are dropped from the sample, the discrepancy between Q² and R ² values is negligible.     

Non parametric regression estimations over Lp risk based on biased data

Using a wavelet basis, Chesneau and Shirazi study the estimation of one-dimensional regression functions in a biased non parametric model over L² risk (see Chesneau, C and Shirazi, E. Non parametric wavelet regression based on biased data, Communication in Statistics – Theory and Methods, 43: 2642–2658, 2014). This article considers d-dimensional regression function estimation over L^p?(1 ? p < ∞) risk. It turns out that our results reduce to the corresponding theorems of Chesneau and Shirazi’s theorems, when d = 1 and p = 2.     

O(N 2)-Operation Approximation of Covariance Matrix Inverse in Gaussian Process Regression Based on Quasi-Newton BFGS Method

Gaussian process (GP) is a Bayesian nonparametric regression model, showing good performance in various applications. However, during its model-tuning procedure, the GP implementation suffers from numerous covariance-matrix inversions of expensive O(N³) operations, where N is the matrix dimension. In this article, we propose using the quasi-Newton BFGS O(N²)-operation formula to approximate/replace recursively the inverse of covariance matrix at every iteration. The implementation accuracy is guaranteed carefully by a matrix-trace criterion and by the restarts technique to generate good initial guesses. A number of numerical tests are then performed based on the sinusoidal regression example and the Wiener–Hammerstein identification example. It is shown that by using the proposed implementation, more than 80% O(N³) operations could be eliminated, and a typical speedup of 5–9 could be achieved as compared to the standard maximum-likelihood-estimation (MLE) implementation commonly used in Gaussian process regression.     

A heteroscedastic structural errors-in-variables model with equation error

It is not uncommon with astrophysical and epidemiological data sets that the variances of the observations are accessible from an analytical treatment of the data collection process. Moreover, in a regression model, heteroscedastic measurement errors and equation errors are common situations when modelling such data. This article deals with the limiting distribution of the maximum-likelihood and method-of-moments estimators for the line parameters of the regression model. We use the delta method to achieve it, making it possible to build joint confidence regions and hypothesis testing. This technique produces closed expressions for the asymptotic covariance matrix of those estimators. In the moment approach we do not assign any distribution for the unobservable covariate while with the maximum-likelihood approach, we assume a normal distribution. We also conduct simulation studies of rejection rates for Wald-type statistics in order to verify the test size and power. Practical applications are reported for a data set produced by the Chandra observatory and also from the WHO MONICA Project on cardiovascular disease.     

An Asymptotic Characterization of Finite Degree U-statistics With Sample Size-Dependent Kernels: Applications to Nonparametric Estimators and Test Statistics

We provide a simple result on the H-decomposition of a U-statistics that allows for easy determination of its magnitude when the statistic’s kernel depends on the sample size n. The result provides a direct and convenient method to characterize the asymptotic magnitude of semiparametric and nonparametric estimators or test statistics involving high dimensional sums. We illustrate the use of our result in previously studied estimators/test statistics and in a novel nonparametric R² test for overall significance of a nonparametric regression model.     

Shrinkage and LASSO strategies in high-dimensional heteroscedastic models

ABSTRACT

In this paper, we consider the estimation problem of the parameter vector in the linear regression model with heteroscedastic errors. First, under heteroscedastic errors, we study the performance of shrinkage-type estimators and their performance as compared to theunrestricted and restricted least squares estimators. In order to accommodate the heteroscedastic structure, we generalize an identity which is useful in deriving the risk function. Thanks to the established identity, we prove that shrinkage estimators dominate the unrestricted estimator. Finally, we explore the performance of high-dimensional heteroscedastic regression estimator as compared to classical LASSO and shrinkage estimators.     

Economic and economic statistical design of T 2 control chart with two adaptive sample sizes

The Hotelling's T ² control chart, a direct analogue of the univariate Shewhart X¯ chart, is perhaps the most commonly used tool in industry for simultaneous monitoring of several quality characteristics. Recent studies have shown that using variable sampling size (VSS) schemes results in charts with more statistical power when detecting small to moderate shifts in the process mean vector. In this paper, we build a cost model of a VSS T ² control chart for the economic and economic statistical design using the general model of Lorenzen and Vance [The economic design of control charts: A unified approach, Technometrics 28 (1986), pp. 3–11]. We optimize this model using a genetic algorithm approach. We also study the effects of the costs and operating parameters on the VSS T ² parameters, and show, through an example, the advantage of economic design over statistical design for VSS T ² charts, and measure the economic advantage of VSS sampling versus fixed sample size sampling.     

Generalized Analysis-of-variance-type Test for the Single-index Quantile Model

In this paper, we are concerned with a test for the index parameter and index function in the single-index model. Based on the estimates obtained by the quantile regression, we extend the generalized analysis-of-variance-type test to the single-index model. We investigate the asymptotic behavior of the proposed test and demonstrate that its limiting null distribution follows an asymptotically χ²-distribution. The simulation studies and real data applications are conducted to illustrate the finite sample performance of the proposed methods.     

Prior Density Selection as a Particular Case of Bayesian Model Selection: A Predictive Approach

A Bayesian model consists of two elements: a sampling model and a prior density. The problem of selecting a prior density is nothing but the problem of selecting a Bayesian model where the sampling model is fixed. A predictive approach is used through a decision problem where the loss function is the squared L ² distance between the sampling density and the posterior predictive density, because the aim of the method is to choose the prior that provides a posterior predictive density as good as possible. An algorithm is developed for solving the problem; this algorithm is based on Lavine's linearization technique.     

Optimal Designs for Random Coefficient Regression Models with Heteroscedastic Errors

The purpose of this article is to present the optimal designs based on D-, G-, A-, I-, and D _β-optimality criteria for random coefficient regression (RCR) models with heteroscedastic errors. A sufficient condition for the heteroscedastic structure is given to make sure that the search of optimal designs can be confined at extreme settings of the design region when the criteria satisfy the assumption of the real valued monotone design criteria. Analytical solutions of D-, G-, A-, I-, and D _β-optimal designs for the RCR models are derived. Two examples are presented for random slope models with specific heteroscedastic errors.     

Estimated Generalized Least Squares for a Heteroscedastic Regression Model

This paper considers the general linear regression model yc = X₁β+u_t under the heteroscedastic structure E(u_t) = 0, E(u²) =σ²- (X_tβ)², E(u_t u_s) = 0, tæs, t, s= 1, T. It is shown that any estimated GLS estimator for β is asymptotically equivalent to the GLS estimator under some regularity conditions. A three-step GLS estimator, which calls upon the assumption E(u_t²) =s?²(X,β)² for the estimation of the disturbance covariance matrix, is considered.     

Comparison of Nonparametric Regression Curves by Spline Smoothing

In this article, procedures are proposed to test the hypothesis of equality of two or more regression functions. Tests are proposed by p-values, first under homoscedastic regression model, which are derived using fiducial method based on cubic spline interpolation. Then, we construct a test in the heteroscedastic case based on Fisher's method of combining independent tests. We study the behaviors of the tests by simulation experiments, in which comparisons with other tests are also given. The proposed tests have good performances. Finally, an application to a data set are given to illustrate the usefulness of the proposed test in practice.     

Optimal design for nonlinear regression models

Two results for D _θ-optimal designs for nonlinear regression models are shown to follow directly from approximate design theory. The first result considered is one concerning the replication of exact designs with minimum support, first established by Atkinson and Hunter and by M.J. Box in 1968, while the second pertains to a heteroscedastic model introduced by Velilla and Llosa in 1992. An illustrative example is provided.