Estimation and hypothesis testing in multivariate linear regression models under non normality

This paper discusses the problem of statistical inference in multivariate linear regression models when the errors involved are non normally distributed. We consider multivariate t-distribution, a fat-tailed distribution, for the errors as alternative to normal distribution. Such non normality is commonly observed in working with many data sets, e.g., financial data that are usually having excess kurtosis. This distribution has a number of applications in many other areas of research as well. We use modified maximum likelihood estimation method that provides the estimator, called modified maximum likelihood estimator (MMLE), in closed form. These estimators are shown to be unbiased, efficient, and robust as compared to the widely used least square estimators (LSEs). Also, the tests based upon MMLEs are found to be more powerful than the similar tests based upon LSEs.     

Median regression for SUR models with the same explanatory variables in each equation

In this paper we introduce an interesting feature of the generalized least absolute deviations method for seemingly unrelated regression equations (SURE) models. Contrary to the collapse of generalized leasts-quares parameter estimations of SURE models to the ordinary least-squares estimations of the individual equations when the same regressors are common between all equations, the estimations of the proposed methodology are not identical to the least absolute deviations estimations of the individual equations. This is important since contrary to the least-squares methods, one can take advantage of efficiency gain due to cross-equation correlations even if the system includes the same regressors in each equation.     

Exact testing in multivariate regression

An F statistic due to Rao (1951,1973) tests uniform mixed linear restrictions in the multivariateregression model. In combination with a generalization of the Bera-Evans-Savin exact functional relationship between the W, LR, and LM statistics, Rao's F serves to unify a number of exact test procedures commonly applied in disparate empirical literatures. Examples in demand analysis and asset pricing are provided. The availability of exact tests of restrictions in certain nonlinear models when the model is linear under the null, originally explored by Milliken-Graybill (1970), is extended to multivariate regression. Generalized RESET, J-, and Hausman-Wu tests are resented. As an extension of Dufour (1989), bounds tests exist for nonlinear and inequality restrictions. Applications include conservative bound tests for symmetry or negativity of the substitution matrix in demand systems.     

Computing projection depth and its associated estimators

To facilitate the application of projection depth, an exact algorithm is proposed from the view of cutting a convex polytope with hyperplanes. Based on this algorithm, one can obtain a finite number of optimal direction vectors, which are x-free and therefore enable us (Liu et al., Preprint, 2011) to compute the projection depth and most of its associated estimators of dimension p≥2, including Stahel-Donoho location and scatter estimators, projection trimmed mean, projection depth contours and median, etc. Both real and simulated examples are also provided to illustrate the performance of the proposed algorithm.     

Local influence in multivariate regression

The method of local influence is generalized to the multivariate regression. The scheme of perturbations adopted in multivariate regression is similar in spirit to the perturbation of case-weights in univariate regression case. The method developed here is useful for identifying influential observations in multivariate regression as an exploratory or confirmatory data analysis. An illustrative example is given for the effectiveness of the local influence approach in multivariate regression.     

Negative binomial and mixed Poisson regression

A number of methods have been proposed for dealing with extra-Poisson variation when doing regression analysis of count data. This paper studies negative-binomial regression models and examines efficiency and robustness properties of inference procedures based on them. The methods are compared with quasilikelihood methods.     

Robust symmetric distribution location estimation and regression

A procedure for estimating the location parameter of an unknown symmetric distribution is developed for application to samples from very light-tailed through very heavy-tailed distributions. This procedure has an easy extension to a technique for estimating the coefficients in a linear regression model whose error distribution is symmetric with arbitrary tail weights. The regression procedure is, in turn, extended to make it applicable to situations where the error distribution is either symmetric or skewed. The potentials of the procedures for robust location parameter and regression coefficient estimation are demonstrated by simulation studies.     

An approach to evaluating sensitivity in Bayesian regression analyses

This paper presents a method for assessing the sensitivity of predictions in Bayesian regression analyses. In parametric Bayesian analyses there is a family s₀ of regression functions, parametrized by a finite-dimensional vector B. The family s₀ is a subset of R, the set of all possible regression functions. A prior π₀ on B induces a prior on R. This paper assesses sensitivity by computing bounds on the predictive probability of a fixed set K over a class of priors, Γ, induced by a class of families of regression functions, Γ_s, and a class of priors, Γ_π. This paper is divided into three parts which (1) define Γ, (2) describe an algorithm for finding accurate bounds on predictive probabilities over Γ and (3) illustrate the method with two examples. It is found that sensitivity to the family of regression functions can be much more important than sensitivity to π₀.     

S-estimator in partially linear regression models

In this paper, a robust estimator is proposed for partially linear regression models. We first estimate the nonparametric component using the penalized regression spline, then we construct an estimator of parametric component by using robust S-estimator. We propose an iterative algorithm to solve the proposed optimization problem, and introduce a robust generalized cross-validation to select the penalized parameter. Simulation studies and a real data analysis illustrate that the our proposed method is robust against outliers in the dataset or errors with heavy tails.     

Bias corrected estimates in multivariate student t regression models

The t distribution has proved to be a useful alternative to the normal distribution especially When robust estimation is desired. We consider the multivariate nonlinear Student-t regression model and show that the biased of the estimates of the regression coefficients can be computed from an auxiliary generalized linear regression. We give a formula for the biases of the estimates of the parameters in the scale matrix, which also can be computed by means of a generalized linear regression. We briefly discuss some important special cases and present simulation results which indicate that our bias-corrected estimates outperform the uncorrected ones in small samples.     

Techniques for nonlinear least squares and robust regression

Recently, the authors and others have made considerable progress in developing algorithms for solving certain large-residual nonlinear least-squares problems where Gauss-Newton (GN) methods can be expected to perform poorly. These methods take account of the term in the Hessian ignored by the GN methods and use quasi-Newton procedures to update this term explicitly. This paper reviews these new approaches and discusses how they can be modified to give good performance on nonlinear models with robust loss functions where lack of scale invariance causes several new problems to arise.     

Weighted quantile regression with nonelliptically structured covariates

Although quantile regression estimators are robust against low leverage observations with atypically large responses (Koenker & Bassett 1978), they can be seriously affected by a few points that deviate from the majority of the sample covariates. This problem can be alleviated by downweighting observations with high leverage. Unfortunately, when the covariates are not elliptically distributed, Mahalanobis distances may not be able to correctly identify atypical points. In this paper the authors discuss the use of weights based on a new leverage measure constructed using Rosenblatt's multivariate transformation which is able to reflect nonelliptical structures in the covariate space. The resulting weighted estimators are consistent, asymptotically normal, and have a bounded influence function. In addition, the authors also discuss a selection criterion for choosing the downweighting scheme. They illustrate their approach with child growth data from Finland. Finally, their simulation studies suggest that this methodology has good finite‐sample properties.     

Sow aspects of robustness in thr functional errors-in-variables regression model

For the functional errors-in-varinbles regression model, we define a class of robust regression estimators and study their properties     

Output-sensitive algorithms for Tukey depth and related problems

The Tukey depth (Proceedings of the International Congress of Mathematicians, vol. 2, pp. 523–531, 1975) of a point p with respect to a finite set S of points is the minimum number of elements of S contained in any closed halfspace that contains p. Algorithms for computing the Tukey depth of a point in various dimensions are considered. The running times of these algorithms depend on the value of the output, making them suited to situations, such as outlier removal, where the value of the output is typically small. This research was partly funded by the NSERC Canada.     

Some multivariate goodness-of-fit tests based on data depth

Based on data depth, three types of nonparametric goodness-of-fit tests for multivariate distribution are proposed in this paper. They are Pearson’s chi-square test, tests based on EDF and tests based on spacings, respectively. The Anderson–Darling (AD) test and the Greenwood test for bivariate normal distribution and uniform distribution are simulated. The results of simulation show that these two tests have low type I error rates and become more efficient with the increase in sample size. The AD-type test performs more powerfully than the Greenwood type test.     

Inference in multivariate linear regression models with elliptically distributed errors

In this study we investigate the problem of estimation and testing of hypotheses in multivariate linear regression models when the errors involved are assumed to be non-normally distributed. We consider the class of heavy-tailed distributions for this purpose. Although our method is applicable for any distribution in this class, we take the multivariate t-distribution for illustration. This distribution has applications in many fields of applied research such as Economics, Business, and Finance. For estimation purpose, we use the modified maximum likelihood method in order to get the so-called modified maximum likelihood estimates that are obtained in a closed form. We show that these estimates are substantially more efficient than least-square estimates. They are also found to be robust to reasonable deviations from the assumed distribution and also many data anomalies such as the presence of outliers in the sample, etc. We further provide test statistics for testing the relevant hypothesis regarding the regression coefficients.     

Rank regression in order restricted randomised designs

This paper uses order restricted randomised design (ORRD) to create a judgment ranked blocking factor based on available subjective information in a small set of experimental units (EUs). The design then performs a carefully designed randomisation scheme with certain restriction to assign the treatment levels to EUs across these subjective judgment blocks. Such an assignment induces positive dependence among within-set units, and the restrictions on the randomisation translate this positive dependence into a variance reduction technique. We provide a unified theory to analyse the data sets collected from an ORRD. The analysis uses the general framework of rank regression methodology in linear models, with some modification to our randomisation scheme, to estimate regression parameter and to test general linear hypotheses. It is shown that the estimators and test statistics have limiting normal and chi-square distributions regardless the quality of ranking information. A simulation study shows that the asymptotic results remain valid even for relatively small sample sizes. The proposed tests are applied to a clinical trial data set.