A predictive approach to the detection of additional information in a multivariate regression model

A predictive approach for the detection of additional information in a multivariate linear regression model is considered for the case of known and unknown error covariance matrices. The predictive density of future Observations on the additional variables under the model that they carry no information has been compared with the predictive density under the model that they do carry information. The Kullback-Leibler measure of divergence is used as a measure of comparison between the models.     

Statistical Inference in Orthogonal Regression for Three-Part Compositional Data Using a Linear Model with Type-II Constraints

Orthogonal regression is a proper tool to analyze relations between two variables when three-part compositional data, i.e., three-part observations carrying relative information (like proportions or percentages), are under examination. When linear statistical models with type-II constraints (constraints involving other parameters besides the ones of the unknown model) are employed for estimating the parameters of the regression line, approximate variances and covariances of the estimated line coefficients can be determined. Moreover, the additional assumption of normality enables to construct confidence domains and perform hypotheses testing. The theoretical results are applied to a real-world example.     

Robust mixture regression modeling using the least trimmed squares (LTS)-estimation method

Mixture regression models are used to investigate the relationship between variables that come from unknown latent groups and to model heterogenous datasets. In general, the error terms are assumed to be normal in the mixture regression model. However, the estimators under normality assumption are sensitive to the outliers. In this article, we introduce a robust mixture regression procedure based on the LTS-estimation method to combat with the outliers in the data. We give a simulation study and a real data example to illustrate the performance of the proposed estimators over the counterparts in terms of dealing with outliers.     

Multivariate regression models for discrete and continuous repeated measurements

A general class of multivariate regression models is considered for repeated measurements with discrete and continuous outcome variables. The proposed model is based on the seemingly unrelated regression model (Zellner, 1962) and an extension of the model of Park and Woolson(1992). The regression parameters of the model are consistently estimated using the two-stage least squares method. When the out come variables are multivariate normal, the two-stage estimator reduces to Zellner’s two-stage estimator. As a special case, we consider the marginal distribution described by Liang and Zeger (1986). Under this this distributional assumption, we show that the two-stage estimator has similar asymptotic properties and comparable small sample properties to Liang and Zeger's estimator. Since the proposed approach is based on the least squares method, however, any distributional assumption is not required for variables outcome variables. As a result, the proposed estimator is more robust to the marginal distribution of outcomes.     

Moment-based estimation of nonlinear regression models with boundary outcomes and endogeneity,with applications to nonnegative and fractional responses

In this article, we suggest simple moment-based estimators to deal with unobserved heterogeneity in a special class of nonlinear regression models that includes as main particular cases exponential models for nonnegative responses and logit and complementary loglog models for fractional responses. The proposed estimators: (i) treat observed and omitted covariates in a similar manner; (ii) can deal with boundary outcomes; (iii) accommodate endogenous explanatory variables without requiring knowledge on the reduced form model, although such information may be easily incorporated in the estimation process; (iv) do not require distributional assumptions on the unobservables, a conditional mean assumption being enough for consistent estimation of the structural parameters; and (v) under the additional assumption that the dependence between observables and unobservables is restricted to the conditional mean, produce consistent estimators of partial effects conditional only on observables.     

基于copula回归模型的损失预测

在非寿险损失预测的广义线性模型中,通常假设损失次数与损失强度相互独立,事实上二者之间往往存在一定的相依关系,可通过copula函数来刻画.在损失已经发生的条件下,假设损失次数服从零截断泊松分布,损失强度服从伽玛分布,可以建立损失次数与损失强度相互依赖的copula回归模型.把损失强度的分布扩展到逆高斯分布,并将此模型应用于一组车险保单数据进行实证研究.结果表明:该模型不但在损失预测方面优于独立假设下的广义线性模型,而且也优于损失强度服从伽马分布假设下的copula回归模型.     

Theoretical evaluation of prediction error in linear regression with a bivariate response variable containing missing data

Methods for linear regression with multivariate response variables are well described in statistical literature. In this study we conduct a theoretical evaluation of the expected squared prediction error in bivariate linear regression where one of the response variables contains missing data. We make the assumption of known covariance structure for the error terms. On this basis, we evaluate three well-known estimators: standard ordinary least squares, generalized least squares, and a James–Stein inspired estimator. Theoretical risk functions are worked out for all three estimators to evaluate under which circumstances it is advantageous to take the error covariance structure into account.     

Optimization of a product performance using mixture experiments including process variables

This article presents a case study of a chemical compound used in the delay mechanism to start a rocket engine. The compound consists in a three-component mixture. Besides the components proportions, two process variables are considered. The aim of the study is to investigate the mix components proportions and the levels of process variables that set the expected delay time as close as possible to the target value and, at the same time, minimize the width of prediction interval for the response. A linear regression model with normal responses was fitted. Through the model developed, the optimal components proportions and the levels of the process variables were determined. For the model selection, the use of the backward method with an information criterion proved to be efficient in the case under study.     

Statistical analysis of rock-burst events in underground mines and excavations to present reasonable data-driven predictors

Rock bursts are sudden and violent failures of surrounding rockmasses in underground mines and excavations. In this paper, a database consisting of 188 case histories was collected. Each case history contains some of the predictor variables ‘overburden thickness, maximum tangential stress, uniaxial compressive strength of rock, tensile strength of rock, stress ratio, brittleness ratio and elastic energy index’ and one of the four defined classes for the dependent variable ‘rock burst intensity’. A strategy, including ‘outlier detection and substitution, normality evaluation, deduction of distribution functions, estimation of mean and mean variation ranges, evaluation of mean-equality and distribution function-equality hypotheses, correlation analysis and factor analysis for in-review variables’, was implemented. The strategy led to conclude that some predictor variables with available case histories have no contributions for rock burst prediction. These inferences were in accordance with the results of regression techniques for qualitative dependent variables. Besides, many predictor variable arrangements were incompatible with factor analysis. In the case of compatible arrangements, the variation of the predictor variables cannot be considerably reflected. Application of nonlinear principal component analysis using auto-associative neural networks did not also lead to representative components. Therefore, the significant predictor variables can only be used to design new classifiers.     

A sequence of improved standard errors under heteroskedasticity of unknown form

The linear regression model is commonly used by practitioners to model the relationship between the variable of interest and a set of explanatory variables. The assumption that all error variances are the same (homoskedasticity) is oftentimes violated. Consistent regression standard errors can be computed using the heteroskedasticity-consistent covariance matrix estimator proposed by White (1980). Such standard errors, however, typically display nonnegligible systematic errors in finite samples, especially under leveraged data. Cribari-Neto et al. (2000) improved upon the White estimator by defining a sequence of bias-adjusted estimators with increasing accuracy. In this paper, we improve upon their main result by defining an alternative sequence of adjusted estimators whose biases vanish at a much faster rate. Hypothesis testing inference is also addressed. An empirical illustration is presented.     

Collinearity diagnostic applied in ridge estimation through the variance inflation factor

The variance inflation factor (VIF) is used to detect the presence of linear relationships between two or more independent variables (i.e. collinearity) in the multiple linear regression model. However, the traditionally used VIF definitions encounter some problems when extended to the case of the ridge estimation (RE). This paper presents an extension of the VIF in RE by providing two alternative VIF expressions that overcome these problems in the general case. Some characteristics of these expressions are also presented and compared with the traditional expression. The results are illustrated with an economic example in the case of three independent variables and with a Monte Carlo simulation for the general case.     

A note on predictive inference for multivariate elliptically contoured distributions

The prediction problem is considered for the multivariate regression model with an elliptically contoured error distribution. We show that the predictive distribution under elliptical errors assumption is the same as that obtained under normally distributed error in both the Bayesian approach using an im-proper prior and the classical approach. This gives inference robustness with respect to departures from the reference case of independent sampling from the normal distribution.     

Sharp Bounds on a Class of Copulas with known Values at Several Points

We derive best-possible bounds on the class of copulas with known values at several points, under the assumption that the points are either in “increasing order” or in “decreasing order”. These bounds may be used to establish best-possible bounds on Kendall's τ and Spearman's ρ, for such copulas. An important special case is when the values of a copula are known at several diagonal points. We also use our results to establish best-possible bounds on the distribution function of the sum of two random variables with known marginal distributions when the values of the joint distribution function are known at several points.     

Order statistics from overlapping samples: bivariate densities and regression properties

In this paper, we are interested in the joint distribution of two order statistics from overlapping samples. We give an explicit formula for the distribution of such a pair of random variables under the assumption that the parent distribution is absolutely continuous. We are also interested in the question to what extent conditional expectation of one of such order statistic given another determines the parent distribution. In particular, we provide a new characterization by linearity of regression of an order statistic from the extended sample given the one from the original sample, special case of which solves a problem explicitly stated in the literature. It appears that to describe the correct parent distribution it is convenient to use quantile density functions. In several other cases of regressions of order statistics we provide new results regarding uniqueness of the distribution in the sample.     

Minimum average deviance estimation for sufficient dimension reduction

Sufficient dimension reduction methods aim to reduce the dimensionality of predictors while preserving regression information relevant to the response. In this article, we develop Minimum Average Deviance Estimation (MADE) methodology for sufficient dimension reduction. The purpose of MADE is to generalize Minimum Average Variance Estimation (MAVE) beyond its assumption of additive errors to settings where the outcome follows an exponential family distribution. As in MAVE, a local likelihood approach is used to learn the form of the regression function from the data and the main parameter of interest is a dimension reduction subspace. To estimate this parameter within its natural space, we propose an iterative algorithm where one step utilizes optimization on the Stiefel manifold. MAVE is seen to be a special case of MADE in the case of Gaussian outcomes with a common variance. Several procedures are considered to estimate the reduced dimension and to predict the outcome for an arbitrary covariate value. Initial simulations and data analysis examples yield encouraging results and invite further exploration of the methodology.     

On Directional Dependence in a Regression Line

Abstract

In this paper, under the assumption of linear relationship between two variables we provide alternative simple method of proving the existing result connecting correlation coefficient with those of skewness of response and explanatory variables. Further we have given a relationship between correlation coefficient and coefficient of kurtosis of response and explanatory variables assuming the linear relationship between the two variables. Simple alternative way of deriving the formula, which helps in finding the direction dependence in linear regression, is discussed.     

Post-hoc analyses in multiple regression based on prediction error

A well-known problem in multiple regression is that it is possible to reject the hypothesis that all slope parameters are equal to zero, yet when applying the usual Student's T-test to the individual parameters, no significant differences are found. An alternative strategy is to estimate prediction error via the 0.632 bootstrap method for all models of interest and declare the parameters associated with the model that yields the smallest prediction error to differ from zero. The main results in this paper are that this latter strategy can have practical value versus Student's T; replacing squared error with absolute error can be beneficial in some situations and replacing least squares with an extension of the Theil-Sen estimator can substantially increase the probability of identifying the correct model under circumstances that are described.     

Nonparametric regression estimation at design poles and zeros

ABSTRACT In most treatments of nonparametric regression, it is assumed that the marginal density of the explanatory variables is strictly bounded away from zero and infinity. This note investigates the pointwise asymptotics for nonparametric regression when this assumption fails, that is, the marginal density of the explanatory variable has either an isolated zero or a pole at the point of interest.     

Bayesian nonparametric smoothers for regular processes

This report is about the analysis of stochastic processes of the form R = S + N, where S is a “smooth” functional and N is noise. The proposed methods derive from the assumption that the observed R-values and unobserved values of R, the assumed inferential objectives of the analysis, are linearly related through Taylor series expansions of observed about unobserved values. The expansion errors and all other priori unspecified quantities have a joint multivariate normal distribution which expresses the prior uncertainty about their values. The results include interpolators, predictors, and derivative estimates, with credibility-interval estimates automatically generated in each case. An analysis of an acid-rain wet-deposition time series is included to indicate the efficacy of the proposed method. It was this problem which led to the methodological developments reported in this paper.     

Linear regression analysis with inequality constraints on the regression parameters via empirical likelihood

An empirical likelihood ratio test is developed for testing for or against inequality constraints on regression parameters in linear regression analysis. The proposed approach imposes no parametric model nor identically distributing assumption on the random errors. The asymptotic distribution of the proposed test statistic under null hypothesis is shown to be of chi-bar-squared type. The asymptotic power under contiguous alternatives is also briefly discussed. Moreover, an adjusted empirical likelihood method is adopted to improve the small sample size behaviour of the proposed test. Several simulation studies are carried out to assess the finite sample performance of the proposed tests. The results reveal that the proposed tests could be valuable for improving inference efficiency. A real-life example is discussed to illustrate the theoretical results.