Updating input–output matrices: assessing alternatives through simulation

A problem that frequently arises in economics, demography, statistics, transportation planning and stochastic modelling is how to adjust the entries of a matrix to fulfil row and column aggregation constraints. Biproportional methods in general and the so-called RAS algorithm in particular, have been used for decades to find solutions to this type of problem. Although alternatives exist, the RAS algorithm and its extensions are still the most popular. Apart from some interesting empirical and theoretical properties, tradition, simplicity and very low computational costs are among the reasons behind the great success of RAS. Nowadays computer hardware and software have made alternative procedures equally attractive. This work analyses, through simulation, the performance of RAS and some minimands when matrix coefficients vary following different schemes of change. Results suggest RAS algorithm as the best option when variations in coefficients are proportional to their size, while the method based on minimizing squared differences is seen to be the best alternative when the standard deviations of variations are either constant, variable, or an inverse function of matrix entries.     

Further identities for the Wishart distribution with applications in regression

Matrix analogues are given for a known scalar identity which relates certain expectations with respect to the Wishart distribution. (The scalar identity was independently derived by C. Stein and L. Haff.) The matrix analogues are more aptly called “matrix extensions.” They can be derived by using the scalar identity; nevertheless, they are seen (in quite elementary terms) to be more general than the latter. A method of doing multivariate calculations is developed from the identities, and several examples are worked in detail. We compute the first two moments of the regression coefficients and another matrix arising in regression analysis. Also, we give a new result for the matrix analogue of squared multiple correlation: the bias correction of Ezekiel (1930), a result often used in model building, is extended to the case of two or more dependent variables.     

Re-weighting estimation of the coefficients in the varying coefficient model with heteroscedastic errors

ABSTRACT

This article explores the estimation problem of the coefficients in the varying coefficient model with heteroscedastic errors. Specifically, we first present a method for estimating the variance function of the error term and the resulting estimator is proved to be consistent. Then, motivated by the generalized least-squares procedure for dealing with heteroscedasticity in the linear regression literature, we re-weight each squared residual term in the local linear smoother with the inverse of the corresponding estimated error variance to construct estimates of the coefficients. Simulation experiments and practical data analysis conducted demonstrate that the re-weighting approach can improve the accuracy of the coefficient estimates under a finite sample size, especially when the error heteroscedasticity is strong.     

Developing a restricted two-parameter Liu-type estimator: A comparison of restricted estimators in the binary logistic regression model

In the context of estimating regression coefficients of an ill-conditioned binary logistic regression model, we develop a new biased estimator having two parameters for estimating the regression vector parameter β when it is subjected to lie in the linear subspace restriction Hβ = h. The matrix mean squared error and mean squared error (MSE) functions of these newly defined estimators are derived. Moreover, a method to choose the two parameters is proposed. Then, the performance of the proposed estimator is compared to that of the restricted maximum likelihood estimator and some other existing estimators in the sense of MSE via a Monte Carlo simulation study. According to the simulation results, the performance of the estimators depends on the sample size, number of explanatory variables, and degree of correlation. The superiority region of our proposed estimator is identified based on the biasing parameters, numerically. It is concluded that the new estimator is superior to the others in most of the situations considered and it is recommended to the researchers.     

Properties of the mixed regression estimator when disturbances are not necessarily normal

Small-disturbance approximations for the bias vector and mean squared error matrix of the mixed regression estimator for the coefficients in a linear regression model are derived and efficiency with respect to least squares estimator is examined.     

On a class of shrinkage estimators of vector of regression coefficients the

This paper studies a class of shrinkage estimators of the vector of regression coefficients. The small disturbance approximations for the bias and the mean squared error matrix of the estimator are derived. In the sense of mean squared error, these estimators dominate the least squares estimator and the generalized Stein estimator developed by Hosmane (1988).     

Covariance-regularized regression and classification for high dimensional problems

Summary.  We propose covariance-regularized regression, a family of methods for prediction in high dimensional settings that uses a shrunken estimate of the inverse covariance matrix of the features to achieve superior prediction. An estimate of the inverse covariance matrix is obtained by maximizing the log-likelihood of the data, under a multivariate normal model, subject to a penalty; it is then used to estimate coefficients for the regression of the response onto the features. We show that ridge regression, the lasso and the elastic net are special cases of covariance-regularized regression, and we demonstrate that certain previously unexplored forms of covariance-regularized regression can outperform existing methods in a range of situations. The covariance-regularized regression framework is extended to generalized linear models and linear discriminant analysis, and is used to analyse gene expression data sets with multiple class and survival outcomes.     

MSE performance of the weighted average estimators consisting of shrinkage estimators

In this paper, we consider a regression model and propose estimators which are the weighted averages of two estimators among three estimators; the Stein-rule (SR), the minimum mean squared error (MMSE), and the adjusted minimum mean-squared error (AMMSE) estimators. It is shown that one of the proposed estimators has smaller mean-squared error (MSE) than the positive-part Stein-rule (PSR) estimator over a moderate region of parameter space when the number of the regression coefficients is small (i.e., 3), and its MSE performance is comparable to the PSR estimator even when the number of the regression coefficients is not so small.     

Adaptive Estimation of the Integral of Squared Regression Derivatives

A problem of estimating the integral of a squared regression function and of its squared derivatives has been addressed in a number of papers. For the case of a heteroscedastic model where smoothness of the underlying regression function, the design density, and the variance of errors are known, the asymptotically sharp minimax lower bound and a sharp estimator were found in Pastuchova & Khasminski (1989). However, there are apparently no results on the either rate optimal or sharp optimal adaptive, or data-driven, estimation when neither the degree of regression function smoothness nor design density, scale function and distribution of errors are known. After a brief review of main developments in non-parametric estimation of non-linear functionals, we suggest a simple adaptive estimator for the integral of a squared regression function and its derivatives and prove that it is sharp-optimal whenever the estimated derivative is sufficiently smooth.     

Model averaging procedure for varying-coefficient partially linear models with missing responses

This paper is concerned with model averaging procedure for varying-coefficient partially linear models with missing responses. The profile least-squares estimation process and inverse probability weighted method are employed to estimate regression coefficients of the partially restricted models, in which the propensity score is estimated by the covariate balancing propensity score method. The estimators of the linear parameters are shown to be asymptotically normal. Then we develop the focused information criterion, formulate the frequentist model averaging estimators and construct the corresponding confidence intervals. Some simulation studies are conducted to examine the finite sample performance of the proposed methods. We find that the covariate balancing propensity score improves the performance of the inverse probability weighted estimator. We also demonstrate the superiority of the proposed model averaging estimators over those of existing strategies in terms of mean squared error and coverage probability. Finally, our approach is further applied to a real data example.     

Some Statistical Methods in Assessing Linear Relational Agreement: An Empirical Comparative Study

Although multiple indices were introduced in the area of agreement measurements, the only documented index for linear relational agreement, which is for interval scale data, is the Pearson product-moment correlation coefficient. Despite its meaningfulness, the Pearson product-moment correlation coefficient does not convey the practical information such as what proportion of observations is within a certain boundary of the target value. To address this need, based on the inverse regression, we proposed the adjusted mean squared deviation (AMSD), adjusted coverage probability (ACP), and adjusted total deviation index (ATDI) for the measurement of the relational agreement. They can serve as reasonable and practically meaningful measurements for relational agreement. Real life data are considered to illustrate the performance of the methods.     

All Invariant Moments of the Wishart Distribution

Abstract.  In this paper, we compute moments of a Wishart matrix variate U of the form E ( Q ( U )) where Q ( u ) is a polynomial with respect to the entries of the symmetric matrix u , invariant in the sense that it depends only on the eigenvalues of the matrix u . This gives us in particular the expected value of any power of the Wishart matrix U or its inverse U ^{− 1}. For our proofs, we do not rely on traditional combinatorial methods but rather on the interplay between two bases of the space of invariant polynomials in U . This means that all moments can be obtained through the multiplication of three matrices with known entries. Practically, the moments are obtained by computer with an extremely simple Maple program.     

The weighted least square based estimators with censoring indicators missing at random

In this paper, we study linear regression analysis when some of the censoring indicators are missing at random. We define regression calibration estimate, imputation estimate and inverse probability weighted estimate for the regression coefficient vector based on the weighted least squared approach due to Stute (1993), and prove all the estimators are asymptotically normal. A simulation study was conducted to evaluate the finite properties of the proposed estimators, and a real data example is provided to illustrate our methods.     

EXACT FINITE-SAMPLE PROPERTIES OF A PRE-TEST ESTIMATOR IN RIDGE REGRESSION

This paper discusses a pre-test regression estimator which uses the least squares estimate when it is “large” and a ridge regression estimate for “small” regression coefficients, where the preliminary test is applied separately to each regression coefficient in turn to determine whether it is “large” or “small.” For orthogonal regressors, the exact finite-sample bias and mean squared error of the pre-test estimator are derived. The latter is less biased than a ridge estimator, and over much of the parameter space the pre-test estimator has smaller mean squared error than least squares. A ridge estimator is found to be inferior to the pre-test estimator in terms of mean squared error in many situations, and at worst the latter estimator is only slightly less efficient than the former at commonly used significance levels.     

On the sampling performance of an inequality pre-test estimator of the regression error variance under LINEX loss

We consider the estimation of the error variance of a linear regression model where prior information is available in the form of an (uncertain) inequality constraint on the coefficients. Previous studies on this and other related problems use the squared error loss in comparing estimator’s performance. Here, by adopting the asymmetric LINEX loss function, we derive and numerically evaluate the exact risks of the inequality constrained estimator and the inequality pre-test estimator which results after a preliminary test for an inequality constraint on the coefficients. The risks based on squared error loss are special cases of our results, and we draw appropriate comparisons.     

Geometric aspects of deletion diagnostics in multivariate regression

In multivariate regression, a graphical diagnostic method of detecting observations that are influential in estimating regression coefficients is introduced. It is based on the principal components and their variances obtained from the covariance matrix of the probability distribution for the change in the estimator of the matrix of unknown regression coefficients due to a single-case deletion. As a result, each deletion statistic obtained in a form of matrix is transformed into a two-dimensional quantity. Its univariate version is also introduced in a little different way. No distributional form is assumed. For illustration, we provide a numerical example in which the graphical method introduced here is seen to be effective in getting information about influential observations.     

Exact small sample properties of an operational variant of the minimum mean squared error estimator

In this paper, we show a sufficient condition for an operational variant of the minimum mean squared error estimator (simply, the minimum MSE estimator) to dominate the ordinary least squares (OLS) estimator. It is also shown numerically that the minimum MSE estimator dominates the OLS estimator if the number of regression coefficients is larger than or equal to three, even if the sufficient condition is not satisfied. When the number of regression coefficients is smaller than three, our numerical results show that the gain in MSE of using the minimum MSE estimator is larger than the loss.     

Aalen's linear model for doubly censored data

Double censoring often occurs in registry studies when left censoring is present in addition to right censoring. In this work, we examine estimation of Aalen's nonparametric regression coefficients based on doubly censored data. We propose two estimation techniques. The first type of estimators, including ordinary least squared (OLS) estimator and weighted least squared (WLS) estimators, are obtained using martingale arguments. The second type of estimator, the maximum likelihood estimator (MLE), is obtained via expectation-maximization (EM) algorithms that treat the survival times of left censored observations as missing. Asymptotic properties, including the uniform consistency and weak convergence, are established for the MLE. Simulation results demonstrate that the MLE is more efficient than the OLS and WLS estimators.     

Proportional hazards regression with interval-censored and left-truncated data

This paper considers the estimation of the regression coefficients in the Cox proportional hazards model with left-truncated and interval-censored data. Using the approaches of Pan [A multiple imputation approach to Cox regression with interval-censored data, Biometrics 56 (2000), pp. 199–203] and Heller [Proportional hazards regression with interval censored data using an inverse probability weight, Lifetime Data Anal. 17 (2011), pp. 373–385], we propose two estimates of the regression coefficients. The first estimate is based on a multiple imputation methodology. The second estimate uses an inverse probability weight to select event time pairs where the ordering is unambiguous. A simulation study is conducted to investigate the performance of the proposed estimators. The proposed methods are illustrated using the Centers for Disease Control and Prevention (CDC) acquired immunodeficiency syndrome (AIDS) Blood Transfusion Data.     

Parsimonious Estimation of the Covariance Matrix in Multinomial Probit Models

This article presents a Bayesian analysis of a multinomial probit model by building on previous work that specified priors on identified parameters. The main contribution of our article is to propose a prior on the covariance matrix of the latent utilities that permits elements of the inverse of the covariance matrix to be identically zero. This allows a parsimonious representation of the covariance matrix when such parsimony exists. The methodology is applied to both simulated and real data, and its ability to obtain more efficient estimators of the covariance matrix and regression coefficients is assessed using simulated data.