A Note on the Regression of Order Statistics

This paper examines the relationships between the mean residual life functions of parallel and k-out-of-n systems with the regression of order statistics. Using these relationships, the results and properties about the mean residual life function of those systems can be used for the regression of order statistics and vice versa. Finally, the paper proposes a definition for the mean residual life function of a k-out-of-n system when the number of failed components of the system is known.     

A Note on Multiple Regression for Single Index Model

Abstract

A simple method based on sliced inverse regression (SIR) is proposed to explore an effective dimension reduction (EDR) vector for the single index model. We avoid the principle component analysis step of the original SIR by using two sample mean vectors in two slices of the response variable and their difference vector. The theories become simpler, the method is equivalent to the multiple linear regression with dichotomized response, and the estimator can be expressed by a closed form, although the objective function might be an unknown nonlinear. It can be applied for the case when the number of covariates is large, and it requires no matrix operation or iterative calculation.     

A Note on the Role of Independence in t Statistics Constructed From Linear Statistics in Regression Models

Although the semilogarithmic chart is one of the most effective, adaptable, and dependable graphic forms, it has been subject to neglect and indifference by contemporary chart makers. At the present time, a large proportion of chart makers are simply not aware of the distinctive qualities and unusual potentials of the semilogarithmic chart. In addition to discussions of the characteristics, applications, interpretation, advantages, and disadvantages of the semilogarithmic chart, actual cases of ill-chosen graphic forms and substandard charts are analyzed and compared, which clearly demonstrates the superiority of the semilogarithmic chart for many purposes.     

A Note on Stagewise Regression

Suppose one estimates the coefficient β₂ in E[Y] = β₀ + β₁ X ₁ + β₂ X ₂ by stagewise regression. That is, first the model E[Y] ≌ β₀ + β₁ X ₁ is fit using simple linear regression followed by a simple linear regression of the residuals from this model on X ₂ to yield the estimator β₂. The ratio of the squared t statistic for the estimate b ₂ from multiple regression to the squared t statistic for β₂ is greater than or equal to 1.0 and is shown to be a convenient function of correlation coefficients among Y, X ₁, and X ₂. Examination of stagewise regression can provide useful insights when introducing concepts of multiple regression.     

A Note on Screening Regression Equations

Consider developing a regression model in a context where substantive theory is weak. To focus on an extreme case, suppose that in fact there is no relationship between the dependent variable and the explanatory variables. Even so, if there are many explanatory variables, the R ² will be high. If explanatory variables with small t statistics are dropped and the equation refitted, the R ² will stay high and the overall F will become highly significant. This is demonstrated by simulation and by asymptotic calculation.     

A Note on Functional Forms of Order Statistics

Functional forms of order statistics, as the solution of a system of equations, are studied. The case of the smaller and the larger of two random variables is discussed in detail. Some applications for normal and binomial distributions are presented.     

A Note on Nonunique MLE's and Sufficient Statistics

A simple classroom example is presented that illustrates nonunique maximum likelihood estimates and the fact that such estimates need not be functions of sufficient statistics.     

A Note on Importance Resampling for Multi-Dimensional Statistics

Johns (1988 Johns , M. V. (1988). Importance sampling for bootstrap confidence intervals. Journal of the American Statistical Association 83:709–714.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]), Davison (1988 Davison , A. C. ( 1988 ). Discussion of paper by D. V. Hinkley . Journal of the Royal Statistical Society Series B 50 : 356 – 357 . [Google Scholar]), and Do and Hall (1991 Do , K. A. , Hall , P. ( 1991 ). On importance sampling for the bootstrap . Biometrika 78 : 161 – 167 .[Crossref], [Web of Science ®] , [Google Scholar]) used importance sampling for calculating bootstrap distributions of one-dimensional statistics. Realizing that their methods can not be extended easily to multi-dimensional statistics, Fuh and Hu (2004 Fuh , C. D. , Hu , I. ( 2004 ). Efficient importance sampling for events of moderate deviations with applications . Biometrika 91 : 471 – 490 .[Crossref], [Web of Science ®] , [Google Scholar]) proposed an exponential tilting formula for statistics of multi-dimension, which is optimal in the sense that the asymptotic variance is minimized for estimating tail probabilities of asymptotically normal statistics. For one-dimensional statistics, Hu and Su (2008 Hu , J. , Su , Z. ( 2008 ). Adaptive resampling algorithms for estimating bootstrap distributions . Journal of Statistical Planning and Inference 138 ( 6 ): 1763 – 1777 .[Crossref], [Web of Science ®] , [Google Scholar]) proposed a multi-step variance minimization approach that can be viewed as a generalization of the two-step variance minimization approach proposed by Do and Hall (1991 Do , K. A. , Hall , P. ( 1991 ). On importance sampling for the bootstrap . Biometrika 78 : 161 – 167 .[Crossref], [Web of Science ®] , [Google Scholar]). In this article, we generalize the approach of Hu and Su (2008 Hu , J. , Su , Z. ( 2008 ). Adaptive resampling algorithms for estimating bootstrap distributions . Journal of Statistical Planning and Inference 138 ( 6 ): 1763 – 1777 .[Crossref], [Web of Science ®] , [Google Scholar]) to multi-dimensional statistics, which applies to general statistics and does not resort to asymptotics. Empirical results on a real survival data set show that the proposed algorithm provides significant computational efficiency gains.     

A Note on High-Dimensional Linear Regression With Interactions

The problem of interaction selection in high-dimensional data analysis has recently received much attention. This note aims to address and clarify several fundamental issues in interaction selection for linear regression models, especially when the input dimension p is much larger than the sample size n. We first discuss how to give a formal definition of “importance” for main and interaction effects. Then we focus on two-stage methods, which are computationally attractive for high-dimensional data analysis but thus far have been regarded as heuristic. We revisit the counterexample of Turlach and provide new insight to justify two-stage methods from the theoretical perspective. In the end, we suggest new strategies for interaction selection under the marginality principle and provide some simulation results.     

A Note on Asymmetry and Robustness in Linear Regression

We discuss the assumption of symmetry in robust linear regression. It is important to distinguish between the intercept term and the slope parameters. Ordinary robust regression requires no assumption of symmetry when interest lies in slope parameters; computer programs, confidence intervals, standard errors, and so forth do not change because the errors are asymmetric. The situation is radically different for bounded-influence estimators. With the exception of the Mallows class, these estimators are inconsistent for slope when the errors are asymmetric.     

A Note on the Best Linear Unbiased Estimation Based on Order Statistics

The appropriateness of replacing the term regression to the mean with regression to the mode is discussed.     

A Note on DRHR Preservation Property of Generalized Order Statistics

In this paper we study some problems associated with count data from a bivariate Poisson distribution, in which the marginal means are functions of explanatory variables. The estimates of these regression coefficients are developed under a variety of conditions: unrestricted linear model; parallelism of the regression planes; the coincidence of the regression planes. Tests are also developed for the validity of hypotheses involved in these models. The techniques are illustrated using simulated data.     

A Cautionary Note on the Analysis of Extreme Data with Cox Regression

This article analyzes a small censored data set to demonstrate the potential dangers of using statistical computing packages without understanding the details of statistical methods. The data, consisting of censored response times with heavy ties in one time point, were analyzed with a Cox regression model utilizing SAS PHREG and BMDP2L procedures. The p values, reported from both SAS PHREG and BMDP2L procedures, for testing the equality of two treatments vary considerably. This article illustrates that (1) the Breslow likelihood used in both BMDP2L and SAS PHREG procedures is too conservative and can have a critical effect on an extreme data set, (2) Wald's test in the SAS PHREG procedure may yield absurd results from most likelihood models, and (3) BMDP2L needs to include more than just the Breslow likelihood in future development.     

A Note on Fitting a Regression without an Intercept Term

By entering the data (y _i ,x _i ) followed by (–y _i ,–x _i ), one can obtain an intercept-free regression Y = Xβ + ε from a program package that normally uses an intercept term. There is no bias in the resultant regression coefficients, but a minor postanalysis adjustment is needed to the residual variance and standard errors.     

A Note on Bayesian Prediction from the Regression Model with Informative Priors

This paper considers the problem of undertaking a predictive analysis from a regression model when proper conjugate priors are used. It shows how the prior information can be incorporated as a virtual experiment by augmenting the data, and it derives expressions for both the prior and the posterior predictive densities. The results obtained are of considerable practical importance to practitioners of Bayesian regression methods.     

A Note on the Estimate of Treatment Effect from a Cox Regression Model When the Proportionality Assumption Is Violated

ABSTRACT

Cox proportional hazards regression model has been widely used to estimate the effect of a prognostic factor on a time-to-event outcome. In a survey of survival analyses in cancer journals, it was found that only 5% of studies using Cox proportional hazards model attempted to verify the underlying assumption. Usually an estimate of the treatment effect from fitting a Cox model was reported without validation of the proportionality assumption. It is not clear how such an estimate should be interpreted if the proportionality assumption is violated. In this article, we show that the estimate of treatment effect from a Cox regression model can be interpreted as a weighted average of the log-scaled hazard ratio over the duration of study. A hypothetic example is used to explain the weights.     

A Note on Comparing the Power of Test Statistics at Low Significance Levels

It is an obvious fact that the power of a test statistic is dependent upon the significance (alpha) level at which the test is performed. It is perhaps a less obvious fact that the relative performance of two statistics in terms of power is also a function of the alpha level. Through numerous personal discussions, we have noted that even some competent statisticians have the mistaken intuition that relative power comparisons at traditional levels such as α=0.05 will be roughly similar to relative power comparisons at very low levels, such as the level α=510^?8, which is commonly used in genome-wide association studies. In this brief note, we demonstrate that this notion is in fact quite wrong, especially with respect to comparing tests with differing degrees of freedom. In fact, at very low alpha levels the cost of additional degrees of freedom is often comparatively low. Thus we recommend that statisticians exercise caution when interpreting the results of power comparison studies which use alpha levels that will not be used in practice.     

A Note on a Commonly Used Ridge Regression Monte Carlo Design

Ridge estimators are usually examined through Monte Carlo simulations since their properties are difficult to obtain analytically. In this paper we argue that a simulation design commonly used in the literature will give biased results of Monte Carlo simulations in favor of ridge regression over ordinary least square estimators. Specifically, it is argued that the properties of ridge estimators that are functions of p distinct regressor eigenvalues should not be evaluated through Monte Carlo designs using only two distinct eigenvalues.     

A Note on Reverse Regression,Collinearity, and Employment Discrimination

This note refutes Whiteside and Narayanan's recent assertion that the conflict between direct and reverse regression for discrimination assessment (as pointed out by Conway and Roberts and others) is due to the collinearity in the data. Their mistake stems from misunderstanding of the elementary relationship between partial and multiple correlation coefficients.