Comparison of algorithms for replacing missing data in discriminant analysis

We examined the impact of different methods for replacing missing data in discriminant analyses conducted on randomly generated samples from multivariate normal and non-normal distributions. The probabilities of correct classification were obtained for these discriminant analyses before and after randomly deleting data as well as after deleted data were replaced using: (1) variable means, (2) principal component projections, and (3) the EM algorithm. Populations compared were: (1) multivariate normal with covariance matrices ∑₁=∑₂, (2) multivariate normal with ∑₁≠∑₂ and (3) multivariate non-normal with ∑₁=∑₂. Differences in the probabilities of correct classification were most evident for populations with small Mahalanobis distances or high proportions of missing data. The three replacement methods performed similarly but all were better than non - replacement.     

A Necessary Power Divergence Type Family Tests of Multivariate Normality

In a recent article, Cardoso de Oliveira and Ferreira have proposed a multivariate extension of the univariate chi-squared normality test, using a known result for the distribution of quadratic forms in normal variables. In this article, we propose a family of power divergence type test statistics for testing the hypothesis of multinormality. The proposed family of test statistics includes as a particular case the test proposed by Cardoso de Oliveira and Ferreira. We assess the performance of the new family of test statistics by using Monte Carlo simulation. In this context, the type I error rates and the power of the tests are studied, for important family members. Moreover, the performance of significant members of the proposed test statistics are compared with the respective performance of a multivariate normality test, proposed recently by Batsidis and Zografos. Finally, two well-known data sets are used to illustrate the method developed in this article as well as the specialized test of multivariate normality proposed by Batsidis and Zografos.     

Test of Normality Against Generalized Exponential Power Alternatives

The family of symmetric generalized exponential power (GEP) densities offers a wide range of tail behaviors, which may be exponential, polynomial, and/or logarithmic. In this article, a test of normality based on Rao's score statistic and this family of GEP alternatives is proposed. This test is tailored to detect departures from normality in the tails of the distribution. The main interest of this approach is that it provides a test with a large family of symmetric alternatives having non-normal tails. In addition, the test's statistic consists of a combination of three quantities that can be interpreted as new measures of tail thickness. In a Monte-Carlo simulation study, the proposed test is shown to perform well in terms of power when compared to its competitors.     

Families of Multivariate Distributions Involving “Triangular” Transformations

Attention is initially focused on certain pseudo-normal distributions. These are multivariate distributions in which one coordinate variable has a normal distribution and the distribution of the remaining variables is determined by a specific triangular transformation model involving normally distributed components. A remarkably flexible family of models is obtainable in this fashion. Some examples are described. In addition, models involving non-normal component distributions are discussed together with their relationship with those models obtainable by means of the beta-generalized-Rosenblatt construction. Inferential questions regarding these models will be the subject of a separate report.     

On comparing several straight lines under heteroscedasticity and robustness with respect to departure from normality

This article considers the problem of testing slopes in k straight lines with'heterogeneous variances. The statistic Fβ is proposed and the null and non-null distributions of Fβ derived under normality assumption. The power function values are then approximated by Laguerre polynomial expansion for normal and non-normal universes. For the example given in Graybill ‘1976, p. 295’, it is shown that the Satterthwaite approximation provides a close approximation to the null and non-null distributions in all the cases; it is also shown that the Fβ test is quite robust with respect to departure from normality in the case of mixtures of two normals.     

A simulation comparison of estimators for a regression coefficient under differential non-response

We consider the estimation of a regression coefficient in a linear regression when observations are missing due to nonresponse. Response is assumed to be determined by a nonobservable variable which is linearly related to an observable variable. The values of the observable variable are assumed to be available for the whole sample but the variable is not includsd in the regression relationship of interest . Several alternative estimators have been proposed for this situation under various simplifying assumptions. A sampling theory approach provides three alternative estimatrs by considering the observatins as obtained from a sub-sample, selected on the basis of the fully observable variable , as formulated by Nathan and Holt (1980). Under an econometric approach, Heckman (1979) proposed a two-stage (probit and OLS) estimator which is consistent under specificconditions. A simulation comparison of the four estimators and the ordinary least squares estimator , under multivariate normality of all the variables involved, indicates that the econometric approach estimator is not robust to departures from the conditions underlying its derivation, while two of the other estimators exhibit a similar degree of stable performance over a wide range of conditions. Simulations for a non-normal distribution show that gains in performance can be obtained if observations on the independent variable are available for the whole population.     

Acceptance sampling plans by variables for a class of symmetric distributions

The estimation of percentage defectives using a normal sampling plan will not be appropriate when the assumption of normality is violated. In this paper, we propose a sampling plan based on a more general symmetric family of distributions with the parameters estimated using the modified maximum likelihood (MML) procedures introduced by Tiku and Suresh . This sampling plan works well for most of the symmetric non-normal distributions. Some numerical study has also been carried out to show the superiority of the proposed plan.     

Robust Two-Sample Statistics for Testing Equality of Means: a Simulation Study

When testing the equality of the means from two independent normally distributed populations given that the variances of the two populations are unknown but assumed equal, the classical two-sample t-test is recommended. If the underlying population distributions are normal with unequal and unknown variances, either Welch's t-statistic or Satterthwaite's Approximate F-test is suggested. However, Welch's procedure is non-robust under most non-normal distributions. There is a variable tolerance level around the strict assumptions of data independence, homogeneity of variances and normality of the distributions. Few textbooks offer alternatives when one or more of the underlying assumptions are not defensible.     

The Combination Test for Multivariate Normality

A basic graphical approach for checking normality is the Q - Q plot that compares sample quantiles against the population quantiles. In the univariate analysis, the probability plot correlation coefficient test for normality has been studied extensively. We consider testing the multivariate normality by using the correlation coefficient of the Q - Q plot. When multivariate normality holds, the sample squared distance should follow a chi-square distribution for large samples. The plot should resemble a straight line. A correlation coefficient test can be constructed by using the pairs of points in the probability plot. When the correlation coefficient test does not reject the null hypothesis, the sample data may come from a multivariate normal distribution or some other distributions. So, we use the following two steps to test multivariate normality. First, we check the multivariate normality by using the probability plot correction coefficient test. If the test does not reject the null hypothesis, then we test symmetry of the distribution and determine whether multivariate normality holds. This test procedure is called the combination test. The size and power of this test are studied, and it is found that the combination test, in general, is more powerful than other tests for multivariate normality.     

Nonparametric test for the homogeneity of the overall variability

In this paper, we propose a nonparametric test for homogeneity of overall variabilities for two multi-dimensional populations. Comparisons between the proposed nonparametric procedure and the asymptotic parametric procedure and a permutation test based on standardized generalized variances are made when the underlying populations are multivariate normal. We also study the performance of these test procedures when the underlying populations are non-normal. We observe that the nonparametric procedure and the permutation test based on standardized generalized variances are not as powerful as the asymptotic parametric test under normality. However, they are reliable and powerful tests for comparing overall variability under other multivariate distributions such as the multivariate Cauchy, the multivariate Pareto and the multivariate exponential distributions, even with small sample sizes. A Monte Carlo simulation study is used to evaluate the performance of the proposed procedures. An example from an educational study is used to illustrate the proposed nonparametric test.     

PARTIAL CORRELATION AND CONDITIONAL CORRELATION AS MEASURES OF CONDITIONAL INDEPENDENCE

This paper investigates the roles of partial correlation and conditional correlation as measures of the conditional independence of two random variables. It first establishes a sufficient condition for the coincidence of the partial correlation with the conditional correlation. The condition is satisfied not only for multivariate normal but also for elliptical, multivariate hypergeometric, multivariate negative hypergeometric, multinomial and Dirichlet distributions. Such families of distributions are characterized by a semigroup property as a parametric family of distributions. A necessary and sufficient condition for the coincidence of the partial covariance with the conditional covariance is also derived. However, a known family of multivariate distributions which satisfies this condition cannot be found, except for the multivariate normal. The paper also shows that conditional independence has no close ties with zero partial correlation except in the case of the multivariate normal distribution; it has rather close ties to the zero conditional correlation. It shows that the equivalence between zero conditional covariance and conditional independence for normal variables is retained by any monotone transformation of each variable. The results suggest that care must be taken when using such correlations as measures of conditional independence unless the joint distribution is known to be normal. Otherwise a new concept of conditional independence may need to be introduced in place of conditional independence through zero conditional correlation or other statistics.     

Multivariate extension of chi-squared univariate normality test

We propose a multivariate extension of the univariate chi-squared normality test. Using a known result for the distribution of quadratic forms in normal variables, we show that the proposed test statistic has an approximated chi-squared distribution under the null hypothesis of multivariate normality. As in the univariate case, the new test statistic is based on a comparison of observed and expected frequencies for specified events in sample space. In the univariate case, these events are the standard class intervals, but in the multivariate extension we propose these become hyper-ellipsoidal annuli in multivariate sample space. We assess the performance of the new test using Monte Carlo simulation. Keeping the type I error rate fixed, we show that the new test has power that compares favourably with other standard normality tests, though no uniformly most powerful test has been found. We recommend the new test due to its competitive advantages.     

Transforming linear functions to normality:optimal component powers

This paper considers the analysis of linear models where the response variable is a linear function of observable component variables. For example, scores on two or more psychometric measures (the component variables) might be weighted and summed to construct a single response variable in a psychological study. A linear model is then fit to the response variable. The question addressed in this paper is how to optimally transform the component variables so that the response is approximately normally distributed. The transformed component variables, themselves, need not be jointly normal. Two cases are considered; in both cases, the Box-Cox power family of transformations is employed. In Case I, the coefficients of the linear transformation are known constants. In Case II, the linear function is the first principal component based on the matrix of correlations among the transformed component variables. For each case, an algorithm is described for finding the transformation powers that minimize a generalized Anderson-Darling statistic. The proposed transformation procedure is compared to likelihood-based methods by means of simulation. The proposed method rarely performed worse than likelihood-based methods and for many data sets performed substantially better. As an illustration, the algorithm is applied to a problem from rural sociology and social psychology; namely scaling family residences along an urban-rural dimension.     

Nonnormality of Linear Combinations of Normal Random Variables

Hamedani and Tata (1975) showed that the bivariate normal distribution is determined uniquely by any countably infinite collection of distinct linear combinations of the variables and by no finite number of them. It is shown here that this characterization of bivariate normal distribution cannot be extended to the multivariate case. More specifically, it is shown that the multivariate normality of subsets (r < n) of the normal variables X ₁, X ₂, …, X_n together with the normality of an infinite number of linear combinations of them do not guarantee the joint normality of these variables.     

Tests for multinormality with applications to time series

Making use of a characterization of multivariate normality by Hermitian polynomials, we propose a multivariate normality test. The approach is then applied to time series analysis by constructing a test for Gaussianity of a stationary univariate series. Simulation study shows that the proposed test has reasonable power and outperforms other tests available in the literature when the innovation series of the time series is symmetric, but non-Gaussian.     

The limiting distribution of a test for multivariate structure

We define a chi-squared statistic for p-dimensional data as follows. First, we transform the data to remove the correlations between the p variables. Then, we discretize each variable into groups of equal size and compute the cell counts in the resulting p-way contingency table. Our statistic is just the usual chi-squared statistic for testing independence in a contingency table. Because the cells have been chosen in a data-dependent manner, this statistic does not have the usual limiting distribution. We derive the limiting joint distribution of the cell counts and the limiting distribution of the chi-squared statistic when the data is sampled from a multivariate normal distribution. The chi-squared statistic is useful in detecting hidden structure in raw data or residuals. It can also be used as a test for multivariate normality.     

A class of robust stepwise alternatives to Hotelling's T 2 tests

Hotelling's T 2 test is known to be optimal under multivariate normality and is reasonably validity-robust when the assumption fails. However, some recently introduced robust test procedures have superior power properties and reasonable type I error control with non-normal populations. These, including the tests due to Tiku & Singh (1982), Tiku & Balakrishnan (1988) and Mudholkar & Srivastava (1999b, c), are asymptotically valid but are useful with moderate size samples only if the population dimension is small. A class of B-optimal modifications of the stepwise alternatives to Hotellings T 2 introduced by Mudholkar & Subbaiah (1980) are simple to implement and essentially equivalent to the T 2 test even with small samples. In this paper we construct and study the robust versions of these modified stepwise tests using trimmed means instead of sample means. We use the robust one- and two-sample trimmed- t procedures as in Mudholkar et al. (1991) and propose statistics based on combining them. The results of an extensive Monte Carlo experiment show that the robust alternatives provide excellent type I error control and a substantial gain in power.     

A Generalization of Shapiro–Wilk's Test for Multivariate Normality

A goodness-of-fit test for multivariate normality is proposed which is based on Shapiro–Wilk's statistic for univariate normality and on an empirical standardization of the observations. The critical values can be approximated by using a transformation of the univariate standard normal distribution. A Monte Carlo study reveals that this test has a better power performance than some of the best known tests for multinormality against a wide range of alternatives.     

Variable selection in joint mean and variance models of Box–Cox transformation

In many applications, a single Box–Cox transformation cannot necessarily produce the normality, constancy of variance and linearity of systematic effects. In this paper, by establishing a heterogeneous linear regression model for the Box–Cox transformed response, we propose a hybrid strategy, in which variable selection is employed to reduce the dimension of the explanatory variables in joint mean and variance models, and Box–Cox transformation is made to remedy the response. We propose a unified procedure which can simultaneously select significant variables in the joint mean and variance models of Box–Cox transformation which provide a useful extension of the ordinary normal linear regression models. With appropriate choice of the tuning parameters, we establish the consistency of this procedure and the oracle property of the obtained estimators. Moreover, we also consider the maximum profile likelihood estimator of the Box–Cox transformation parameter. Simulation studies and a real example are used to illustrate the application of the proposed methods.