A Note on Discriminant Analysis Using LAD

Using a regression approach to discriminant analysis is often incorrect because it forces the use of a binary dependent variable which violates virtually any distributional assumption for a linear model. However, assuming a Laplace distribution in an LP framework leads to a theoretical foundation for MSD discriminant analysis.     

EVALUATING ALTERNATIVE LINEAR PROGRAMMING MODELS TO SOLVE THE TWO-GROUP DISCRIMINANT PROBLEM

The two-group discriminant problem has applications in many areas, for example, differentiating between good credit risks and poor ones, between promising new firms and those likely to fail, or between patients with strong prospects for recovery and those highly at risk. To expand our tools for dealing with such problems, we propose a class of nonpara-metric discriminant procedures based on linear programming (LP). Although these procedures have attracted considerable attention recently, only a limited number of computational studies have examined the relative merits of alternative formulations. In this paper we provide a detailed study of three contrasting formulations for the two-group problem. The experimental design provides a variety of test conditions involving both normal and nonnormal populations. Our results establish the LP model which seeks to minimize the sum of deviations beyond the two-group boundary as a promising alternative to more conventional linear discriminant techniques.     

A NEW CLASS OF MODELS FOR THE DISCRIMINANT PROBLEM

Discriminant analysis is relevant to business decision making in a variety of contexts, such as when one decides to make or buy a specified component, fund a venture project, or hire a particular person. Potential applications in artificial intelligence, particularly in the area of pattern recognition, have further underscored the importance of the field. A recent innovation in discriminant analysis is provided by special linear programming (LP) models, which offer attractive alternatives to classical statistical approaches. The scope of application in which discriminant analysis can be advantageously employed is broadened by the flexibility to tailor parameters in the LP approaches to reflect diverse goals and by the power to explore the sensitivity of these parameters. In spite of the promise of the LP formulations, however, limitations to their effectiveness have been uncovered in certain settings. A recent advance involving a normalization construct removes some of the limitations but entails solving the LP model twice (to allow for different signs of a normalization constant) and does not yield equivalent solutions for different rotations of the problem data. This paper introduces a new model and a new class of normalizations that remedy both remaining limitations, making it possible to take advantage of the modeling capabilities of the LP formulations without the attendant shortcomings encountered by earlier investigations. Our development shows by empirical testing and illustrative analysis that the quality of solutions from LP discriminant approaches is more favorable (relative to the classical model) than previously supposed.     

Improved Linear Programming Models for Discriminant Analysis*

Discriminant analysis is an important tool for practical problem solving. Classical statistical applications have been joined recently by applications in the fields of management science and artificial intelligence. In a departure from the methodology of statistics, a series of proposals have appeared for capturing the goals of discriminant analysis in a collection of linear programming formulations. The evolution of these formulations has brought advances that have removed a number of initial shortcomings and deepened our understanding of how these models differ in essential ways from other familiar classes of LP formulations. We will demonstrate, however, that the full power of the LP discriminant analysis models has not been achieved, due to a previously undetected distortion that inhibits the quality of solutions generated. The purpose of this paper is to show how to eliminate this distortion and thereby increase the scope and flexibility of these models. We additionally show how these outcomes open the door to special model manipulations and simplifications, including the use of a successive goal method for establishing a series of conditional objectives to achieve improved discrimination.     

SOME DIFFICULTIES AND IMPROVEMENTS IN APPLYING LINEAR PROGRAMMING FORMULATIONS TO THE DISCRIMINANT PROBLEM

In this paper, we discuss some disturbing features of two linear programming (LP) approaches to the discriminant problem. Specifically, we show that both approaches are sensitive to the choice of origin for the data although, intuitively, placement of origin should have no effect on the method of assigning cases to groups. In addition, we show that these LP approaches may lead to discriminant functions which assign all cases to the same group. We show that the usual statistical approach to this problem does not share these difficulties, and we make recommendations for implementing these LP approaches which help to alleviate the difficulties.     

Decision Power and Solutions of LP Discriminant Models: Rejoinder

A major problem of LP discriminant analysis is the validity of the solution. This note shows that to comprehend the effectiveness of the solution, conditions for unacceptable solutions need to be tightly characterized.     

FOUR APPROACHES TO THE CLASSIFICATION PROBLEM IN DISCRIMINANT ANALYSIS: AN EXPERIMENTAL STUDY*

Four discriminant models were compared in a simulation study: Fisher's linear discriminant function [14], Smith's quadratic discriminant function [34], the logistic discriminant model, and a model based on linear programming [17]. The study was conducted to estimate expected rates of misclassification for these four procedures when observations were sampled from a variety of normal and nonnormal distributions. In contrast to previous research, data were taken from four types of Kurtotic population distributions. The results indicate the four discriminant procedures are robust toward data from many types of distributions. The misclassification rates for both the logistic discriminant model and the formulation based on linear programming consistently decreased as the kurtosis in the data increased. The decreases, however, were of small magnitude. None of these procedures yielded statistically significant lower rates of misclassification under nonnormality. The quadratic discriminant function produced significantly lower error rates when the variances across groups were heterogeneous.     

Necessary and Sufficient Conditions of Unacceptable Solutions in LP Discriminant Analysis*

Characterization of unacceptable solutions for linear programming (LP) discriminant models have been discussed in the literature and the results presented so far are not satisfactory. This paper establishes necessary and sufficient conditions of unacceptable solutions for a number of LP models, addresses the practical implications of these conditions, and discusses the relationship between unacceptable solutions and multiple solutions.     

Refining Two-Group Multivariable Classification Models Using Univariate Optimal Discriminant Analysis*

Fisher's discriminant analysis (FDA) is often used to obtain a prediction model for dichotomous classifications on the basis of two or more independent variables. FDA provides an equation whereby values on independent variables are combined into a single predicted value (Y*) that is compared against a cutpoint and direction in order to make classifications. Theoretically, univariate optimal discriminant analysis employed on these Y* will maximize training classification accuracy. This methodology is illustrated using three examples.     

Separation Failure in Linear Programming Discriminant Models

Linear programming discriminant analysis (LPDA) models are designed around a variety of objective functions, each representing a different measure of separation of the training samples by the resulting discriminant function. A separation failure is defined to be the selection of an “optimal” discriminant function which incompletely separates a pair of completely separable training samples. Occurrence of a separation failure suggests that the chosen discriminant function may have an unnecessarily low classification accuracy on the actual populations involved. In this paper, a number of the LPDA models proposed for the two-group case are examined to learn which are subject to separation failure. It appears that separation failure in any model can be avoided by applying the model twice, reversing group designations.     

Notes and Communications RESOLVING CERTAIN DIFFICULTIES AND IMPROVING THE CLASSIFICATION POWER OF LP DISCRIMINANT ANALYSIS FORMULATIONS

In certain settings, difficulties arise that limit the effectiveness of LP formulations for the discriminant problem. Explanations and possible remedies have been offered, but these have had only limited success. We provide a simple way to overcome these problems based on an appropriate use and interpretation of normalizations. In addition, we demonstrate a normalization that is invariant under all translations of the problem data, providing a stability property not shared by previous approaches. Finally, we discuss the possibility of using more general models to improve discrimination.     

A Multivariate Methodology for Simultaneously Capturing and Clustering Judgment Policies

This paper presents a new linear model methodology for clustering judges with homogeneous decision policies and differentiating dimensions which distinguish judgment policies. This linear policy capturing model based on canonical correlation analysis is compared to the standard model based on regression analysis and hierarchical agglomerative clustering. Potential advantages of the new methodology include simultaneous instead of sequential consideration of information in the dependent and independent variable sets, decreased interpretational difficulty in the presence of multicollinearity and/or suppressor/moderator variables, and a more clearly defined solution structure allowing assessment of a judge's relationship to all of the derived, ideal policy types. An application to capturing policies of information systems recruiters responsible for hiring entry-level personnel is used to compare and contrast the two techniques.     

A Comparison of Linear Programming and Parametric Approaches to the Two-Group Discriminant Problem*

Recent simulation-based studies of linear programming models for discriminant analysis have used the Fisher linear discriminant function as the benchmark for parametric methods. This article reports experimental evidence which suggests that, while some linear programming models may match or even exceed the Fisher approach in classification accuracy, none of the fifteen models tested is as accurate on normally distributed data as the Smith quadratic discriminant function. At the minimum, further testing is warranted with an emphasis on data sets that arise from significantly non-Gaussian populations.     

Theoretical Distributions of Optima for Univariate Discrimination of Random Data*

Optimal linear discriminant models maximize percentage accuracy for dichotomous classifications, but are rarely used because a theoretical framework that allows one to make valid statements about the statistical significance of the outcomes of such analyses does not exist. This paper describes an analytic solution for the theoretical distribution of optimal values for univariate optimal linear discriminant analysis, under the assumption that the data are random and continuous. We also present the theoretical distribution for sample sizes up to N= 30. The discovery of a statistical framework for evaluating the performance of optimal discriminant models should greatly increase their use by scientists in all disciplines.     

Solving the Classification Problem in Discriminant Analysis Via Linear and Nonlinear Programming Methods*

This paper demonstrates the feasibility of applying nonlinear programming methods to solve the classification problem in discriminant analysis. The application represents a useful extension of previously proposed linear programming-based solutions for discriminant analysis. The analysis of data obtained by conducting a Monte Carlo simulation experiment shows that these new procedures are promising. Future research that should promote application of the proposed methods for solving classification problems in a business decision-making environment is discussed.     

MATCHED PAIRS DISCRIMINATION: METHODOLOGY AND AN INVESTIGATION OF CORPORATE ACCOUNTING POLICIES

The matched-pairs methodology is becoming increasingly popular as a means of controlling extraneous factors in business research. This paper develops discriminant procedures for matched data and examines the properties of these methods. Data from a recent study by Hunt [14] on the determinants of inventory method choice are used to contrast the performance of the different methods. While all of the methods yield the same set of discriminating variables, those procedures that allow for the dependence among observations within a pair provide greater classificatory power than traditional multivariate techniques.     

Mathematical Programming Formulations for the Discriminant Problem: An Old Dog Does New Tricks*

In recent years, much research has been done on the application of mathematical programming (MP) techniques to the discriminant problem. While promising results have been obtained, many of these techniques are plagued by a number of problems associated with the model formulation including unbounded, improper, and unacceptable solutions as well as solution instability under linear transformation of the data. In attempting to solve these problems, numerous formulations have been proposec involving additional variables and/or normalization constraints. While effective, these models can also become quite complex. In this paper we demonstrate that a simple, well-known special case of Hand's [13] original formulation provides an implicit normalization which avoids the problems for which various complicated remedies have been devised. While other researchers have made use of this formulation, its properties have not previously been fully recognized.     

Characterization of Unacceptable Solutions in LP Discriminant Analysis*

Many linear programming models have been proposed for performing discriminant analysis. Partial characterizations for unacceptable solutions have been presented and new models proposed to circumvent these problems. In this paper those conditions leading to unacceptable solutions for all two-group models are characterized.     

METHODS FOR ANALYZING EXPERIMENTS WITH MULTIPLE CRITERIA*

An approach to analyzing experimental data with multiple criteria is explained and demonstrated on data from a test of the effectiveness of two posters. As a supplement to traditional multivariate analysis of variance and covariance, the application of a step-down F test is advocated when an ordering of the criterion is meaningful, and an analysis of contrasts is recommended when such an ordering is not managerially relevant. The step-down procedure has the advantage of simultaneously testing an overall hypothesis and hypotheses on each criterion variable.     

ESTIMATION OF MISCLASSIFICATION PROBABILITIES IN THE LINEAR PROGRAMMING APPROACHES TO THE TWO-GROUP DISCRIMINANT PROBLEM

Expressions for misclassification probabilities are derived under a contaminated multivariate normal model for the linear-programming approaches to the two-group discriminant problem.