Solving the Classification Problem in Discriminant Analysis Via Linear and Nonlinear Programming Methods*

This paper demonstrates the feasibility of applying nonlinear programming methods to solve the classification problem in discriminant analysis. The application represents a useful extension of previously proposed linear programming-based solutions for discriminant analysis. The analysis of data obtained by conducting a Monte Carlo simulation experiment shows that these new procedures are promising. Future research that should promote application of the proposed methods for solving classification problems in a business decision-making environment is discussed.     

Concepts,Theory, and Techniques ROBUST METHODS: AN ALTERNATIVE APPROACH FOR ANALYZING DATA SETS CONTAINING INFLUENTIAL DATA POINTS

Recent advances in statistical estimation theory have resulted in the development of new procedures, called robust methods, that can be used to estimate the coefficients of a regression model. Because such methods take into account the impact of discrepant data points during the initial estimation process, they offer a number of advantages over ordinary least squares and other analytical procedures (such as the analysis of outliers or regression diagnostics). This paper describes the robust method of analysis and illustrates its potential usefulness by applying the technique to two data sets. The first application uses artificial data; the second uses a data set analyzed previously by Tufte [15] and, more recently, by Chatterjee and Wiseman [6].     

Discriminant Analysis Using Least Absolute Deviations

Several linear programming methods have been suggested as discrimination procedures. A least absolute deviations regression procedure is developed here which is simpler to use and does not suffer from any lack of invariance. A simulation study shows it to be at least as effective as any of the methods previously discussed for normal and heavy-tailed distributions.     

FOUR APPROACHES TO THE CLASSIFICATION PROBLEM IN DISCRIMINANT ANALYSIS: AN EXPERIMENTAL STUDY*

Four discriminant models were compared in a simulation study: Fisher's linear discriminant function [14], Smith's quadratic discriminant function [34], the logistic discriminant model, and a model based on linear programming [17]. The study was conducted to estimate expected rates of misclassification for these four procedures when observations were sampled from a variety of normal and nonnormal distributions. In contrast to previous research, data were taken from four types of Kurtotic population distributions. The results indicate the four discriminant procedures are robust toward data from many types of distributions. The misclassification rates for both the logistic discriminant model and the formulation based on linear programming consistently decreased as the kurtosis in the data increased. The decreases, however, were of small magnitude. None of these procedures yielded statistically significant lower rates of misclassification under nonnormality. The quadratic discriminant function produced significantly lower error rates when the variances across groups were heterogeneous.     

Least Absolute Value Regression: Necessary Sample Sizes to Use Normal Theory Inference Procedures*

Recently developed large sample inference procedures for least absolute value (LAV) regression are examined via Monte Carlo simulation to determine when sample sizes are large enough for the procedures to work effectively. A variety of different experimental settings were created by varying the disturbance distribution, the number of explanatory variables and the way the explanatory variables were generated. Necessary sample sizes range from as small as 20 when disturbances are normal to as large as 200 in extreme outlier-producing distributions.     

EVALUATING ALTERNATIVE LINEAR PROGRAMMING MODELS TO SOLVE THE TWO-GROUP DISCRIMINANT PROBLEM

The two-group discriminant problem has applications in many areas, for example, differentiating between good credit risks and poor ones, between promising new firms and those likely to fail, or between patients with strong prospects for recovery and those highly at risk. To expand our tools for dealing with such problems, we propose a class of nonpara-metric discriminant procedures based on linear programming (LP). Although these procedures have attracted considerable attention recently, only a limited number of computational studies have examined the relative merits of alternative formulations. In this paper we provide a detailed study of three contrasting formulations for the two-group problem. The experimental design provides a variety of test conditions involving both normal and nonnormal populations. Our results establish the LP model which seeks to minimize the sum of deviations beyond the two-group boundary as a promising alternative to more conventional linear discriminant techniques.     

A VARIANCE-REDUCTION STRATEGY FOR RSM SIMULATION STUDIES

The application of optimization techniques in digital simulation experiments is frequently complicated by the presence of large experimental error variances. Two of the more widely accepted design strategies for the resolution of this problem include the assignment of common pseudorandom number streams and the assignment of antithetic pseudorandom number streams to the experimental points. When considered separately, however, each of these variance-reduction procedures has rather restrictive limitations. This paper examines the simultaneous use of these two techniques as a variance-reduction strategy in response surface methodology (RSM) analysis of simulation models. A simulation of an inventory system is used to illustrate the application and benefits of this assignment procedure, as well as the basic components of an RSM analysis.     

A MINIMUM-COST AUDIT SAMPLING METHODOLOGY UNDER CONDITIONS OF PREDETERMINED BETA RISK

This paper presents a minimum-cost methodology for determining a statistical sampling plan in substantive audit tests. In this model, the auditor specifies β, the risk of accepting an account balance as correct when it is not, according to audit evidence requirements. Using β as a constraint, the auditor then selects a sampling plan to optimize the trade-off between sampling costs and the costs of follow-up audit procedures. Tables to aid in this process and an illustration are provided.     

THE ANALYSIS OF OUTLYING DATA POINTS USING ROBUST REGRESSION: A MULTIVARIATE PROBLEM-BANK IDENTIFICATION MODEL*

Because the eight largest bank failures in United States history have occurred since 1973 [24], the development of early-warning problem-bank identification models is an important undertaking. It has been shown previously [3] [5] that M-estimator robust regression provides such a model. The present paper develops a similar model for the multivariate case using both a robustified Mahalanobis distance analysis [21] and principal components analysis [10]. In addition to providing a successful presumptive problem-bank identification model, combining the use of the M-estimator robust regression procedure and the robust Mahalanobis distance procedure with principal components analysis is also demonstrated to be a general method of outlier detection. The results from using these procedures are compared to some previously suggested procedures, and general conclusions are drawn.     

A NOTE ON PROCEDURES FOR TESTING THE QUALITY OF A CLUSTERING OF A SET OF OBJECTS

Despite the increased application of cluster analysis in decision sciences, few attempts have been made to derive hypothesis-testing procedures for the evaluation of clustering solutions. In fact, the present paper shows that at least one such attempt failed to specify a meaningful sampling distribution for the test procedure. An alternative index based on the concept of point-biserial correlation is proposed as a possible recovery measure. The index is subsequently used to form the basis of a valid statistical test for the existence of cluster structure.     

A Response to Xiao's “Necessary and Sufficient Conditions of Unacceptable Solutions in LP Discriminant Analysis”: Something is Amiss

Characterization of unacceptable solutions of linear programming methods for linear discriminant problems have been studied by many researchers. This note shows that a recent correction was not a correction but a tightening of prior results.     

IMPORTANCE OF HYPOTHESES: GROUP COMPARISON ANALYSIS

A method is presented for measuring the importance of a mean-difference hypothesis and for a set of such hypotheses. The method is based on the population prediction uncertainty associated with the models for the alternative and null hypotheses. Previous methods for ANOVA designs are special cases of the method presented here. The method is demonstrated showing its flexibility for the analysis of mean-difference hypotheses. The method encourages tests of meaningful research hypotheses rather than fitting the hypotheses to the method.     

Applications and Implementation

We propose an alternative solution to the discriminant problem, one that requires little more than a minimum familiarity with linear programming. The approach shows promise for eliminating the complexities of conventional statistical approaches without sacrificing the essential power of existing methods.     

A TIME-SERIES APPROACH FOR CONSTRUCTING EXPECTATIONS MODELS*

This article presents an efficient way of dealing with adaptive expectations models—a way that makes use of all the information available in the data. The procedure is based on multiple-input transfer functions (MITFs): by calculating lead and lag cross correlations between innovations associated with the variables in the model, it is possible to determine which periods have the greatest effects on the dependent variable. If information about k periods ahead is required, fitted values for the expectation variables are used to generate k-period-ahead forecasts. These in turn can be used in the estimation of the transfer function equation, which not only contains the usual lagged variables but also allows for incorporation of lead-fitted values for the expectation variables. The MITF identification and estimation procedures used are based on the corner method. The method is contrasted with the Almon distributed-lag approach using a model relating stock market prices to interest rates and expected corporate profits.     

ROBUST METHODS APPLIED TO ENERGY-CONSUMPTION DATA

In many instances a model is based on nonexperimental data and is used for prediction. In this situation, using both the least-squares and the robust methods simultaneously can result in models with improved predictive performance.     

The Interpretation of Coefficients in Models with Qualitative Dependent Variables*

A substantial body of empirical accounting, finance, management, and marketing research utilizes single equation models with discrete dependent variables. Generally, the interpretation of the coefficients of the exogenous variables is limited to the sign and relative magnitude. This paper presents three methods of interpreting the coefficients in these models. The first method interprets the coefficients as marginal probabilities and the second method interprets the coefficients as elasticities of probability. The third method utilizes sensitivity analysis and examines the effect of hypothetical changes in exogenous variables on the probability of choice. This paper applies these methods to a published research study.     

A Computational Study of Replicated Clustering with an Application to Market Segmentation*

In most commercial applications of k-means clustering, researchers choose one set of kseed points to start the partitioning process; often, the initial set of seeds is chosen randomly. Using Monte Carlo simulation, we show that significant benefits are associated with replicated starting configurations that incorporate seed selection procedures based on a hierarchical clustering of sample points drawn from the original data matrix. A real-world application of the approach is then presented.     

USING AN UNCERTAINTY MODEL TO ASSESS SALES RESPONSE TO ADVERTISING*

Implications of a probabilistic “random walk” model of incremental sales response to advertising are developed for various timing patterns of advertising expenditures. Maximum likelihood procedures for assessing advertising effectiveness and for estimating a decay (forgetting) rate are developed and applied to artificial data of known configuration and are used to assess the impact over time of a brochure program on mail-order sales. Results are also compared to those from alternative models involving various lag patterns in advertising effects.     

MULTIVARIATE NORMALITY AND A BOND RATING DECISION MODEL*

An assumption of multivariate normality for a decision model is validated in this paper. Measurements for the independent variables of a bond rating model were taken from a sample of municipal bonds. Three methods for examining both univariate and multivariate normality (including normal probability plots) are described and applied to the bond data. The results imply, after applying normalizing transformations to four of the variables, that the data reasonably approximate multivariate normality, thereby validating a distributional requirement of the discriminant-analysis-based decision model. The methods described in the paper may also be used by others interested in examining multivariate normality assumptions of decision models.