Genetic programming as a means for programming computers by natural selection

Many seemingly different problems in machine learning, artificial intelligence, and symbolic processing can be viewed as requiring the discovery of a computer program that produces some desired output for particular inputs. When viewed in this way, the process of solving these problems becomes equivalent to searching a space of possible computer programs for a highly fit individual computer program. The recently developed genetic programming paradigm described herein provides a way to search the space of possible computer programs for a highly fit individual computer program to solve (or approximately solve) a surprising variety of different problems from different fields. In genetic programming, populations of computer programs are genetically bred using the Darwinian principle of survival of the fittest and using a genetic crossover (sexual recombination) operator appropriate for genetically mating computer programs. Genetic programming is illustrated via an example of machine learning of the Boolean 11-multiplexer function and symbolic regression of the econometric exchange equation from noisy empirical data.Hierarchical automatic function definition enables genetic programming to define potentially useful functions automatically and dynamically during a run, much as a human programmer writing a complex computer program creates subroutines (procedures, functions) to perform groups of steps which must be performed with different instantiations of the dummy variables (formal parameters) in more than one place in the main program. Hierarchical automatic function definition is illustrated via the machine learning of the Boolean 11-parity function.     

Comparing multiple treatments to both positive and negative controls

In the past, most comparison to control problems have dealt with comparing k test treatments to either positive or negative controls. Dasgupta et al. [2006. Using numerical methods to find the least favorable configuration when comparing k test treatments to both positive and negative controls. Journal of Statistical Computation and Simulation 76, 251–265] enumerate situations where it is imperative to compare several test treatments to both a negative as well as a positive control simultaneously. Specifically, the aim is to see if the test treatments are worse than the negative control, or if they are better than the positive control when the two controls are sufficiently apart. To find critical regions for this problem, one needs to find the least favorable configuration (LFC) under the composite null. In their paper, Dasgupta et al. [2006. Using numerical methods to find the least favorable configuration when comparing k test treatments to both positive and negative controls. Journal of Statistical Computation and Simulation 76, 251–265] came up with a numerical technique to find the LFC. In this paper we verify their result analytically. Via Monte Carlo simulation we compare the proposed method to the logical single step alternatives: Dunnett's [1955. A multiple comparison procedure for comparing several treatments with a control. Journal of the American Statistical Association 50, 1096–1121] or the Bonferroni correction. The proposed method is superior in terms of both the Type I error and the marginal power.     

On exact confidence intervals in a competing risks model with generalized hybrid type-I censored exponential data

In a recent paper by Mao, Shi and Sun that appeared in Journal of Statistical Computation and Simulation, the authors discuss, among other approaches, the construction of exact confidence intervals for the underlying parameters by ‘pivoting the cumulative distribution functions’ of the corresponding maximum likelihood estimators (MLEs). The authors assume that this method is applicable without providing the appropriate justification. In this short note the two requirements for the applicability of this method are discussed, namely, the stochastic monotonicity of the MLEs and the existence of solutions to the equations defining the exact confidence interval's endpoints.     

Statistical Computing Software Reviews

This section is similar in organization to a Book Review section in other journals; however, software of interest to statisticians is the subject of review here. Emphasis is on software for microcomputers. Programs that operate only in larger mainframe computers will seldom receive review. Normally, producers of programs make a copy of their product available to the Section Editor, who then selects one or more persons to test the product and prepare a review.

Producers of computer software who wish to have their product reviewed are invited to contact the Section Editor, Professor Kenneth Berk, Department of Mathematics, 313 Stevenson Hall, Illinois State University, Normal, IL 61761.

Findings and opinions expressed in every review are solely those of the author. They should not be construed as reflecting endorsement of the product, or opinions held, by the American Statistical Association, nor is any warranty implied about any product reviewed.

STAN, Version II.0. David M. Allen. Available from Statistical Consultants, Inc., 462 E. High Street, Lexington, KY 40508. $300. Reviewed by Peter A. Lachenbruch     

Using multivariate rank sum tests to evaluate effectiveness of computer applications in teaching business statistics

Arguments about using computer facilities in classroom teaching have received a lot of attention over time. Using the computer facilities will be helpful to demonstrate real-world applications, while poor data or inappropriate case studies might hinder the applications of the computer programs in classroom teaching. In this paper, we examine the impacts that using computer programs to teach business statistics have on students in the Krannert School of Management at Purdue University. The results show that students are attracted to the interactive computer programs designed for the business statistics course, and students are more motivated to attend classes when computer programs are applied in teaching. Furthermore, computer programs help students to understand confusing topics, and students feel that teaching them to use computer facilities really improves their own abilities to apply similar programs in analyzing real-world problems.     

A numerical nonmetric approach for analyzing time series data

A numerical nonmetric approach to data analysis of periodic series with polytone trend is suggested. Estimation is made of thesmallest number of tone (monotone segments) possible for the trend. The seasonal component is estimated without need for first removing the (estimated) polytone trend. A computer program has been developed which enables analysis of arbitrary series, either by a prespecified length of period or by estimating the period length if not known in advance. Robustness of the proposed approach enables analysis of very short series, series with missing values, and other series with limitations that cannot be easily handled otherwise. In a separate appendix some empirical results obtained by this approach are compared with those from the X-ll program; this appendix will be sent upon request.     

Statistical Computing Software Reviews

This department includes the two sections New Developments in Statistical Computing and Statistical Computing Software Reviews; suitable contents for each of these sections is described under the respective section heading. Articles submitted for the department, outside the two sections, should not be highly technical and should be relevant to the teaching or practice of statistical computing.

This section is similar in organization to a Book Review section in other journals; however, software of interest to statisticians is the subject of review here. Emphasis is on software for microcomputers. Programs that operate only in larger mainframe computers will seldom receive review. Normally, producers of programs make a copy of their product available to the Section Editor, who then selects one or more persons to test the product and prepare a review.

Producers of computer software who wish to have their product reviewed are invited to contact the Section Editor, Professor Kenneth Berk, Department of Mathematics, 313 Stevenson Hall, Illinois State University, Normal, IL 61761.

Findings and opinions expressed in every review are solely those of the author. They should not be construed as reflecting endorsement of the product, or opinions held, by the American Statistical Association, nor is any warranty implied about any product reviewed.

SYSTAT Leland Wilkinson. Available from Systat, Inc., 603 Main St., Evanston, IL 60202. $495 (multiple copy and site fees available). Reviewed by Mark I. Scherwish

AIDA, Version 9/82 David A. Lingwood, Available from Action-Research Northwest, 11442 Marine View Drive, S.W., Seattle, WA 98146. $235. Reviewed by Walter Liggett

MSUSTAT, MS/PC-DOS and CP/M-80 Version 2.20 Richard E. Lund. Available from Research and Development, Inc., Montana State University, Bozeman, MT 59717–0002. $187.50 (university and multiple-copy discounts available). Reviewed by Mervyn G. Marasinghe     

Testing hypotheses and estimating survival from capture histories with CR

Thanks to the progress of the methodology for survival analysis of capture-mark-recapture data, today biologists become able to test the individual or environmental factors that are likely to affect survival and, relatedly, they can estimate survival with a model that describes satisfactorily and efficiently the population and the experiment. Assessment of fit, adjustment and model selection are the main tasks in the process. Several computer programs exist with complementary abilities in those respects and, most often, one must use successively several of them in a single analysis. As there is no standardized presentation of the data, the transition from one program to another is not particularly easy. CR is a software package that intends to alleviate those difficulties by putting together some of the most popular programs and providing passageways between them. We explain how a typical analysis is carried out with CR and insist on the flexibility that can be achieved with SURGE, a program for designing and fitting survival models which is an integral part of CR. A real example is treated for illustration.     

Testing hypotheses and estimating survival from capture histories with CR

Thanks to the progress of the methodology for survival analysis of capture-mark-recapture data, today biologists become able to test the individual or environmental factors that are likely to affect survival and, relatedly, they can estimate survival with a model that describes satisfactorily and efficiently the population and the experiment. Assessment of fit, adjustment and model selection are the main tasks in the process. Several computer programs exist with complementary abilities in those respects and, most often, one must use successively several of them in a single analysis. As there is no standardized presentation of the data, the transition from one program to another is not particularly easy. CR is a software package that intends to alleviate those difficulties by putting together some of the most popular programs and providing passageways between them. We explain how a typical analysis is carried out with CR and insist on the flexibility that can be achieved with SURGE, a program for designing and fitting survival models which is an integral part of CR. A real example is treated for illustration.     

The statistician and the computer

This paper reviews the impact of the computer on the analysis and interpretation of data. It suggests the need for professonal statisticians to recognize that almost all future analysis of data will be carried out by non-statisticians with the aid of statistical program packages. Therefore, the emphasis of statistical training for scientists, engineers, administrators and decision-makers should be on the design of data collection and the choice of appropriate methods of analysis. Both in the teaching of statistics and in the development of computer programs for statistical analysis there are important and urgent tasks to be addressed by professional statisticians.     

A Laguerre polynomial approximation for a goodness-of-fit test for exponential distribution based on progressively censored data

This paper proposes an approximation to the distribution of a goodness-of-fit statistic proposed recently by Balakrishnan et al. [Balakrishnan, N., Ng, H.K.T. and Kannan, N., 2002, A test of exponentiality based on spacings for progressively Type-II censored data. In: C. Huber-Carol et al. (Eds.), Goodness-of-Fit Tests and Model Validity (Boston: Birkhäuser), pp. 89–111.] for testing exponentiality based on progressively Type-II right censored data. The moments of this statistic can be easily calculated, but its distribution is not known in an explicit form. We first obtain the exact moments of the statistic using Basu's theorem and then the density approximants based on these exact moments of the statistic, expressed in terms of Laguerre polynomials, are proposed. A comparative study of the proposed approximation to the exact critical values, computed by Balakrishnan and Lin [Balakrishnan, N. and Lin, C.T., 2003, On the distribution of a test for exponentiality based on progressively Type-II right censored spacings. Journal of Statistical Computation and Simulation, 73 (4), 277–283.], is carried out. This reveals that the proposed approximation is very accurate.     

A computer program for the generation of random variables from any discrete distribution

This paper provides specific directions for the preparation of a discrete pseudo-random number generator computer program using basic machine instructions. The scheme is a table look-up first suggested by Marsaglia. It is applicable to any discrete probability distribution. The general procedure is described for probabilities expressed as fractions in a number system of arbitrary base β. A brief example is given using the decimal system. Flow diagrams accompany the directions which will enable an experienced programmer to write the program for any computer system with only modest storage requirements. Results of chi-square tests performed on samples from specific binomial, Poisson, negative binomial, and hypergeornetric distributions generated using this procedure are given     

A practical method for analysing heavy tailed data

An important practical issue of applying heavy tailed distributions is how to choose the sample fraction or threshold, since only a fraction of upper order statistics can be employed in the inference. Recently, Guillou & Hall ( 2001 ; Journal of Royal Statistical Society B, 63, 293–305) proposed a simple way to choose the threshold in estimating a tail index. In this article, the author first gives an intuitive explanation of the approach in Guillou & Hall ( 2001 ; it Journal of Royal Statistical Society B, 63, 293–305) and then proposes an alternative method, which can be extended to other settings like extreme value index estimation and tail dependence function estimation. Further the author proposes to combine this method for selecting a threshold with a bias reduction estimator to improve the performance of the tail index estimation, interval estimation of a tail index, and high quantile estimation. Simulation studies on both point estimation and interval estimation for a tail index show that both selection procedures are comparable and bias reduction estimation with the threshold selected by either method is preferred. The Canadian Journal of Statistics © 2009 Statistical Society of Canada     

The S-system computation of non-central gamma distribution

The non-central gamma distribution can be regarded as a general form of non-central χ² distributions whose computations were thoroughly investigated (Ruben, H., 1974, Non-central chi-square and gamma revisited. Communications in Statistics, 3(7), 607–633; Knüsel, L., 1986, Computation of the chi-square and Poisson distribution. SIAM Journal on Scientific and Statistical Computing, 7, 1022–1036; Voit, E.O. and Rust, P.F., 1987, Noncentral chi-square distributions computed by S-system differential equations. Proceedings of the Statistical Computing Section, ASA, pp. 118–121; Rust, P.F. and Voit, E.O., 1990, Statistical densities, cumulatives, quantiles, and power obtained by S-systems differential equations. Journal of the American Statistical Association, 85, 572–578; Chattamvelli, R., 1994, Another derivation of two algorithms for the noncentral χ² and F distributions. Journal of Statistical Computation and Simulation, 49, 207–214; Johnson, N.J., Kotz, S. and Balakrishnan, N., 1995, Continuous Univariate Distributions, Vol. 2 (2nd edn) (New York: Wiley). Both distributional function forms are usually in terms of weighted infinite series of the central one. The ad hoc approximations to cumulative probabilities of non-central gamma were extended or discussed by Chattamvelli, Knüsel and Bablok (Knüsel, L. and Bablok, B., 1996, Computation of the noncentral gamma distribution. SIAM Journal on Scientific Computing, 17, 1224–1231), and Ruben (Ruben, H., 1974, Non-central chi-square and gamma revisited. Communications in Statistics, 3(7), 607–633). However, they did not implement and demonstrate proposed numerical procedures. Approximations to non-central densities and quantiles are not available. In addition, its S-system formulation has not been derived. Here, approximations to cumulative probabilities, density, and quantiles based on the method of Knüsel and Bablok are derived and implemented in R codes. Furthermore, two alternate S-system forms are recast on the basis of techniques of Savageau and Voit (Savageau, M.A. and Voit, E.O., 1987, Recasting nonlinear differential equations as S-systems: A canonical nonlinear form. Mathematical Biosciences, 87, 83–115) as well as Chen (Chen, Z.-Y., 2003, Computing the distribution of the squared sample multiple correlation coefficient with S-Systems. Communications in Statistics—Simulation and Computation, 32(3), 873–898.) and Chen and Chou (Chen, Z.-Y. and Chou, Y.-C., 2000, Computing the noncentral beta distribution with S-system. Computational Statistics and Data Analysis, 33, 343–360.). Statistical densities, cumulative probabilities, quantiles can be evaluated by only one numerical solver power low analysis and simulation (PLAS). With the newly derived S-systems of non-central gamma, the specialized non-central χ² distributions are demonstrated under five cases in the same three situations studied by Rust and Voit. Both numerical values in pairs are almost equal. Based on these, nine cases in three similar situations are designed for demonstration and evaluation. In addition, exact values in finite significant digits are provided for comparison. Demonstrations are conducted by R package and PLAS solver in the same PC system. By doing these, very accurate and consistent numerical results are obtained by three methods in two groups. On the other hand, these three methods are performed competitively with respect to speed of computation. Numerical advantages of S-systems over the ad hoc approximation and related properties are also discussed.     

Applied regression analysis bibliography update 1988-89

The 25-page Bibliography in Applied Regression Analysis, 2nd edition, by N.R. Draper and H. Smith, published by Wiley in 1981, is extended by a list of selected references available during 1988-89, and a few older references inadvertently omitted from previous lists. It is hoped that this will be useful to regression practictioners. Items were chosen on the basis of their perceived relevance to practical applications (sometimes rather widely interpreted). The classification system used is that of the book.

The references were selected mostly from the issues of these journals: Annals of Statistics; Applied Statistics: Biometrika; Canadian Journal of Statistics; Bulletin of the International Statistical Institute; Journal of the American Statistical Association; Journal of Quality Technology; Journal of the Royal Statistical Society, Series A and B; and Technometrics.

The author would appreciate being notified of errors, however slight, so that these do not persist in future compilations.     

An Eight-Step Guide to Creating and Sustaining a Mentoring Program

Mentoring is an extremely valuable activity for both individuals and organizations. Mentoring within organizations can develop and integrate employees into their corporate culture. Mentoring outside the mentees’ work groups or through professional development organizations can give broader perspective and support, especially in times of transition. But mentoring programs require tremendous effort to start, organize, and maintain. Few last more than two years. This article provides a structured approach to starting and sustaining a successful program. The steps include understanding an organization’s particular needs, learning from small pilot programs, following up with mentoring pairs during a committed formal mentoring period, and evaluating results from each program’s cycle to learn and grow the program. Supplementary materials for this article are available online.     

Comparison of programs MULT,ESTIMATE and BROWNIE

I describe and compare programs ESTIMATE (one-age class band recovery), BROWNIE (two-age class band recovery) and MULT (one- and two-age class band recovery and extensions). All are easy to use and suited for the purposes for which they were designed. MULT offers considerable advantages over the other two programs, in providing a unified framework for both one- and two-age class band recovery problems, as well as extensions to allow inclusion of a covariate, banding twice per year, unequal intervals between banding periods, and analysis of reward-band data. MULT is menu-driven, and offers additional advantages in terms of ease of input and output and user-modified defaults (e.g. selection or suppression of some printed output). Users seeking additional flexibility are referred to program SURVIV but advised that both user sophistication and software (i.e. a FORTRAN compiler) beyond that required for MULT, ESTIMATE or BROWNIE would be required.