The jack-knife with a stepwise discriminant algorithm-a warning to BMDP usesr

Program 7M in the BMDP statistical package carries out Fisher's linear discriminant analysis and offers a number of options including stepwise variable selection and jack-knife cross-validation. If these two options are used in combination, then a difficulty arises. A Monte Carlo demonstration was performed to demonstrate the problem. The results showed that the classification performance of the stepwise technique under jack-knife appeared to improve when pure noise variables were added to the data set. After studying the manual, it was hypothesized that this phenomenon was due to the stepwise variable selection phase of the procedure being conducted before the jack-knife, rather than being subject to it. This was verified by a second Monte Carlo experiment which compared the jack-knife in BMDP with a complete jack-knife, in which the variable selection phase of the procedure was also subject to the jack-knife.