General lack of fit tests based on families of groupings

Lack of fit tests based on groupings of the observations are developed. These tests are first applied to models with replication. In this case, the classic Fisher test assumes that the true model is contained in the one-way ANOVA model. However, Christensen (2003). Significantly insignificant F tests. Amer. Statist. 57, 27–32] has noted that small values of the F-statistic may indicate lack of fit due to features which are not part of the proposed model. Such model inadequacy is called within-cluster lack of fit, whereas the standard Fisher lack of fit is called between-cluster lack of fit. Typically, lack of fit exists as a combination of these two pure types, and can be extremely difficult to detect depending on the nature of the mixture. In this paper, the one-way ANOVA model is embedded in larger models using groupings of the observations, which provides tests with good power for detecting all of the above types of model inadequacies, including mixtures. In particular, several such tests are considered, each based on a different grouping of the observations, and the multiple testing approach of Baraud et al. (2003). Adaptive tests of linear hypotheses by model selection. Ann. Statist. 31, 225–251] is followed. More generally, the preceding testing procedure based on families of groupings is extended to the case of nonreplication. For this case, it is proposed that such families be determined by linear orders on the predictors based on disjoint parallel tubes in predictor space. Test statistics follow the cluster-based regression lack of fit tests presented by Christensen (1989). Lack of fit based on near or exact replicates. Ann. Statist. 17, 673–683; (1991). Small sample characterizations of near replicate lack of fit tests. J. Amer. Statist. Assoc. 86, 752–756], by considering the groupings as determining special types of clusterings. In order to detect general lack of fit, several such tests are again considered, each based on a different grouping of the observations, and the multiple testing approach given by Baraud et al. (2003). Adaptive tests of linear hypotheses by model selection. Ann. Statist. 31, 225–251] is followed. Simulation results illustrating the power of the proposed testing procedure are given.