Estimation of the Predictive Ability of Ecological Models

The conventional criteria for predictive model selection do not indicate the absolute goodness of models. For example, the quantity of Akaike Information Criterion (AIC) has meanings only when we compare AIC of different models for a given amount of data. Thus, the existing criteria do not tell us whether the quantity and quality of data is satisfactory, and hence we cannot judge whether we should collect more data to further improve the model or not. To solve such a practical problem, we propose a criterion R_D that lies between 0 and 1. R_D is an asymptotic estimate of the proportion of improvement in the predictive ability under a given error structure, where the predictive ability is defined by the expected logarithmic probability by which the next dataset (2nd dataset) occurs under a model constructed from the current dataset (1st dataset). That is, the predictive ability is defined by the expected logarithmic probability of the 2nd dataset evaluated at the model constructed from the 1st dataset. Appropriate choice of error structures is important in the calculation of R_D. We illustrate examples of calculations of R_D by using a small dataset about the moth abundance.