An empirical application of multidimensional scaling to advertising pretesting

One of the least-managed aspects of advertising management may well be that of effectiveness measurement. The problem of obtaining satisfactory indications of advertising effectiveness is particularly critical in pretesting since the rationale for pretesting is to evaluate advertisements before their release for broadscale media distribution. Therefore, the primary purpose of the research reported here is to provide empirical evidence of the value of multidimensional scaling methods to pretesting advertising. The article demonstrates how nonmetric methodology obviates the disadvantages of such methods as consumer jury tests and rating scales. A secondary purpose of the research project is to demonstrate the value of pretesting itself by comparing the perceptions of target market consumers with those of selected group of advertising professionals.     

Adaptive Confidence Intervals for the Test Error in Classification

The estimated test error of a learned classifier is the most commonly reported measure of classifier performance. However, constructing a high quality point estimator of the test error has proved to be very difficult. Furthermore, common interval estimators (e.g. confidence intervals) are based on the point estimator of the test error and thus inherit all the difficulties associated with the point estimation problem. As a result, these confidence intervals do not reliably deliver nominal coverage. In contrast we directly construct the confidence interval by use of smooth data-dependent upper and lower bounds on the test error. We prove that for linear classifiers, the proposed confidence interval automatically adapts to the non-smoothness of the test error, is consistent under fixed and local alternatives, and does not require that the Bayes classifier be linear. Moreover, the method provides nominal coverage on a suite of test problems using a range of classification algorithms and sample sizes.