Numerical optimization and surface estimation with imprecise function evaluations

This paper presents an investigation of a method for minimizing functions of several parameters where the function need not be computed precisely. Motivated by problems requiring the optimization of negative log-likelihoods, we also want to estimate the (inverse) Hessian at the point of minimum. The imprecision of the function values impedes the application of conventional optimization methods, and the goal of Hessian estimation adds a lot to the difficulty of developing an algorithm. The present class of methods is based on statistical approximation of the functional surface by a quadratic model, so is similar in motivation to many conventional techniques. The present work attempts to classify both problems and algorithmic tools in an effort to prescribe suitable techniques in a variety of situations. The codes are available from the authors' web site http://macnash.admin.uottawa.ca/~rsmin/.