I-robust and D-robust designs on a finite design space

We present and discuss the theory of minimax I- and D-robust designs on a finite design space, and detail three methods for their construction that are new in this context: (i) a numerical search for the optimal parameters in a provably minimax robust parametric class of designs, (ii) a first-order iterative algorithm similar to that of Wynn (Ann Math Stat 5:1655–1664, 1970), and (iii) response-adaptive designs. These designs minimize a loss function, based on the mean squared error of the predicted responses or the parameter estimates, when the regression response is possibly misspecified. The loss function being minimized has first been maximized over a neighbourhood of the approximate and possibly inadequate response being fitted by the experimenter. The methods presented are all vastly more economical, in terms of the computing time required, than previously available algorithms.     

Minimax A‐ and D‐optimal integer‐valued wavelet designs for estimation

The author discusses integer‐valued designs for wavelet estimation of nonparametric response curves in the possible presence of heteroscedastic noise using a modified wavelet version of the Gasser‐Müller kernel estimator or weighted least squares estimation. The designs are constructed using a minimax treatment and the simulated annealing algorithm. The author presents designs for three case studies in experiments for investigating Gompertz's theory on mortality rates, nitrite utilization in bush beans and the impact of crash helmets in motorcycle accidents.     

Testing in mixed-effects FANOVA models

We consider the testing problem in the mixed-effects functional analysis of variance models. We develop asymptotically optimal (minimax) testing procedures for testing the significance of functional global trend and the functional fixed effects based on the empirical wavelet coefficients of the data. Wavelet decompositions allow one to characterize various types of assumed smoothness conditions on the response function under the nonparametric alternatives. The distribution of the functional random-effects component is defined in the wavelet domain and captures the sparseness of wavelet representation for a wide variety of functions. The simulation study presented in the paper demonstrates the finite sample properties of the proposed testing procedures. We also applied them to the real data from the physiological experiments.     

D-optimal minimax regression designs on discrete design space

One classical design criterion is to minimize the determinant of the covariance matrix of the regression estimates, and the designs are called D-optimal designs. To reflect the nature that the proposed models are only approximately true, we propose a robust design criterion to study response surface designs. Both the variance and bias are considered in the criterion. In particular, D-optimal minimax designs are investigated and constructed. Examples are given to compare D-optimal minimax designs with classical D-optimal designs.     

Robust designs for one-point extrapolation

We consider the construction of designs for the extrapolation of a regression response to one point outside of the design space. The response function is an only approximately known function of a specified linear function. As well, we allow for variance heterogeneity. We find minimax designs and corresponding optimal regression weights in the context of the following problems: (P1) for nonlinear least squares estimation with homoscedasticity, determine a design to minimize the maximum value of the mean squared extrapolation error (MSEE), with the maximum being evaluated over the possible departures from the response function; (P2) for nonlinear least squares estimation with heteroscedasticity, determine a design to minimize the maximum value of MSEE, with the maximum being evaluated over both types of departures; (P3) for nonlinear weighted least squares estimation, determine both weights and a design to minimize the maximum MSEE; (P4) choose weights and design points to minimize the maximum MSEE, subject to a side condition of unbiasedness. Solutions to (P1)–(P4) are given in complete generality. Numerical comparisons indicate that our designs and weights perform well in combining robustness and efficiency. Applications to accelerated life testing are highlighted.     

Designs for weighted least squares regression, with estimated weights

We study designs, optimal up to and including terms that are O(n ^?1), for weighted least squares regression, when the weights are intended to be inversely proportional to the variances but are estimated with random error. We take a finite, but arbitrarily large, design space from which the support points are to be chosen, and obtain the optimal proportions of observations to be assigned to each point. Specific examples of D- and I-optimal design for polynomial responses are studied. In some cases the same designs that are optimal under homoscedasticity remain so for a range of variance functions; in others there tend to be more support points than are required in the homoscedastic case. We also exhibit minimax designs, that minimize the maximum, over finite classes of variance functions, value of the loss. These also tend to have more support points, often resulting from the breaking down of replicates into clusters.     

Wavelet thresholding via a Bayesian approach

We discuss a Bayesian formalism which gives rise to a type of wavelet threshold estimation in nonparametric regression. A prior distribution is imposed on the wavelet coefficients of the unknown response function, designed to capture the sparseness of wavelet expansion that is common to most applications. For the prior specified, the posterior median yields a thresholding procedure. Our prior model for the underlying function can be adjusted to give functions falling in any specific Besov space. We establish a relationship between the hyperparameters of the prior model and the parameters of those Besov spaces within which realizations from the prior will fall. Such a relationship gives insight into the meaning of the Besov space parameters. Moreover, the relationship established makes it possible in principle to incorporate prior knowledge about the function's regularity properties into the prior model for its wavelet coefficients. However, prior knowledge about a function's regularity properties might be difficult to elicit; with this in mind, we propose a standard choice of prior hyperparameters that works well in our examples. Several simulated examples are used to illustrate our method, and comparisons are made with other thresholding methods. We also present an application to a data set that was collected in an anaesthesiological study.     

A semi-infinite programming based algorithm for finding minimax optimal designs for nonlinear models

Minimax optimal experimental designs are notoriously difficult to study largely because the optimality criterion is not differentiable and there is no effective algorithm for generating them. We apply semi-infinite programming (SIP) to solve minimax design problems for nonlinear models in a systematic way using a discretization based strategy and solvers from the General Algebraic Modeling System (GAMS). Using popular models from the biological sciences, we show our approach produces minimax optimal designs that coincide with the few theoretical and numerical optimal designs in the literature. We also show our method can be readily modified to find standardized maximin optimal designs and minimax optimal designs for more complicated problems, such as when the ranges of plausible values for the model parameters are dependent and we want to find a design to minimize the maximal inefficiency of estimates for the model parameters.     

Robust extrapolation designs and weights for biased regression models with heteroscedastic errors

We consider the construction of designs for the extrapolation of regression responses, allowing both for possible heteroscedasticity in the errors and for imprecision in the specification of the response function. We find minimax designs and correspondingly optimal estimation weights in the context of the following problems: (1) for ordinary least squares estimation, determine a design to minimize the maximum value of the integrated mean squared prediction error (IMSPE), with the maximum being evaluated over both types of departure; (2) for weighted least squares estimation, determine both weights and a design to minimize the maximum IMSPE; (3) choose weights and design points to minimize the maximum IMSPE, subject to a side condition of unbiasedness. Solutions to (1) and (2) are given for multiple linear regression with no interactions, a spherical design space and an annular extrapolation space. For (3) the solution is given in complete generality; as one example we consider polynomial regression. Applications to a dose-response problem for bioassays are discussed. Numerical comparisons, including a simulation study, indicate that, as well as being easily implemented, the designs and weights for (3) perform as well as those for (1) and (2) and outperform some common competitors for moderate but undetectable amounts of model bias.     

Posterior probability intervals for wavelet thresholding

Summary. We use cumulants to derive Bayesian credible intervals for wavelet regression estimates. The first four cumulants of the posterior distribution of the estimates are expressed in terms of the observed data and integer powers of the mother wavelet functions. These powers are closely approximated by linear combinations of wavelet scaling functions at an appropriate finer scale. Hence, a suitable modification of the discrete wavelet transform allows the posterior cumulants to be found efficiently for any given data set. Johnson transformations then yield the credible intervals themselves. Simulations show that these intervals have good coverage rates, even when the underlying function is inhomogeneous, where standard methods fail. In the case where the curve is smooth, the performance of our intervals remains competitive with established nonparametric regression methods.     

Minimax A-, c-, and I-optimal regression designs for models with heteroscedastic errors

It is well known that it is difficult to construct minimax optimal designs. Furthermore, since in practice we never know the true error variance, it is important to allow small deviations and construct robust optimal designs. We investigate a class of minimax optimal regression designs for models with heteroscedastic errors that are robust against possible misspecification of the error variance. Commonly used A-, c-, and I-optimality criteria are included in this class of minimax optimal designs. Several theoretical results are obtained, including a necessary condition and a reflection symmetry for these minimax optimal designs. In this article, we focus mainly on linear models and assume that an approximate error variance function is available. However, we also briefly discuss how the methodology works for nonlinear models. We then propose an effective algorithm to solve challenging nonconvex optimization problems to find minimax designs on discrete design spaces. Examples are given to illustrate minimax optimal designs and their properties.     

Robust estimation and wavelet thresholding in partially linear models

This paper is concerned with a semiparametric partially linear regression model with unknown regression coefficients, an unknown nonparametric function for the non-linear component, and unobservable Gaussian distributed random errors. We present a wavelet thresholding based estimation procedure to estimate the components of the partial linear model by establishing a connection between an l ₁-penalty based wavelet estimator of the nonparametric component and Huber’s M-estimation of a standard linear model with outliers. Some general results on the large sample properties of the estimates of both the parametric and the nonparametric part of the model are established. Simulations are used to illustrate the general results and to compare the proposed methodology with other methods available in the recent literature.     

Semiparametric Efficient Estimation of the Mean of a Time Series in the Presence of Conditional Heterogeneity of Unknown Form

We obtain semiparametric efficiency bounds for estimation of a location parameter in a time series model where the innovations are stationary and ergodic conditionally symmetric martingale differences but otherwise possess general dependence and distributions of unknown form. We then describe an iterative estimator that achieves this bound when the conditional density functions of the sample are known. Finally, we develop a “semi-adaptive” estimator that achieves the bound when these densities are unknown by the investigator. This estimator employs nonparametric kernel estimates of the densities. Monte Carlo results are reported.     

Semiparametric Efficient Estimation of the Mean of a Time Series in the Presence of Conditional Heterogeneity of Unknown Form

Abstract

We obtain semiparametric efficiency bounds for estimation of a location parameter in a time series model where the innovations are stationary and ergodic conditionally symmetric martingale differences but otherwise possess general dependence and distributions of unknown form. We then describe an iterative estimator that achieves this bound when the conditional density functions of the sample are known. Finally, we develop a “semi-adaptive” estimator that achieves the bound when these densities are unknown by the investigator. This estimator employs nonparametric kernel estimates of the densities. Monte Carlo results are reported.     

for misspecified regression models

The authors propose minimax robust designs for regression models whose response function is possibly misspecified. These designs, which minimize the maximum of the mean squared error matrix, can control the bias caused by model misspecification and provide the desired efficiency through one parameter. The authors call on a nonsmooth optimization technique to derive these designs analytically. Their results extend those of Heo, Schmuland & Wiens (2001). The authors also discuss several examples for approximately polynomial regression.     

Admissible minimax estimators for the shape parameter of Topp–Leone distribution

Abstract

The shape parameter of Topp–Leone distribution is estimated in this article from the Bayesian viewpoint under the assumption of known scale parameter. Bayes and empirical Bayes estimates of the unknown parameter are proposed under non informative and suitable conjugate priors. These estimates are derived under the assumption of squared and linear-exponential error loss functions. The risk functions of the proposed estimates are derived in analytical forms. It is shown that the proposed estimates are minimax and admissible. The consistency of the proposed estimates under the squared error loss function is also proved. Numerical examples are provided.     

Remember the curse of dimensionality: the case of goodness-of-fit testing in arbitrary dimension

Despite a substantial literature on nonparametric two-sample goodness-of-fit testing in arbitrary dimensions, there is no mention there of any curse of dimensionality. In fact, in some publications, a parametric rate is derived. As we discuss below, this is because a directional alternative is considered. Indeed, even in dimension one, Ingster, Y. I. [(1987). Minimax testing of nonparametric hypotheses on a distribution density in the l_p metrics. Theory of Probability & Its Applications, 31(2), 333–337] has shown that the minimax rate is not parametric. In this paper, we extend his results to arbitrary dimension and confirm that the minimax rate is not only nonparametric, exhibits but also a prototypical curse of dimensionality. We further extend Ingster's work to show that the chi-squared test achieves the minimax rate. Moreover, we show that the test adapts to the intrinsic dimensionality of the data. Finally, in the spirit of Ingster, Y. I. [(2000). Adaptive chi-square tests. Journal of Mathematical Sciences, 99(2), 1110–1119], we consider a multiscale version of the chi-square test, showing that one can adapt to unknown smoothness without much loss in power.     

Full Bayesian wavelet inference with a nonparametric prior

In this paper, we introduce a new Bayesian nonparametric model for estimating an unknown function in the presence of Gaussian noise. The proposed model involves a mixture of a point mass and an arbitrary (nonparametric) symmetric and unimodal distribution for modeling wavelet coefficients. Posterior simulation uses slice sampling ideas and the consistency under the proposed model is discussed. In particular, the method is shown to be computationally competitive with some of best Empirical wavelet estimation methods.     

Minimax Designs for the Stability of Slope Estimation on Second-order Response Surfaces

In this paper, designs for the stability of the slope estimation on a second-order response surface are considered. Minimization of the point dispersion measure, which is maximized over all points in the region of interest is taken as the optimality criterion, and the minimax properties in some class of designs are derived in spherical and cubic regions of interest. We study the efficiencies of the minimax designs relative to other optimal designs with various criteria.     

Switching nonparametric regression models

We propose a methodology to analyse data arising from a curve that, over its domain, switches among J states. We consider a sequence of response variables, where each response y depends on a covariate x according to an unobserved state z. The states form a stochastic process and their possible values are j=1,?…?, J. If z equals j the expected response of y is one of J unknown smooth functions evaluated at x. We call this model a switching nonparametric regression model. We develop an Expectation–Maximisation algorithm to estimate the parameters of the latent state process and the functions corresponding to the J states. We also obtain standard errors for the parameter estimates of the state process. We conduct simulation studies to analyse the frequentist properties of our estimates. We also apply the proposed methodology to the well-known motorcycle dataset treating the data as coming from more than one simulated accident run with unobserved run labels.