A relaxation procedure for calculating (G–)minimax optimal designs

Summary: In nonlinear statistical models, standard optimality functions for experimental designs depend on the unknown parameters of the model. An appealing and robust concept for choosing a design is the minimax criterion. However, so far, minimax optimal designs have been calculated efficiently under various restrictive conditions only. We extend an iterative relaxation scheme originally proposed by Shimizu and Aiyoshi (1980) and prove its convergence under very general assumptions which cover a variety of situations considered in experimental design. Application to different specific design criteria is discussed and issues of practical implementation are addressed. First numerical results suggest that the method may be very efficient with respect to the number of iterations required.*Supported by a grant from the Deutsche Forschungsgemeinschaft. We are grateful to a referee for his constructive suggestions.     

Nonlinear mixed‐effects models with misspecified random‐effects distribution

Nonlinear mixed‐effects models are being widely used for the analysis of longitudinal data, especially from pharmaceutical research. They use random effects which are latent and unobservable variables so the random‐effects distribution is subject to misspecification in practice. In this paper, we first study the consequences of misspecifying the random‐effects distribution in nonlinear mixed‐effects models. Our study is focused on Gauss‐Hermite quadrature, which is now the routine method for calculation of the marginal likelihood in mixed models. We then present a formal diagnostic test to check the appropriateness of the assumed random‐effects distribution in nonlinear mixed‐effects models, which is very useful for real data analysis. Our findings show that the estimates of fixed‐effects parameters in nonlinear mixed‐effects models are generally robust to deviations from normality of the random‐effects distribution, but the estimates of variance components are very sensitive to the distributional assumption of random effects. Furthermore, a misspecified random‐effects distribution will either overestimate or underestimate the predictions of random effects. We illustrate the results using a real data application from an intensive pharmacokinetic study.     

A generalized Bayesian nonlinear mixed‐effects regression model for zero‐inflated longitudinal count data in tuberculosis trials

In this paper, we investigate Bayesian generalized nonlinear mixed‐effects (NLME) regression models for zero‐inflated longitudinal count data. The methodology is motivated by and applied to colony forming unit (CFU) counts in extended bactericidal activity tuberculosis (TB) trials. Furthermore, for model comparisons, we present a generalized method for calculating the marginal likelihoods required to determine Bayes factors. A simulation study shows that the proposed zero‐inflated negative binomial regression model has good accuracy, precision, and credibility interval coverage. In contrast, conventional normal NLME regression models applied to log‐transformed count data, which handle zero counts as left censored values, may yield credibility intervals that undercover the true bactericidal activity of anti‐TB drugs. We therefore recommend that zero‐inflated NLME regression models should be fitted to CFU count on the original scale, as an alternative to conventional normal NLME regression models on the logarithmic scale.     

Optimal tests in AR (m) time series model

In the univariate framework, two problems of testing the nonlinearity are investigated in Hwang and Basawa. The first one is concerned with the testing problem for a nonlinear class contiguous to the AR(1) process. The second one is focused on the testing problem of the ARCH model contiguous to the AR(1) models. In each case, an efficient test of linearity was obtained, the local asymptotic normality (LAN) was proved, an efficient test of linearity was constructed, and the asymptotic power function was derived. All these results were obtained under the assumption where the parameter of the time series model is assumed to be known. In practical situation, this parameter is unspecified and its estimation induces an error that has an impact on the asymptotic limit distribution. A new method for the good evaluation of this error is introduced and investigated in the present article. Consequently, its application allows us to preserve the local asymptotic optimality with the estimated parameter. An extension to testing in class of ARCH models contiguous to p-order autoregressive processes is obtained. The LAN property plays a fundamental role in the present study.     

Sample‐size calculations for multi‐group comparison in population pharmacokinetic experiments

This paper describes an approach for calculating sample size for population pharmacokinetic experiments that involve hypothesis testing based on multi‐group comparison detecting the difference in parameters between groups under mixed‐effects modelling. This approach extends what has been described for generalized linear models and nonlinear population pharmacokinetic models that involve only binary covariates to more complex nonlinear population pharmacokinetic models. The structural nonlinear model is linearized around the random effects to obtain the marginal model and the hypothesis testing involving model parameters is based on Wald's test. This approach provides an efficient and fast method for calculating sample size for hypothesis testing in population pharmacokinetic models. The approach can also handle different design problems such as unequal allocation of subjects to groups and unbalanced sampling times between and within groups. The results obtained following application to a one compartment intravenous bolus dose model that involved three different hypotheses under different scenarios showed good agreement between the power obtained from NONMEM simulations and nominal power. Copyright © 2009 John Wiley & Sons, Ltd.     

CVX‐based algorithms for constructing various optimal regression designs

CVX‐based numerical algorithms are widely and freely available for solving convex optimization problems but their applications to solve optimal design problems are limited. Using the CVX programs in MATLAB, we demonstrate their utility and flexibility over traditional algorithms in statistics for finding different types of optimal approximate designs under a convex criterion for nonlinear models. They are generally fast and easy to implement for any model and any convex optimality criterion. We derive theoretical properties of the algorithms and use them to generate new A‐, c‐, D‐ and E‐optimal designs for various nonlinear models, including multi‐stage and multi‐objective optimal designs. We report properties of the optimal designs and provide sample CVX program codes for some of our examples that users can amend to find tailored optimal designs for their problems. The Canadian Journal of Statistics 47: 374–391; 2019 © 2019 Statistical Society of Canada     

Bias‐reduced marginal Akaike information criteria based on a Monte Carlo method for linear mixed‐effects models

In linear mixed‐effects (LME) models, if a fitted model has more random‐effect terms than the true model, a regularity condition required in the asymptotic theory may not hold. In such cases, the marginal Akaike information criterion (AIC) is positively biased for (?2) times the expected log‐likelihood. The asymptotic bias of the maximum log‐likelihood as an estimator of the expected log‐likelihood is evaluated for LME models with balanced design in the context of parameter‐constrained models. Moreover, bias‐reduced marginal AICs for LME models based on a Monte Carlo method are proposed. The performance of the proposed criteria is compared with existing criteria by using example data and by a simulation study. It was found that the bias of the proposed criteria was smaller than that of the existing marginal AIC when a larger model was fitted and that the probability of choosing a smaller model incorrectly was decreased.     

Estimation for High‐Dimensional Linear Mixed‐Effects Models Using ℓ1‐Penalization

Abstract. We propose an ?₁‐penalized estimation procedure for high‐dimensional linear mixed‐effects models. The models are useful whenever there is a grouping structure among high‐dimensional observations, that is, for clustered data. We prove a consistency and an oracle optimality result and we develop an algorithm with provable numerical convergence. Furthermore, we demonstrate the performance of the method on simulated and a real high‐dimensional data set.     

Response surface design evaluation and comparison

Designing an experiment to fit a response surface model typically involves selecting among several candidate designs. There are often many competing criteria that could be considered in selecting the design, and practitioners are typically forced to make trade-offs between these objectives when choosing the final design. Traditional alphabetic optimality criteria are often used in evaluating and comparing competing designs. These optimality criteria are single-number summaries for quality properties of the design such as the precision with which the model parameters are estimated or the uncertainty associated with prediction. Other important considerations include the robustness of the design to model misspecification and potential problems arising from spurious or missing data. Several qualitative and quantitative properties of good response surface designs are discussed, and some of their important trade-offs are considered. Graphical methods for evaluating design performance for several important response surface problems are discussed and we show how these techniques can be used to compare competing designs. These graphical methods are generally superior to the simplistic summaries of alphabetic optimality criteria. Several special cases are considered, including robust parameter designs, split-plot designs, mixture experiment designs, and designs for generalized linear models.     

Optimal design for the proportional odds model

The authors construct locally optimal designs for the proportional odds model for ordinal data. While they investigate the standard D‐optimal design, they also investigate optimality criteria for the simultaneous estimation of multiple quantiles, namely D_A ‐optimality and the omnibus criterion. The design of experiments for the simultaneous estimation of multiple quantiles is important in both toxic and effective dose studies in medicine. As with c‐optimality in the binary response problem, the authors find that there are distinct phase changes when exploring extreme quantiles that require additional design points. The authors also investigate relative efficiencies of the criteria.     

Selection Strategy for Covariance Structure of Random Effects in Linear Mixed‐effects Models

Linear mixed‐effects models are a powerful tool for modelling longitudinal data and are widely used in practice. For a given set of covariates in a linear mixed‐effects model, selecting the covariance structure of random effects is an important problem. In this paper, we develop a joint likelihood‐based selection criterion. Our criterion is the approximately unbiased estimator of the expected Kullback–Leibler information. This criterion is also asymptotically optimal in the sense that for large samples, estimates based on the covariance matrix selected by the criterion minimize the approximate Kullback–Leibler information. Finite sample performance of the proposed method is assessed by simulation experiments. As an illustration, the criterion is applied to a data set from an AIDS clinical trial.     

Kernel‐based Generalized Cross‐validation in Non‐parametric Mixed‐effect Models

Abstract. Although generalized cross‐validation (GCV) has been frequently applied to select bandwidth when kernel methods are used to estimate non‐parametric mixed‐effect models in which non‐parametric mean functions are used to model covariate effects, and additive random effects are applied to account for overdispersion and correlation, the optimality of the GCV has not yet been explored. In this article, we construct a kernel estimator of the non‐parametric mean function. An equivalence between the kernel estimator and a weighted least square type estimator is provided, and the optimality of the GCV‐based bandwidth is investigated. The theoretical derivations also show that kernel‐based and spline‐based GCV give very similar asymptotic results. This provides us with a solid base to use kernel estimation for mixed‐effect models. Simulation studies are undertaken to investigate the empirical performance of the GCV. A real data example is analysed for illustration.     

Iterative Bias Correction of the Cross‐Validation Criterion

Abstract. The cross‐validation (CV) criterion is known to be asecond‐order unbiased estimator of the risk function measuring the discrepancy between the candidate model and the true model, as well as the generalized information criterion (GIC) and the extended information criterion (EIC). In the present article, we show that the 2kth‐order unbiased estimator can be obtained using a linear combination from the leave‐one‐out CV criterion to the leave‐k‐out CV criterion. The proposed scheme is unique in that a bias smaller than that of a jackknife method can be obtained without any analytic calculation, that is, it is not necessary to obtain the explicit form of several terms in an asymptotic expansion of the bias. Furthermore, the proposed criterion can be regarded as a finite correction of a bias‐corrected CV criterion by using scalar coefficients in a bias‐corrected EIC obtained by the bootstrap iteration.     

Circular balanced repeated measurements designs

The concept of a circular design is defined and when proper balance for various effects is assumed, its universal optimality is proved over the class of all designs with the same set of parameters, Such designs are shown to minimize the variance of the best linear unbiased estimators of contrasts of residual and direct effects over the class of equireplicated designs. All models assume first order residual effects and are of a circular nature. The proofs are presented in a unified manner for several models at a time. They are based on certain matrix domination which occurs when parameters are eliminated from a linear modelj this latter fact is proved for a general linear model.     

A likelihood ratio test for goodness‐of‐fit of recessive and dominant models for case–control studies

Testing goodness‐of‐fit of commonly used genetic models is of critical importance in many applications including association studies and testing for departure from Hardy–Weinberg equilibrium. Case–control design has become widely used in population genetics and genetic epidemiology, thus it is of interest to develop powerful goodness‐of‐fit tests for genetic models using case–control data. This paper develops a likelihood ratio test (LRT) for testing recessive and dominant models for case–control studies. The LRT statistic has a closed‐form formula with a simple $\chi^{2}(1)$ null asymptotic distribution, thus its implementation is easy even for genome‐wide association studies. Moreover, it has the same power and optimality as when the disease prevalence is known in the population. The Canadian Journal of Statistics 41: 341–352; 2013 © 2013 Statistical Society of Canada     

Uniformity of Node Level Conflict Measures in Bayesian Hierarchical Models Based on Directed Acyclic Graphs

Hierarchical models defined by means of directed, acyclic graphs are a powerful and widely used tool for Bayesian analysis of problems of varying degrees of complexity. A simulation‐based method for model criticism in such models has been suggested by O'Hagan in the form of a conflict measure based on contrasting separate local information sources about each node in the graph. This measure is however not well calibrated. In order to rectify this, alternative mutually similar tail probability‐based measures have been proposed independently and have been proved to be uniformly distributed under the assumed model in quite general normal models with known covariance matrices. In the present paper, we extend this result to a variety of models. An advantage of this is that computationally costly pre‐calibration schemes needed for some other suggested methods can be avoided. Another advantage is that non‐informative prior distributions can be used when performing model criticism.     

Marginally restricted sequential D‐optimal designs

In many experiments, not all explanatory variables can be controlled. When the units arise sequentially, different approaches may be used. The authors study a natural sequential procedure for “marginally restricted” D‐optimal designs. They assume that one set of explanatory variables (x₁) is observed sequentially, and that the experimenter responds by choosing an appropriate value of the explanatory variable x₂. In order to solve the sequential problem a priori, the authors consider the problem of constructing optimal designs with a prior marginal distribution for x₁. This eliminates the influence of units already observed on the next unit to be designed. They give explicit designs for various cases in which the mean response follows a linear regression model; they also consider a case study with a nonlinear logistic response. They find that the optimal strategy often consists of randomizing the assignment of the values of x₂.     

A Semi‐parametric Transformation Frailty Model for Semi‐competing Risks Survival Data

In the analysis of semi‐competing risks data interest lies in estimation and inference with respect to a so‐called non‐terminal event, the observation of which is subject to a terminal event. Multi‐state models are commonly used to analyse such data, with covariate effects on the transition/intensity functions typically specified via the Cox model and dependence between the non‐terminal and terminal events specified, in part, by a unit‐specific shared frailty term. To ensure identifiability, the frailties are typically assumed to arise from a parametric distribution, specifically a Gamma distribution with mean 1.0 and variance, say, σ². When the frailty distribution is misspecified, however, the resulting estimator is not guaranteed to be consistent, with the extent of asymptotic bias depending on the discrepancy between the assumed and true frailty distributions. In this paper, we propose a novel class of transformation models for semi‐competing risks analysis that permit the non‐parametric specification of the frailty distribution. To ensure identifiability, the class restricts to parametric specifications of the transformation and the error distribution; the latter are flexible, however, and cover a broad range of possible specifications. We also derive the semi‐parametric efficient score under the complete data setting and propose a non‐parametric score imputation method to handle right censoring; consistency and asymptotic normality of the resulting estimators is derived and small‐sample operating characteristics evaluated via simulation. Although the proposed semi‐parametric transformation model and non‐parametric score imputation method are motivated by the analysis of semi‐competing risks data, they are broadly applicable to any analysis of multivariate time‐to‐event outcomes in which a unit‐specific shared frailty is used to account for correlation. Finally, the proposed model and estimation procedures are applied to a study of hospital readmission among patients diagnosed with pancreatic cancer.     

Maximum likelihood mean and covariance matrix estimation constrained to general positive semi-definiteness

Maximum likelihood estimation of a mean and a covariance matrix whose structure is constrained only to general positive semi-definiteness is treated in this paper. Necessary and sufficient conditions for the local optimality of mean and covariance matrix estimates are given. Observations are assumed to be independent. When the observations are also assumed to be identically distributed, the optimality conditions are used to obtain the mean and covariance matrix solutions in closed form. For the nonidentically distributed observation case, a general numerical technique which integrates scoring and Newton's iterations to solve the optimality condition equations is presented, and convergence performance is examined.     

Robust Estimation for Zero‐Inflated Poisson Regression

Abstract. The zero‐inflated Poisson regression model is a special case of finite mixture models that is useful for count data containing many zeros. Typically, maximum likelihood (ML) estimation is used for fitting such models. However, it is well known that the ML estimator is highly sensitive to the presence of outliers and can become unstable when mixture components are poorly separated. In this paper, we propose an alternative robust estimation approach, robust expectation‐solution (RES) estimation. We compare the RES approach with an existing robust approach, minimum Hellinger distance (MHD) estimation. Simulation results indicate that both methods improve on ML when outliers are present and/or when the mixture components are poorly separated. However, the RES approach is more efficient in all the scenarios we considered. In addition, the RES method is shown to yield consistent and asymptotically normal estimators and, in contrast to MHD, can be applied quite generally.