On the Assessment of Monte Carlo Error in Simulation-Based Statistical Analyses

Statistical experiments, more commonly referred to as Monte Carlo or simulation studies, are used to study the behavior of statistical methods and measures under controlled situations. Whereas recent computing and methodological advances have permitted increased efficiency in the simulation process, known as variance reduction, such experiments remain limited by their finite nature and hence are subject to uncertainty; when a simulation is run more than once, different results are obtained. However, virtually no emphasis has been placed on reporting the uncertainty, referred to here as Monte Carlo error, associated with simulation results in the published literature, or on justifying the number of replications used. These deserve broader consideration. Here we present a series of simple and practical methods for estimating Monte Carlo error as well as determining the number of replications required to achieve a desired level of accuracy. The issues and methods are demonstrated with two simple examples, one evaluating operating characteristics of the maximum likelihood estimator for the parameters in logistic regression and the other in the context of using the bootstrap to obtain 95% confidence intervals. The results suggest that in many settings, Monte Carlo error may be more substantial than traditionally thought.     

Parametric non-mixture cure models for schedule finding of therapeutic agents

We propose a phase I clinical trial design that seeks to determine the cumulative safety of a series of administrations of a fixed dose of an investigational agent. In contrast with traditional phase I trials that are designed solely to find the maximum tolerated dose of the agent, our design instead identifies a maximum tolerated schedule that includes a maximum tolerated dose as well as a vector of recommended administration times. Our model is based on a non-mixture cure model that constrains the probability of dose limiting toxicity for all patients to increase monotonically with both dose and the number of administrations received. We assume a specific parametric hazard function for each administration and compute the total hazard of dose limiting toxicity for a schedule as a sum of individual administration hazards. Throughout a variety of settings motivated by an actual study in allogeneic bone marrow transplant recipients, we demonstrate that our approach has excellent operating characteristics and performs as well as the only other currently published design for schedule finding studies. We also present arguments for the preference of our non-mixture cure model over the existing model.     

Reports of Conferences,Institutes, and Seminars

This installment of "Serials Spoken Here" covers events that transpired between late September and late October 2008. Reported herein are two Webinars, one on ONIX for Serials, the other on SUSHI, and two conferences: the eighty-fourth annual Meeting of the Potomac Technical Processing Librarians and the New England Library Association's Annual Conference.     

The academic attitudes of American teenagers, 1990–2002: Cohort and gender effects on math achievement

Using data from the National Education Longitudinal Study of 1988 and the Education Longitudinal Study of 2002, I compare the academic attitudes of high school students from Generation X and the Millennial Generation. I then analyze the effects these attitudes have on mathematics achievement test scores. Compared to the earlier group, students in the later cohort were less likely to indicate academic reasons for attending school and largely perceived their friends as being less engaged academically. Students in both cohorts whose friends held academic values experienced an increase in math scores. The more students in the earlier cohort disagreed that they came to school for academic reasons, the more their math scores decreased; this relationship did not appear for the later cohort. Females in each cohort showed stronger academic attitudes than males; additionally, believing that popularity was important was associated with lower test scores for females, but not males, in the Millennial cohort.     

A robust QTL mapping procedure

In quantitative trait linkage studies using experimental crosses, the conventional normal location-shift model or other parameterizations may be unnecessarily restrictive. We generalize the mapping problem to a genuine nonparametric setup and provide a robust estimation procedure for the situation where the underlying phenotype distributions are completely unspecified. Classical Wilcoxon–Mann–Whitney statistics are employed for point and interval estimation of QTL positions and effects.     

Order statistics from trivariate normal and -distributions in terms of generalized skew-normal and skew- distributions

We consider here a generalization of the skew-normal distribution, GSN(λ₁,λ₂,ρ), defined through a standard bivariate normal distribution with correlation ρ, which is a special case of the unified multivariate skew-normal distribution studied recently by Arellano-Valle and Azzalini [2006. On the unification of families of skew-normal distributions. Scand. J. Statist. 33, 561–574]. We then present some simple and useful properties of this distribution and also derive its moment generating function in an explicit form. Next, we show that distributions of order statistics from the trivariate normal distribution are mixtures of these generalized skew-normal distributions; thence, using the established properties of the generalized skew-normal distribution, we derive the moment generating functions of order statistics, and also present expressions for means and variances of these order statistics.Next, we introduce a generalized skew-t_ν distribution, which is a special case of the unified multivariate skew-elliptical distribution presented by Arellano-Valle and Azzalini [2006. On the unification of families of skew-normal distributions. Scand. J. Statist. 33, 561–574] and is in fact a three-parameter generalization of Azzalini and Capitanio's [2003. Distributions generated by perturbation of symmetry with emphasis on a multivariate skew t distribution. J. Roy. Statist. Soc. Ser. B 65, 367–389] univariate skew-t_ν form. We then use the relationship between the generalized skew-normal and skew-t_ν distributions to discuss some properties of generalized skew-t_ν as well as distributions of order statistics from bivariate and trivariate t_ν distributions. We show that these distributions of order statistics are indeed mixtures of generalized skew-t_ν distributions, and then use this property to derive explicit expressions for means and variances of these order statistics.     

Hypotheses testing: Poisson versus stress-release

We consider the problem of hypotheses testing with the basic simple hypothesis: observed sequence of points corresponds to stationary Poisson process with known intensity against a composite one-sided parametric alternative that this is a stress-release point process. The underlying family of measures is locally asymptotically quadratic and we describe the behavior of score-function, likelihood ratio and Wald tests in the asymptotics of large samples. The results of numerical simulations are presented.     

Nonparametric density deconvolution by weighted kernel estimators

Nonparametric density estimation in the presence of measurement error is considered. The usual kernel deconvolution estimator seeks to account for the contamination in the data by employing a modified kernel. In this paper a new approach based on a weighted kernel density estimator is proposed. Theoretical motivation is provided by the existence of a weight vector that perfectly counteracts the bias in density estimation without generating an excessive increase in variance. In practice a data driven method of weight selection is required. Our strategy is to minimize the discrepancy between a standard kernel estimate from the contaminated data on the one hand, and the convolution of the weighted deconvolution estimate with the measurement error density on the other hand. We consider a direct implementation of this approach, in which the weights are optimized subject to sum and non-negativity constraints, and a regularized version in which the objective function includes a ridge-type penalty. Numerical tests suggest that the weighted kernel estimation can lead to tangible improvements in performance over the usual kernel deconvolution estimator. Furthermore, weighted kernel estimates are free from the problem of negative estimation in the tails that can occur when using modified kernels. The weighted kernel approach generalizes to the case of multivariate deconvolution density estimation in a very straightforward manner.     

Automating and evaluating reversible jump MCMC proposal distributions

The reversible jump Markov chain Monte Carlo (MCMC) sampler (Green in Biometrika 82:711–732, 1995) has become an invaluable device for Bayesian practitioners. However, the primary difficulty with the sampler lies with the efficient construction of transitions between competing models of possibly differing dimensionality and interpretation. We propose the use of a marginal density estimator to construct between-model proposal distributions. This provides both a step towards black-box simulation for reversible jump samplers, and a tool to examine the utility of common between-model mapping strategies. We compare the performance of our approach to well established alternatives in both time series and mixture model examples.     

A counterexample to Beder's conjectures about Hadamard matrices

In this note we provide a counterexample which resolves conjectures about Hadamard matrices made in this journal. Beder [1998. Conjectures about Hadamard matrices. Journal of Statistical Planning and Inference 72, 7–14] conjectured that if H$\mathit{H}$ is a maximal m×n$m\times n$ row-Hadamard matrix then m is a multiple of 4; and that if n   is a power of 2 then every row-Hadamard matrix can be extended to a Hadamard matrix. Using binary integer programming we obtain a maximal 13×32$13\times 32$ row-Hadamard matrix, which disproves both conjectures. Additionally for n being a multiple of 4 up to 64, we tabulate values of m   for which we have found a maximal row-Hadamard matrix. Based on the tabulated results we conjecture that a m×n$m\times n$ row-Hadamard matrix with m?n-7$m?n-7$ can be extended to a Hadamard matrix.