Cross-Validated Assessments of Test Scoring Models

On a multiple choice test in which each item has r alternative options, a given number c of which are correct, various scoring models have been proposed. In one case the test-taker is allowed to choose any size solution subset and he/she is graded according to whether the subset is small and according to how many correct answers the subset contains. In a second case the test-taker is allowed to select only solution subsets of a prespecified maximum size and is graded as above. The first case is analogous to the situation where the test-taker is given a set of r options with each question; each question calls for a solution which consists of selecting that subset of the r responses which he/she believes to be correct. In the second case, when the prespecified solution subset is restricted to be of size at most one, the resulting scoring model corresponds to the usual model, referred to below as standard. The number c of correct options per item is usually known to the test-taker in this case.

Scoring models are evaluated according to how well they correctly identify the total scores of the individuals in the class of test-takers. Loss functions are constructed which penalize scoring models resulting in student scores which are not associated with the students true (or average) total score on the exam. Scoring models are compared on the basis of cross-validated assessments of the loss incurred by using each of the given models. It is shown that in many cases the assessment of the loss for scoring models which allow students the opportunity to choose more than one option for each question are smaller than the assessment of the loss for the standard scoring model.     

A note on second order slope rotatable designs over all directions

Hader and Park (1978) introduced second order slope rotatability in axial directions. Park (1987) introduced second order slope rotatabilty over all directions. It is shown that these designs have the additional properly that the sum of the variances of estimates of slopes in all directions at any point is a function of the distance of the point from the design origin.     

Estimation in an empirical bayes model for longitudinal and cross‐sectionally clustered binary data

Some studies generate data that can be grouped into clusters in more than one way. Consider for instance a smoking prevention study in which responses on smoking status are collected over several years in a cohort of students from a number of different schools. This yields longitudinal data, also cross‐sectionaliy clustered in schools. The authors present a model for analyzing binary data of this type, combining generalized estimating equations and estimation of random effects to address the longitudinal and cross‐sectional dependence, respectively. The estimation procedure for this model is discussed, as are the results of a simulation study used to investigate the properties of its estimates. An illustration using data from a smoking prevention trial is given.     

Correlation in a Bayesian framework

The authors consider the correlation between two arbitrary functions of the data and a parameter when the parameter is regarded as a random variable with given prior distribution. They show how to compute such a correlation and use closed form expressions to assess the dependence between parameters and various classical or robust estimators thereof, as well as between p‐values and posterior probabilities of the null hypothesis in the one‐sided testing problem. Other applications involve the Dirichlet process and stationary Gaussian processes. Using this approach, the authors also derive a general nonparametric upper bound on Bayes risks.     

Confidence curves and improved exact confidence intervals for discrete distributions

The author describes a method for improving standard “exact” confidence intervals in discrete distributions with respect to size while retaining correct level. The binomial, negative binomial, hypergeometric, and Poisson distributions are considered explicitly. Contrary to other existing methods, the author's solution possesses a natural nesting condition: if α < α', the 1 ‐ α' confidence interval is included in the 1 ‐ α interval. Nonparametric confidence intervals for a quantile are also considered.     

Tests of symmetry based on transformed empirical processes

A probability distribution function F is said to be symmetric when 1 ‐ F(x) ‐ F(‐x) = 0 for all x∈ R. Given a sequence of alternatives contiguous to a certain symmetric F₀, the authors are concerned with testing for the null hypothesis of symmetry. The proposed tests are consistent against any nonsymmetric alternative, and their power with respect to the given sequence can easily be optimized. The tests are constructed by means of transformed empirical processes with an adequate selection of the underlying isometry, and the optimum power is obtained by suitably choosing the score functions. The test statistics are very easy to compute and their asymptotic distributions are simple.     

Tolerance‐based error reduction in multi‐agent bioassays

Some degree of error is inevitable in multi‐agent bioassays regardless of design or measurement technology. Estimation error can be reduced post facto by exploiting the matrix partial ordering of the bioassay survival estimates. The standard method for this is order‐restricted regression (ORR). If the joint action of the bioassay agents admits a tolerance‐based interpretation, additional structure beyond matrix partial ordering is available, leading to a new method of error reduction. This tolerance‐based error reduction (TBER) procedure almost always outperforms ORR. Like ORR, TBER applies to complete factorial bioassay designs and, using weighting, to incomplete designs.     

Geometric ergodicity of nonlinear autoregressive models with changing conditional variances

The authors give easy‐to‐check sufficient conditions for the geometric ergodicity and the finiteness of the moments of a random process x_t = ?(x_t‐1,…, x_t‐p) + ?_tσ(x_t‐1,…, x_t‐q) in which ?: R^p → R, σ R^q → R and (?_t) is a sequence of independent and identically distributed random variables. They deduce strong mixing properties for this class of nonlinear autoregressive models with changing conditional variances which includes, among others, the ARCH(p), the AR(p)‐ARCH(p), and the double‐threshold autoregressive models.     

Differential equation models for statistical functions

Differential equations have been used in statistics to define functions such as probability densities. But the idea of using differential equation formulations of stochastic models has a much wider scope. The author gives several examples, including simultaneous estimation of a regression model and residual density, monotone smoothing, specification of a link function, differential equation models of data, and smoothing over complicated multidimensional domains. This paper aims to stimulate interest in this approach to functional estimation problems, rather than provide carefully worked out methods.     

Bayesian prediction of waiting times in stochastic models

The authors show how saddlepoint techniques lead to highly accurate approximations for Bayesian predictive densities and cumulative distribution functions in stochastic model settings where the prior is tractable, but not necessarily the likelihood or the predictand distribution. They consider more specifically models involving predictions associated with waiting times for semi‐Markov processes whose distributions are indexed by an unknown parameter θ. Bayesian prediction for such processes when they are not stationary is also addressed and the inverse‐Gaussian based saddlepoint approximation of Wood, Booth & Butler (1993) is shown to accurately deal with the nonstationarity whereas the normal‐based Lugannani & Rice (1980) approximation cannot, Their methods are illustrated by predicting various waiting times associated with M/M/q and M/G/1 queues. They also discuss modifications to the matrix renewal theory needed for computing the moment generating functions that are used in the saddlepoint methods.