Designs for approximately linear regression: Maximizing the minimum coverage probability of confidence ellipsoids

The classical D-optimality principle in regression design may be motivated by a desire to maximize the coverage probability of a fixed-volume confidence ellipsoid on the regression parameters. When the fitted model is exactly correct, this amounts to minimizing the determinant of the covariance matrix of the estimators. We consider an analogue of this problem, under the approximately linear model E[y|x] = θ^Tz(x) + f(x). The nonlinear disturbance f(x) is essentially unknown, and the experimenter fits only to the linear part of the response. The resulting bias affects the coverage probability of the confidence ellipsoid on θ. We study the construction of designs which maximize the minimum coverage probability as f varies over a certain class. Explicit designs are given in the case that the fitted response surface is a plane.     

Linear model estimation errors with estimated design parameters

The problem of error estimation of parameters b in a linear model,Y = Xb+ e, is considered when the elements of the design matrix X are functions of an unknown ‘design’ parameter vector c. An estimated value c is substituted in X to obtain a derived design matrix [Xtilde]. Even though the usual linear model conditions are not satisfied with [Xtilde], there are situations in physical applications where the least squares solution to the parameters is used without concern for the magnitude of the resulting error. Such a solution can suffer from serious errors.

This paper examines bias and covariance errors of such estimators. Using a first-order Taylor series expansion, we derive approximations to the bias and covariance matrix of the estimated parameters. The bias approximation is a sum of two terms:One is due to the dependence between ? and Y; the other is due to the estimation errors of ? and is proportional to b, the parameter being estimated. The covariance matrix approximation, on the other hand, is composed of three omponents:One component is due to the dependence between ? and Y; the second is the covariance matrix ∑_b corresponding to the minimum variance unbiased b, as if the design parameters were known without error; and the third is an additional component due to the errors in the design parameters. It is shown that the third error component is directly proportional to bb'. Thus, estimation of large parameters with wrong design matrix [Xtilde] will have larger errors of estimation. The results are illustrated with a simple linear example.     

Restricted maximum likelihood estimation of joint mean‐covariance models

The class of joint mean‐covariance models uses the modified Cholesky decomposition of the within subject covariance matrix in order to arrive to an unconstrained, statistically meaningful reparameterisation. The new parameterisation of the covariance matrix has two sets of parameters that separately describe the variances and correlations. Thus, with the mean or regression parameters, these models have three sets of distinct parameters. In order to alleviate the problem of inefficient estimation and downward bias in the variance estimates, inherent in the maximum likelihood estimation procedure, the usual REML estimation procedure adjusts for the degrees of freedom lost due to the estimation of the mean parameters. Because of the parameterisation of the joint mean covariance models, it is possible to adapt the usual REML procedure in order to estimate the variance (correlation) parameters by taking into account the degrees of freedom lost by the estimation of both the mean and correlation (variance) parameters. To this end, here we propose adjustments to the estimation procedures based on the modified and adjusted profile likelihoods. The methods are illustrated by an application to a real data set and simulation studies. The Canadian Journal of Statistics 40: 225–242; 2012 © 2012 Statistical Society of Canada     

Robust sequential designs for approximately linear models

We consider the problem of the sequential choice of design points in an approximately linear model. It is assumed that the fitted linear model is only approximately correct, in that the true response function contains a nonrandom, unknown term orthogonal to the fitted response. We also assume that the parameters are estimated by M-estimation. The goal is to choose the next design point in such a way as to minimize the resulting integrated squared bias of the estimated response, to order n^-1. Explicit applications to analysis of variance and regression are given. In a simulation study the sequential designs compare favourably with some fixed-sample-size designs which are optimal for the true response to which the sequential designs must adapt.     

The Effect of Estimation of Unknown Degrees of Freedom on Estimation of Remaining Parameters in Spanos’ Conditional t Heteroskedastic Model

This article investigates the effect of estimation of unknown degrees of freedom on efficient estimation of remaining parameters in Spanos’ conditional t heteroskedastic model. We compare by simulation three maximum likelihood estimators (MLEs) of the remaining parameters in the model: the MLE of the remaining parameters when all the parameters are estimated by the MLE, when the degrees of freedom is estimated by a method of moments estimator, and when the degrees of freedom is erroneously specified. The latter two methods are found to perform poorly compared to the former method for the inference of variance parameters in the model. Thus, efficient estimation of degrees of freedom by the MLE is important to estimate efficiently the remaining variance parameters.     

Bayesian Estimation of Regression Coefficients Under Extended Balanced Loss Function

Appreciating the desirability of simultaneously using both the criteria of goodness of fitted model and clustering of estimates around true parameter values, an extended version of the balanced loss function is presented and the Bayesian estimation of regression coefficients is discussed. The thus obtained optimal estimator is then compared with the least squares estimator and posterior mean vector with respect to the criteria like posterior expected loss, Bayes risk, bias vector, mean squared error matrix and risk function.     

Geometry and Degrees of Freedom of Linearly Constrained Generalized Lasso

The least squares fit in a linear regression is always unique even when the design matrix has rank deficiency. In this paper, we extend this classic result to linearly constrained generalized lasso. It is shown that under a mild condition, the fit can be represented as a projection onto a polytope and, hence, is unique no matter whether design matrix X has full column rank or not. Furthermore, a formula for the degrees of freedom is derived to characterize the effective number of parameters. It directly yields an unbiased estimate of degrees of freedom, which can be incorporated in an information criterion for model selection.     

On hsu's model in regression analysis

The paper examplifies with Hsu’s model a general pattern as how to derive results of variance component estimation from well known results on mean estimation, as far as linear model theory is concerned. This ’ dispersion-mean-correspondence‘provides new and short proofs for various theorems from the literature, concerning unbiased invariant quadratic estimators with minimum BAYES risk or minimum variance. For pure variance component models, unbiased non-negative quadratic estimability is characterized in terms of the design matrices.     

Comparison of designs for multivariate generalized linear models

The purpose of this paper is to discuss response surface designs for multivariate generalized linear models (GLMs). Such models are considered whenever several response variables can be measured for each setting of a group of control variables, and the response variables are adequately represented by GLMs. The mean-squared error of prediction (MSEP) matrix is used to assess the quality of prediction associated with a given design. The MSEP incorporates both the prediction variance and the prediction bias, which results from using maximum likelihood estimates of the parameters of the fitted linear predictor. For a given design, quantiles of a scalar-valued function of the MSEP are obtained within a certain region of interest. The quantiles depend on the unknown parameters of the linear predictor. The dispersion of these quantiles over the space of the unknown parameters is determined and then depicted by the so-called quantile dispersion graphs. An application of the proposed methodology is presented using the special case of the bivariate binary distribution.     

Alternative moment approximations for berkson's minimum logit chi-squared estimator

Expressions are derived for the bias to order J^-1 , the variance to order J^-2 and the mean squared error to order J^-2 of Berkson's minimum logit chi-squared estimator where J is the number of distinct design points. These moment approximations are numerically compared to Monte Carlo estimates of the true moments and the moment approximations of Amemiya (1980) which are appropriate when the “average” number of observations per design point is large. They are used to compare the mean squared error of the minimum logit chi-squared estimator to that of the maximum likelihood estimator and to investigate the effect of bias on confidence intenrals constructed using the minimum logit chi-squared estimator.     

On an adjustment of degrees of freedom in the minimim mean squared error ertimator

In this paper, we consider an adjustment of degrees of freedom in the minimum mean squared error (MMSE) estimator, We derive the exact MSE of the adjusted MMSE (AMMSE) estimator, and compare the MSE of the AMMSE estimator with those of the Stein-(SR), positive-part Stein-rule (PSR) and MMSE estimators by numerical evaluations. It is shown that the adjustment of degrees of freedom is effective when the noncentrality parameter is close to zero, and the MSE performance of the MMSE estimator can be improved in the wide region of the noncentrality parameter by the adjustment, ft is also shown that the AMMSE estimator can have the smaller MSE than the PSR estimator in the wide region of the noncentrality parameter     

Bootstrap estimation of the variance of the error term in monotonic regression models

The variance of the error term in ordinary regression models and linear smoothers is usually estimated by adjusting the average squared residual for the trace of the smoothing matrix (the degrees of freedom of the predicted response). However, other types of variance estimators are needed when using monotonic regression (MR) models, which are particularly suitable for estimating response functions with pronounced thresholds. Here, we propose a simple bootstrap estimator to compensate for the over-fitting that occurs when MR models are estimated from empirical data. Furthermore, we show that, in the case of one or two predictors, the performance of this estimator can be enhanced by introducing adjustment factors that take into account the slope of the response function and characteristics of the distribution of the explanatory variables. Extensive simulations show that our estimators perform satisfactorily for a great variety of monotonic functions and error distributions.     

Measurement error in regression analysis

Consider the linear regression model Y = Xθ+ ε where Y denotes a vector of n observations on the dependent variable, X is a known matrix, θ is a vector of parameters to be estimated and e is a random vector of uncorrelated errors. If X'X is nearly singular, that is if the smallest characteristic root of X'X s small then a small perurbation in the elements of X, such as due to measurement errors, induces considerable variation in the least squares estimate of θ. In this paper we examine for the asymptotic case when n is large the effect of perturbation with regard to the bias and mean squared error of the estimate.     

Estimability and linearity in normal regression

In the normal regresion model, a relation between the notion of estimability of a distribution and the existence of a linear function of the observations with some desired property is studied. Several definitions of estimability are shown to be equivalent.     

Two-step calibration of design weights under two auxiliary variables in sample survey

Calibration on the available auxiliary variables is widely used to increase the precision of the estimates of parameters. Singh and Sedory [Two-step calibration of design weights in survey sampling. Commun Stat Theory Methods. 2016;45(12):3510–3523.] considered the problem of calibration of design weights under two-step for single auxiliary variable. For a given sample, design weights and calibrated weights are set proportional to each other, in the first step. While, in the second step, the value of proportionality constant is determined on the basis of objectives of individual investigator/user for, for example, to get minimum mean squared error or reduction of bias. In this paper, we have suggested to use two auxiliary variables for two-step calibration of the design weights and compared the results with single auxiliary variable for different sample sizes based on simulated and real-life data set. The simulated and real-life application results show that two-auxiliary variables based two-step calibration estimator outperforms the estimator under single auxiliary variable in terms of minimum mean squared error.     

Singular gauss-markov models

For the general linear model Y = X$sZ + e in which e has a singular dispersion matrix $sG²A, $sG > 0, where A is n x n and singular, Mitra [2] considers the problem of testing F$sZ, where F is a known q x q matrix and claims that the sum of squares (SS) due to hypothesis is not distributed (as a x² variate with degrees of freedom (d. f.) equal to the rank of F) independent of the SS due to error, when a generalized inverse of A is chosen as (A + X'X)^–. This claim does not hold if a pseudo-inverse of A is taken to be (A + X'X)⁺ where A⁺ denotes the unique Moore-Penrose inverse (MPI) of A.     

Generation of multivariate distributions by vertical density representation

Troutt (1991,1993) proposed the idea of the vertical density representation (VDR) based on Box-Millar method. Kotz, Fang and Liang (1997) provided a systematic study on the multivariate vertical density representation (MVDR). Suppose that we want to generate a random vector X[d]Rⁿthat has a density function ?(x). The key point of using the MVDR is to generate the uniform distribution on [D]_?(v) = {x :?(x) = v} for any v > 0 which is the surface in RⁿIn this paper we use the conditional distribution method to generate the uniform distribution on a domain or on some surface and based on it we proposed an alternative version of the MVDR(type 2 MVDR), by which one can transfer the problem of generating a random vector X with given density f to one of generating (X, X_n+i) that follows the uniform distribution on a region in Rⁿ⁺¹defined by ?. Several examples indicate that the proposed method is quite practical.     

Expected loss functions as additional measures to assess performance of multiple testing procedures for combination drug dose finding

There are several measures that are commonly used to assess performance of a multiple testing procedure (MTP). These measures include power, overall error rate (family‐wise error rate), and lack of power. In settings where the MTP is used to estimate a parameter, for example, the minimum effective dose, bias is of interest. In some studies, the parameter has a set‐like structure, and thus, bias is not well defined. Nevertheless, the accuracy of estimation is one of the essential features of an MTP in such a context. In this paper, we propose several measures based on the expected values of loss functions that resemble bias. These measures are constructed to be useful in combination drug dose response studies when the target is to identify all minimum efficacious drug combinations. One of the proposed measures allows for assigning different penalties for incorrectly overestimating and underestimating a true minimum efficacious combination. Several simple examples are considered to illustrate the proposed loss functions. Then, the expected values of these loss functions are used in a simulation study to identify the best procedure among several methods used to select the minimum efficacious combinations, where the measures take into account the investigator's preferences about possibly overestimating and/or underestimating a true minimum efficacious combination. The ideas presented in this paper can be generalized to construct measures that resemble bias in other settings. These measures can serve as an essential tool to assess performance of several methods for identifying set‐like parameters in terms of accuracy of estimation. Copyright © 2012 John Wiley & Sons, Ltd.     

On estimation of the scale matrix of the multivariate f distribution

Let F _p×phave a multivariate F distribution with a scale p×p matrix Δ and degrees of freedom k₁ and k₂ such that k_i - p - 1 > 0, i = 1,2. The estimation of Δ under entropy and squared error loss functions are considered. In both cases a new class of orthogonally invariant estimators are obtained which dominate the best unbiased estimator.     

Small sample properties of the logit maximum likelihood estimator:a simulation study

This paper explores the effect of sample size, scale of parameters and size of the choice set on the maximum likelihood estimator of the parameters of the multinomial logit model. Data were generated by simulations under a three-way factorial experimental design for logit models containing three, four and five explanatory variables. Simulation data were analyzed by analysis of covariance and a regression model of the performance measure, the log root mean-squared error (LRMSE), fitted against the three factors and their interactions. Several important conclusions emerged. First, the LRMSE improves, but at a decreasing rate, with increases in the model's degrees of freedom. Second, the number of choice alternatives in the decision makers' choice sets has a significant impact on the LRMSE; however, heterogeneity in the choice sets across the sample has little or no impact. Finally, the scale of parameters and all of its two-way interactions with the other two factors significantly affect the LRMSE. Using the regression results, a family of iso-LRMSE curves are derived in the space of model degrees of freedom and scale of parameters. Their implications for researchers in choosing sample size and scale of parameters is discussed.