A Coefficient of Determination for Generalized Linear Models

The coefficient of determination, a.k.a. R², is well-defined in linear regression models, and measures the proportion of variation in the dependent variable explained by the predictors included in the model. To extend it for generalized linear models, we use the variance function to define the total variation of the dependent variable, as well as the remaining variation of the dependent variable after modeling the predictive effects of the independent variables. Unlike other definitions that demand complete specification of the likelihood function, our definition of R² only needs to know the mean and variance functions, so applicable to more general quasi-models. It is consistent with the classical measure of uncertainty using variance, and reduces to the classical definition of the coefficient of determination when linear regression models are considered.     

Asymptotic Properties of an Estimate of the Prediction Error Variance

An estimate of the variance of the prediction error for a linear stationary time series is constructed by using only a part, ?₀, ?₁,…, ?_M, of the sample autocovariance sequence, where M increases with T, the length of the time series observed. M increases as T^α (α < ¹₄). Under a higher moment condition, but not assuming Gaussianity, the estimate obtained is shown to be strongly consistent and asymptotically normal. The asymptotic variance of the estimation error is also obtained.     

Partly linear models on Riemannian manifolds

In partly linear models, the dependence of the response y on (x ^T, t) is modeled through the relationship y=x ^T β+g(t)+?, where ? is independent of (x ^T, t). We are interested in developing an estimation procedure that allows us to combine the flexibility of the partly linear models, studied by several authors, but including some variables that belong to a non-Euclidean space. The motivating application of this paper deals with the explanation of the atmospheric SO₂ pollution incidents using these models when some of the predictive variables belong in a cylinder. In this paper, the estimators of β and g are constructed when the explanatory variables t take values on a Riemannian manifold and the asymptotic properties of the proposed estimators are obtained under suitable conditions. We illustrate the use of this estimation approach using an environmental data set and we explore the performance of the estimators through a simulation study.     

Marginally restricted sequential D‐optimal designs

In many experiments, not all explanatory variables can be controlled. When the units arise sequentially, different approaches may be used. The authors study a natural sequential procedure for “marginally restricted” D‐optimal designs. They assume that one set of explanatory variables (x₁) is observed sequentially, and that the experimenter responds by choosing an appropriate value of the explanatory variable x₂. In order to solve the sequential problem a priori, the authors consider the problem of constructing optimal designs with a prior marginal distribution for x₁. This eliminates the influence of units already observed on the next unit to be designed. They give explicit designs for various cases in which the mean response follows a linear regression model; they also consider a case study with a nonlinear logistic response. They find that the optimal strategy often consists of randomizing the assignment of the values of x₂.     

Control variates and importance sampling for efficient bootstrap simulations

Importance sampling and control variates have been used as variance reduction techniques for estimating bootstrap tail quantiles and moments, respectively. We adapt each method to apply to both quantiles and moments, and combine the methods to obtain variance reductions by factors from 4 to 30 in simulation examples.We use two innovations in control variates—interpreting control variates as a re-weighting method, and the implementation of control variates using the saddlepoint; the combination requires only the linear saddlepoint but applies to general statistics, and produces estimates with accuracy of order n ^-1/2 B ^-1, where n is the sample size and B is the bootstrap sample size.We discuss two modifications to classical importance sampling—a weighted average estimate and a mixture design distribution. These modifications make importance sampling robust and allow moments to be estimated from the same bootstrap simulation used to estimate quantiles.     

Biased estimation of a variance

Summary Let , whereX _i are i.i.d. random variables with a finite variance σ² and is the usual estimate of the mean ofX _i. We consider the problem of finding optimal α with respect to the minimization of the expected value of |S ²(σ)−σ²|^k for variousk and with respect to Pitman's nearness criterion. For the Gaussian case analytical results are obtained and for some non-Gaussian cases we present Monte Carlo results regarding Pitman's criteron. This research was supported by Science Fund of Serbia, grant number 04M03, through Mathematical Institute, Belgrade.     

The Relative Effectiveness of Procedures Commonly Used in Multiple Regression Analysis for Dealing with Missing Values

Expressions are derived for the bias and variance associated with procedures frequently used to estimate partial regression coefficients in a linear model having the two explanatory variables x ₁ and x ₂, with missing values on x ₂ only. The expressions are used to help gain insight into the relative effectiveness of these procedures for handling more complex patterns of missing data.     

Identifying Nonlinear Relationships in Regression using the ACE Algorithm

This paper introduces an alternating conditional expectation (ACE) algorithm: a non-parametric approach for estimating the transformations that lead to the maximal multiple correlation of a response and a set of independent variables in regression and correlation analysis. These transformations can give the data analyst insight into the relationships between these variables so that this can be best described and non-linear relationships uncovered. Using the Bayesian information criterion (BIC), we show how to find the best closed-form approximations for the optimal ACE transformations. By means of ACE and BIC, the model fit can be considerably improved compared with the conventional linear model as demonstrated in the two simulated and two real datasets in this paper.     

On the Optimal Use of Suboptimal Forecasts of Explanatory Variables

Ashley (1983) gave a simple condition for determining when a forecast of an explanatory variable (X_t ) is sufficiently inaccurate that direct replacement of X_t by the forecast yields worse forecasts of the dependent variable than does respecification of the equation to omit X_t . Many available macroeconomic forecasts were shown to be of limited usefulness in direct replacement. Direct replacement, however, is not optimal if the forecast's distribution is known. Here optimal linear forms in commercial forecasts of several macroeconomic variables are obtained by using estimates of their distributions. Although they are an improvement on the raw forecasts (direct replacement), these optimal forms are still too inaccurate to be useful in replacing the actual explanatory variables in forecasting models. The results strongly indicate that optimal forms involving several commercial forecasts will not be very useful either. Thus Ashley's (1983) sufficient condition retains its value in gauging the usefulness of a forecast of an explanatory variable in a forecasting model, even though it focuses on direct replacement.     

Variance reduction for quantile estimates in simulations via nonlinear controls

Linear controls are a well known simple technique for achieving variance reduction in computer simulation. Unfortunately the effectiveness of a linear control depends upon the correlation between the statistic of interest and the control, which is often low. Since statistics often have a nonlinear relation-ship with the potential control variables, nonlinear controls offer a means for improvement over linear controls. This paper focuses on the use of nonlinear controls for reducing the variance of quantile estimates in simulation. It is shown that one can substantially reduce the analytic effort required to develop a nonlinear control from a quantile estimator by using a strictly monotone transformation to create the nonlinear control. It is also shown that as one increases the sample size for the quantile estimator, the asymptotic multivariate normal distribution of the quantile of interest and the control reduces the effectiveness of the nonlinear control to that of the linear control. However, the data has to be sectioned to obtain an estimate of the variance of the controlled quantile estimate. Graphical methods are suggested for selecting the section size that maximizes the effectiveness of the nonlinear control     

Planning step-stress test plans under Type-I censoring for the log-location-scale case

In this paper, we consider a k-level step-stress accelerated life-testing (ALT) experiment with unequal duration steps τ=(τ₁, …, τ _k ). Censoring is allowed only at the change-stress point in the final stage. A general log-location-scale lifetime distribution with mean life which is a linear function of stress, along with a cumulative exposure model, is considered as the working model. Under this model, the determination of the optimal choice of τ for both Weibull and lognormal distributions are addressed using the variance–optimality criterion. Numerical results show that for a general log-location-scale distributions, the optimal k-step-stress ALT model with unequal duration steps reduces just to a 2-level step-stress ALT model.     

Improvement over variance estimation using auxiliary information in sample surveys

This paper addresses the problem of estimating the population variance S²_y of the study variable y using auxiliary information in sample surveys. We have suggested a class of estimators of the population variance S²_y of the study variable y when the population variance S²_x of the auxiliary variable x is known. Asymptotic expressions of bias and mean squared error (MSE) of the proposed class of estimators have been obtained. Asymptotic optimum estimators in the proposed class of estimators have also been identified along with its MSE formula. A comparison has been provided. We have further provided the double sampling version of the proposed class of estimators. The properties of the double sampling version have been provided under large sample approximation. In addition, we support the present study with aid of a numerical illustration.     

The Target Parameter of Adjusted R-Squared in Fixed-Design Experiments

R-squared (R²) and adjusted R-squared (R²_Adj) are sometimes viewed as statistics detached from any target parameter, and sometimes as estimators for the population multiple correlation. The latter interpretation is meaningful only if the explanatory variables are random. This article proposes an alternative perspective for the case where the x’s are fixed. A new parameter is defined, in a similar fashion to the construction of R², but relying on the true parameters rather than their estimates. (The parameter definition includes also the fixed x values.) This parameter is referred to as the “parametric” coefficient of determination, and denoted by ρ²_*. The proposed ρ²_* remains stable when irrelevant variables are removed (or added), unlike the unadjusted R², which always goes up when variables, either relevant or not, are added to the model (and goes down when they are removed). The value of the traditional R²_Adj may go up or down with added (or removed) variables, either relevant or not. It is shown that the unadjusted R² overestimates ρ²_*, while the traditional R²_Adj underestimates it. It is also shown that for simple linear regression the magnitude of the bias of R²_Adj can be as high as the bias of the unadjusted R² (while their signs are opposite). Asymptotic convergence in probability of R²_Adj to ρ²_* is demonstrated. The effects of model parameters on the bias of R² and R²_Adj are characterized analytically and numerically. An alternative bi-adjusted estimator is presented and evaluated.     

Tests for detecting overdispersion in poisson models

Collings and Margolin(1985) developed a locally most powerful unbiased test for detecting negative binomial departures from a Poisson model, when the variance is a quadratic function of the mean. Kim and Park(1992) developed a locally most powerful unbiased test, when the variance is a linear function of the mean. It is found that a different mean-variance structure of a negative binomial derives a different locally optimal test statistic.

In this paper Collings and Margolin's and Kim and Park's results are unified and extended by developing a test for overdispersion in Poisson model against Katz family of distributions, Our setup has two extensions: First, Katz family of distributions is employed as an extension of the negative binomial distribution. Second, the mean-variance structure of the mixed Poisson model is given by σ² = μ+cμ^r for arbitrary but fixed r. We derive a local score test for testing H₀ : c = 0. Superiority of a new test is proved by the asymtotic relative efficiency as well as the simulation study.     

Autoregressive Model with Partial Forgetting within Rao-Blackwellized Particle Filter

The authors are concerned with Bayesian identification and prediction of a nonlinear discrete stochastic process. The fact that a nonlinear process can be approximated by a piecewise linear function advocates the use of adaptive linear models. They propose a linear regression model within Rao-Blackwellized particle filter. The parameters of the linear model are adaptively estimated using a finite mixture, where the weights of components are tuned with a particle filter. The mixture reflects a priori given hypotheses on different scenarios of (expected) parameters' evolution. The resulting hybrid filter locally optimizes the weights to achieve the best fit of a nonlinear signal with a single linear model.     

Predictive performance of missing data methods for logistic regression,classification trees and neural networks

Although the effect of missing data on regression estimates has received considerable attention, their effect on predictive performance has been neglected. We studied the performance of three missing data strategies—omission of records with missing values, replacement with a mean and imputation based on regression—on the predictive performance of logistic regression (LR), classification tree (CT) and neural network (NN) models in the presence of data missing completely at random (MCAR). Models were constructed using datasets of size 500 simulated from a joint distribution of binary and continuous predictors including nonlinearities, collinearity and interactions between variables. Though omission produced models that fit better on the data from which the models were developed, imputation was superior on average to omission for all models when evaluating the receiver operating characteristic (ROC) curve area, mean squared error (MSE), pooled variance across outcome categories and calibration X ² on an independently generated test set. However, in about one-third of simulations, omission performed better. Performance was also more variable with omission including quite a few instances of extremely poor performance. Replacement and imputation generally produced similar results except with neural networks for which replacement, the strategy typically used in neural network algorithms, was inferior to imputation. Missing data affected simpler models much less than they did more complex models such as generalized additive models that focus on local structure For moderate sized datasets, logistic regressions that use simple nonlinear structures such as quadratic terms and piecewise linear splines appear to be at least as robust to randomly missing values as neural networks and classification trees.     

Testing the equality of correlation matrices

We present results that extend an existing test of equality of correlation matrices. A new test statistic is proposed and is shown to be asymptotically distributed as a linear combination of independent x ² random variables. This new formulation allows us to find the power of the existing test and our extensions by deriving the distribution under the alternative using a linear combination of independent non-central x ² random variables. We also investigate the null and the alternative distribution of two related statistics. The first one is a quadratic form in deviations from a control group with which the remaining k-1 groups are to be compared. The second test is designed for comparing adjacent groups. Several approximations for the null and the alternative distribution are considered and two illustrative examples are provided.     

Jackknifing in Categorical Data Analysis

Estimation of nonlinear functions of a multinomial parameter vector is necessary in many categorical data problems. The first and second order jackknife are explored for the purpose of reduction of bias. The second order jackknife of a function g(.) of a multinomial parameter is shown to be asymptotically normal if all second order partials ?²g( p )?dp_i?p_j obey a Hölder condition with exponent α>1/2. Numerical results for the estimation of the log odds ratio in a 2times2 table demonstrate the efficiency of the jackknife method for reduction of mean-square-error and the construction of approximate confidence intervals.     

Minimum Variance and VOQL Chain Sampling Plans–ChSP-4(c 1, c 2)

In this article the outgoing quality and the total inspection for the chain sampling plan ChSP-4(c ₁, c ₂) are introduced as well-defined random variables. The probability distributions of outgoing quality and total inspection are stated based on total rectification of non conforming units. The variances of these random variables are studied. The aim of this article is to develop procedures for minimum variance ChSP-4(c ₁, c ₂) sampling plans and their determination. In addition to minimum variance sampling plans, a procedure is developed for designing plans with a designated maximum variance, a VOQL (Variance of Outgoing Quality Limit) plan. The VOQL concept is analogous to the AOQL (Average Outgoing Quality Limit) except in the VOQL plan, it is the maximum variance which is established instead of the usual maximum AOQ.     

A Transformation Characterizing the Normal Distribution

In this article we propose a new transformation of random variables (RVs) which characterizes the normal distribution. It allows us to transform n i.i.d. normal RVs whose mean and variance are unknown into new n ? 2 i.i.d. new normal variables with zero mean while maintaining the same unknown variance. This belongs to the class of transformations designed to reduce the number of unknown parameters or remove them altogether.

Some historical remarks concerning methods for removing parameters in the normal distribution are given and two possible applications of the new transformation are described.