A Coefficient of Determination for Generalized Linear Models

The coefficient of determination, a.k.a. R², is well-defined in linear regression models, and measures the proportion of variation in the dependent variable explained by the predictors included in the model. To extend it for generalized linear models, we use the variance function to define the total variation of the dependent variable, as well as the remaining variation of the dependent variable after modeling the predictive effects of the independent variables. Unlike other definitions that demand complete specification of the likelihood function, our definition of R² only needs to know the mean and variance functions, so applicable to more general quasi-models. It is consistent with the classical measure of uncertainty using variance, and reduces to the classical definition of the coefficient of determination when linear regression models are considered.     

Improved estimators of common variance of <Emphasis Type="Italic">p</Emphasis>-populations when Kurtosis is known

The pooled variance of p samples presumed to have been obtained from p populations having common variance σ², has invariably been adopted as the default estimator for σ². In this paper, alternative estimators of the common population variance are developed. These estimators are biased and have lower mean-squared error values than . The comparative merit of these estimators over the unbiased estimator is explored using relative efficiency (a ratio of mean-squared error values).     

Collinearity: revisiting the variance inflation factor in ridge regression

Ridge regression has been widely applied to estimate under collinearity by defining a class of estimators that are dependent on the parameter k. The variance inflation factor (VIF) is applied to detect the presence of collinearity and also as an objective method to obtain the value of k in ridge regression. Contrarily to the definition of the VIF, the expressions traditionally applied in ridge regression do not necessarily lead to values of VIFs equal to or greater than 1. This work presents an alternative expression to calculate the VIF in ridge regression that satisfies the aforementioned condition and also presents other interesting properties.     

Estimation of the variance when kurtosis is known

The unbiased estimator of a population variance σ², S ² has traditionally been overemphasized, regardless of sample size. In this paper, alternative estimators of population variance are developed. These estimators are biased and have the minimum possible mean-squared error [and we define them as the “minimum mean-squared error biased estimators” (MBBE)]. The comparative merit of these estimators over the unbiased estimator is explored using relative efficiency (RE) (a ratio of mean-squared error values). It is found that, across all population distributions investigated, the RE of the MBBE is much higher for small samples and progressively diminishes to 1 with increasing sample size. The paper gives two applications involving the normal and exponential distributions.     

Improvement over variance estimation using auxiliary information in sample surveys

This paper addresses the problem of estimating the population variance S²_y of the study variable y using auxiliary information in sample surveys. We have suggested a class of estimators of the population variance S²_y of the study variable y when the population variance S²_x of the auxiliary variable x is known. Asymptotic expressions of bias and mean squared error (MSE) of the proposed class of estimators have been obtained. Asymptotic optimum estimators in the proposed class of estimators have also been identified along with its MSE formula. A comparison has been provided. We have further provided the double sampling version of the proposed class of estimators. The properties of the double sampling version have been provided under large sample approximation. In addition, we support the present study with aid of a numerical illustration.     

A Note on Asymptotic Behavior of the Nonparametric Density Estimators in Multivariate Mixtures

The asymptotic behavior of the nonparametric density estimator has been given for a multivariate mixture model. It has been observed that the estimator is asymptotically normally distributed with bias of size h ² and variance of size (nh)^?1.     

The double sampling S2 chart with estimated process variance

This paper proposes useful exact bounds for the parameters of the double sampling S² chart with known process variance and it also investigates the properties of the double sampling S² chart with estimated process variance, in terms of the average run length, the standard deviation of the run length and the average sample size, providing a numerical comparison with the known process variance case. It also provides guidelines to systematically design the double sampling S² chart both with known and estimated process variance and proposes two optimal design procedures with estimated process variance, for (a) minimizing the out-of-control average run length and (b) minimizing the out-of-control average sample size.     

Finite mixtures of natural exponential families

Let μ be a positive measure concentrated on R⁺ generating a natural exponential family (NEF) F with quadratic variance function V_F(m), m being the mean parameter of F. It is shown that v(dx) = (γ+x)μ(γ ≥ 0) (γ ≥ 0) generates a NEF G whose variance function is of the form l(m)Δ+cΔ(m), where l(m) is an affine function of m, Δ(m) is a polynomial in m (the mean of G) of degree 2, and c is a constant. The family G turns out to be a finite mixture of F and its length-biased family. We also examine the cases when F has cubic variance function and show that for suitable choices of γ the family G has variance function of the form P(m) + Q(m)m where P, Q are polynomials in m of degree m2 while Δ is an affine function of m. Finally we extend the idea to two dimensions by considering a bivariate Poisson and bivariate gamma mixture distribution.     

Variance Components Testing in ANOVA‐Type Mixed Models

The purpose of this article is threefold. First, variance components testing for ANOVA ‐type mixed models is considered, in which response may not be divided into independent sub‐vectors, whereas most of existing methods are for models where response can be divided into independent sub‐vectors. Second, testing that a certain subset of variance components is zero. Third, as normality is often violated in practice, it is desirable to construct tests under very mild assumptions. To achieve these goals, an adaptive difference‐based test and an adaptive trace‐based test are constructed. The test statistics are asymptotically normal under the null hypothesis, are consistent against all global alternatives and can detect local alternatives distinct from the null at a rate as close to n ^{? 1 ∕ 2} as possible with n being the sample size. Moreover, when the dimensions of variance components in different sets are bounded, we develop a test with chi‐square as its limiting null distribution. The finite sample performance of the tests is examined via simulations, and a real data set is analysed for illustration.     

Variability explained by covariates in linear mixed‐effect models for longitudinal data

Variability explained by covariates or explained variance is a well‐known concept in assessing the importance of covariates for dependent outcomes. In this paper we study R² statistics of explained variance pertinent to longitudinal data under linear mixed‐effect models, where the R² statistics are computed at two different levels to measure, respectively, within‐ and between‐subject variabilities explained by the covariates. By deriving the limits of R² statistics, we find that the interpretation of explained variance for the existing R² statistics is clear only in the case where the covariance matrix of the outcome vector is compound symmetric. Two new R² statistics are proposed to address the effect of time‐dependent covariate means. In the general case where the outcome covariance matrix is not compound symmetric, we introduce the concept of compound symmetry projection and use it to define level‐one and level‐two R² statistics. Numerical results are provided to support the theoretical findings and demonstrate the performance of the R² statistics. The Canadian Journal of Statistics 38: 352–368; 2010 © 2010 Statistical Society of Canada     

EMPIRICAL LIKELIHOOD-BASED KERNEL DENSITY ESTIMATION

This paper considers the estimation of a probability density function when extra distributional information is available (e.g. the mean of the distribution is known or the variance is a known function of the mean). The standard kernel method cannot exploit such extra information systematically as it uses an equal probability weight n^-1 at each data point. The paper suggests using empirical likelihood to choose the probability weights under constraints formulated from the extra distributional information. An empirical likelihood-based kernel density estimator is given by replacing n^-1 by the empirical likelihood weights, and has these advantages: it makes systematic use of the extra information, it is able to reflect the extra characteristics of the density function, and its variance is smaller than that of the standard kernel density estimator.     

A Class of Estimators Using Auxiliary Information for Estimating Finite Population Variance in Presence of Measurement Errors

This article addresses the problem of estimating the population variance using auxiliary information in the presence of measurement errors. When the measurement error variance associated with study variable is known, a class of estimators of the population variance using auxiliary information has been proposed. We obtain the bias and mean squared errors of the suggested class of estimators upto the terms of order n ^?1, and also optimum estimators in asymptotic sense of the class with approximate mean squared error formula.     

Sensitivity analysis in maximum likelihood factor analysis

The present paper deals with sensitivity analysis in maximum likelihood factor analysis. To investigate the influence of a small change of data we derive theoretical influence functions I(x; LL^T ) and I(x; Δ) for a common variance matrix T= LL^T and a unique variance matrix Δ respectively. Numerical examples are shown to illustrate our procedure.     

Asymptotic Properties of an Estimate of the Prediction Error Variance

An estimate of the variance of the prediction error for a linear stationary time series is constructed by using only a part, ?₀, ?₁,…, ?_M, of the sample autocovariance sequence, where M increases with T, the length of the time series observed. M increases as T^α (α < ¹₄). Under a higher moment condition, but not assuming Gaussianity, the estimate obtained is shown to be strongly consistent and asymptotically normal. The asymptotic variance of the estimation error is also obtained.     

Tests for detecting overdispersion in poisson models

Collings and Margolin(1985) developed a locally most powerful unbiased test for detecting negative binomial departures from a Poisson model, when the variance is a quadratic function of the mean. Kim and Park(1992) developed a locally most powerful unbiased test, when the variance is a linear function of the mean. It is found that a different mean-variance structure of a negative binomial derives a different locally optimal test statistic.

In this paper Collings and Margolin's and Kim and Park's results are unified and extended by developing a test for overdispersion in Poisson model against Katz family of distributions, Our setup has two extensions: First, Katz family of distributions is employed as an extension of the negative binomial distribution. Second, the mean-variance structure of the mixed Poisson model is given by σ² = μ+cμ^r for arbitrary but fixed r. We derive a local score test for testing H₀ : c = 0. Superiority of a new test is proved by the asymtotic relative efficiency as well as the simulation study.     

Shortest confidence and prediction intervals for the log-normal

We provide the shortest prediction interval for X, and the shortest confidence interval for the median of X, when X has the log-normal distribution for both the case σ², the variance of log X, known and unknown. Tables are given to assist the practitioner in constructing these intervals. A real-life example is provided to illustrate the results.     

Corrected asymptotic distribution of statistics based on the multinomial law

Investigators and epidemiologists often use statistics based on the parameters of a multinomial distribution. Two main approaches have been developed to assess the inferences of these statistics. The first one uses asymptotic formulae which are valid for large sample sizes. The second one computes the exact distribution, which performs quite well for small samples. They present some limitations for sample sizes N neither large enough to satisfy the assumption of asymptotic normality nor small enough to allow us to generate the exact distribution. We analytically computed the 1/N corrections of the asymptotic distribution for any statistics based on a multinomial law. We applied these results to the kappa statistic in 2×2 and 3×3 tables. We also compared the coverage probability obtained with the asymptotic and the corrected distributions under various hypothetical configurations of sample size and theoretical proportions. With this method, the estimate of the mean and the variance were highly improved as well as the 2.5 and the 97.5 percentiles of the distribution, allowing us to go down to sample sizes around 20, for data sets not too asymmetrical. The order of the difference between the exact and the corrected values was 1/N² for the mean and 1/N³ for the variance.     

Estimating a positive normal mean

Let X ₁, X ₂, ..., X _n be a random sample from a normal population with mean μ and variance σ ². In many real life situations, specially in lifetime or reliability estimation, the parameter μ is known a priori to lie in an interval [a, ∞). This makes the usual maximum likelihood estimator (MLE) ̄ an inadmissible estimator of μ with respect to the squared error loss. This is due to the fact that it may take values outside the parameter space. Katz (1961) and Gupta and Rohatgi (1980) proposed estimators which lie completely in the given interval. In this paper we derive some new estimators for μ and present a comparative study of the risk performance of these estimators. Both the known and unknown variance cases have been explored. The new estimators are shown to have superior risk performance over the existing ones over large portions of the parameter space.     

Model Selection for Linear Mixed Models Using Predictive Criteria

Predictive criteria, including the adjusted squared multiple correlation coefficient, the adjusted concordance correlation coefficient, and the predictive error sum of squares, are available for model selection in the linear mixed model. These criteria all involve some sort of comparison of observed values and predicted values, adjusted for the complexity of the model. The predicted values can be conditional on the random effects or marginal, i.e., based on averages over the random effects. These criteria have not been investigated for model selection success.

We used simulations to investigate selection success rates for several versions of these predictive criteria as well as several versions of Akaike's information criterion and the Bayesian information criterion, and the pseudo F-test. The simulations involved the simple scenario of selection of a fixed parameter when the covariance structure is known.

Several variance–covariance structures were used. For compound symmetry structures, higher success rates for the predictive criteria were obtained when marginal rather than conditional predicted values were used. Information criteria had higher success rates when a certain term (normally left out in SAS MIXED computations) was included in the criteria. Various penalty functions were used in the information criteria, but these had little effect on success rates. The pseudo F-test performed as expected. For the autoregressive with random effects structure, the results were the same except that success rates were higher for the conditional version of the predictive error sum of squares.

Characteristics of the data, such as the covariance structure, parameter values, and sample size, greatly impacted performance of various model selection criteria. No one criterion was consistently better than the others.