On a superiority problem in misspecified restricted linear models

Razzaghi (1987) conjectures that a wrong choice of covariance matrix in a restricted linear model results in loss of efficiency, This conjecture is proved to be correct.     

Linear latent structure analysis: Mixture distribution models with linear constraints

A new method for analyzing high-dimensional categorical data, Linear Latent Structure (LLS) analysis, is presented. LLS models belong to the family of latent structure models, which are mixture distribution models constrained to satisfy the local independence assumption. LLS analysis explicitly considers a family of mixed distributions as a linear space, and LLS models are obtained by imposing linear constraints on the mixing distribution.LLS models are identifiable under modest conditions and are consistently estimable. A remarkable feature of LLS analysis is the existence of a high-performance numerical algorithm, which reduces parameter estimation to a sequence of linear algebra problems. Simulation experiments with a prototype of the algorithm demonstrated a good quality of restoration of model parameters.     

Performance of the principal component two-parameter estimator in misspecified linear regression model

In this article, we consider the performance of the principal component two-parameter estimator in situation of multicollinearity for misspecified linear regression model where misspecification is due to omission of some relevant explanatory variables. The conditions of superiority of the principal component two-parameter estimator over some estimators under the Mahalanobis loss function by the average loss criterion are derived. Furthermore, a real data example and a Monte Carlo simulation study are provided to illustrate some of the theoretical results.     

Performance of tests of association in misspecified generalized linear models

We examine the effects of modelling errors, such as underfitting and overfitting, on the asymptotic power of tests of association between an explanatory variable x and an outcome in the setting of generalized linear models. The regression function for x is approximated by a polynomial or another simple function, and a chi-square statistic is used to test whether the coefficients of the approximation are simultaneously equal to zero. Adding terms to the approximation increases asymptotic power if and only if the fit of the model increases by a certain quantifiable amount. Although a high degree of freedom approximation offers robustness to the shape of the unknown regression function, a low degree of freedom approximation can yield much higher asymptotic power even when the approximation is very poor. In practice, it is useful to compute the power of competing test statistics across the range of alternatives that are plausible a priori. This approach is illustrated through an application in epidemiology.     

A Procedure for Identification of Principal Variables by Least Generalized Dependence

Principal components are often used for reducing dimensions in multivariate data, but they frequently fail to provide useful results and their interpretation is rather difficult. In this article, the use of entropy optimization principles for dimensional reduction in multivariate data is proposed. Under the assumptions of multivariate normality, a four-step procedure is developed for selecting principal variables and hence discarding redundant variables. For comparative performance of the information theoretic procedure, we use simulated data with known dimensionality. Principal variables of cluster bean (Guar) are identified by applying this procedure to a real data set generated in a plant breeding experiment.     

Robust prediction and extrapolation designs for misspecified generalized linear regression models

We study minimax robust designs for response prediction and extrapolation in biased linear regression models. We extend previous work of others by considering a nonlinear fitted regression response, by taking a rather general extrapolation space and, most significantly, by dropping all restrictions on the structure of the regressors. Several examples are discussed.     

Classical testing in functional linear models

We extend four tests common in classical regression – Wald, score, likelihood ratio and F tests – to functional linear regression, for testing the null hypothesis, that there is no association between a scalar response and a functional covariate. Using functional principal component analysis, we re-express the functional linear model as a standard linear model, where the effect of the functional covariate can be approximated by a finite linear combination of the functional principal component scores. In this setting, we consider application of the four traditional tests. The proposed testing procedures are investigated theoretically for densely observed functional covariates when the number of principal components diverges. Using the theoretical distribution of the tests under the alternative hypothesis, we develop a procedure for sample size calculation in the context of functional linear regression. The four tests are further compared numerically for both densely and sparsely observed noisy functional data in simulation experiments and using two real data applications.     

Cross-validation methods in principal component analysis: A comparison

In principal component analysis (PCA), it is crucial to know how many principal components (PCs) should be retained in order to account for most of the data variability. A class of “objective” rules for finding this quantity is the class of cross-validation (CV) methods. In this work we compare three CV techniques showing how the performance of these methods depends on the covariance matrix structure. Finally we propose a rule for the choice of the “best” CV method and give an application to real data.     

A cell means analysis of mixed linear models

The purpose of this paper is to describe a simple procedure for the estima-tion of parameters in the unbalanced mixed linear model. There are implications for hypothesis testing and interval estimation, A feature of these estimators is that they are expressed in terms of simple formulas. This has obvious advantages for computations and small sample analysis. In addition, the formulas suggest useful diagnostic procedures for assessing the quality of the data as well as possible defects in the model assumptions. The concepts are illustrated with several examples. Evidence is presented to indicate that, in cases of modest imbalance, these estimators are highly efficient and dominate AOV estimates over most of the parameter space. In cases of more extreme imbalance, the results are qualitatively the same but the estimators are less efficient than the AOV estimators for small values of the parameters. The extension of this method to factorial models with missing cells is not complete.     

A comparison of various methods for multivariate regression with highly collinear variables

Regression tends to give very unstable and unreliable regression weights when predictors are highly collinear. Several methods have been proposed to counter this problem. A subset of these do so by finding components that summarize the information in the predictors and the criterion variables. The present paper compares six such methods (two of which are almost completely new) to ordinary regression: Partial least Squares (PLS), Principal Component regression (PCR), Principle covariates regression, reduced rank regression, and two variants of what is called power regression. The comparison is mainly done by means of a series of simulation studies, in which data are constructed in various ways, with different degrees of collinearity and noise, and the methods are compared in terms of their capability of recovering the population regression weights, as well as their prediction quality for the complete population. It turns out that recovery of regression weights in situations with collinearity is often very poor by all methods, unless the regression weights lie in the subspace spanning the first few principal components of the predictor variables. In those cases, typically PLS and PCR give the best recoveries of regression weights. The picture is inconclusive, however, because, especially in the study with more real life like simulated data, PLS and PCR gave the poorest recoveries of regression weights in conditions with relatively low noise and collinearity. It seems that PLS and PCR are particularly indicated in cases with much collinearity, whereas in other cases it is better to use ordinary regression. As far as prediction is concerned: Prediction suffers far less from collinearity than recovery of the regression weights.     

Estimation on semi-functional linear errors-in-variables models

Abstract

Semi-functional linear regression models are important in practice. In this paper, their estimation is discussed when function-valued and real-valued random variables are all measured with additive error. By means of functional principal component analysis and kernel smoothing techniques, the estimators of the slope function and the non parametric component are obtained. To account for errors in variables, deconvolution is involved in the construction of a new class of kernel estimators. The convergence rates of the estimators of the unknown slope function and non parametric component are established under suitable norm and conditions. Simulation studies are conducted to illustrate the finite sample performance of our method.     

A Note on Unwanted Variance in Exploratory Factor Models

One strategy of exploratory factor analysis is to decide on the number of factors to extract by means of the eigenvalues of an initial principal component analysis. The present article proves that there is a non zero covariance of the factors with the components rejected when the number of factors to extract is determined by means of principal components analysis. Thus, some of the variance declared as irrelevant or unwanted in an initial principal component analysis is again part of the final factor model.     

Estimation in linear mixed models for longitudinal data under linear restricted conditions

In the paper, we consider a linear mixed model (LMM) for longitudinal data under linear restriction and find the estimators for the parameters of interest. The strong consistency and asymptotic normality of the estimators are obtained under some regularity conditions. Besides, we derive the strong consistent estimator of the fourth moment for the error which is useful for statistical inference for random effects and error variance. Simulations and an example are reported for illustration.     

Bayes estimates in multivariate semiparametric linear models

Bayes estimates are derived in multivariate linear models with unknown distribution. The prior distribution is defined using a Dirichlet prior for the unknown error distribution and a normal-Wishart distribution for the parameters. The posterior distribution is determined and explicit expressions are given in the special cases of location-scale and two-sample models. The calculation of self-informative limits of Bayes estimates yields standard estimates.     

Bounds on the covariate-time transformation for competing-risks survival analysis

A fundamental problem with the latent-time framework in competing risks is the lack of identifiability of the joint distribution. Given observed covariates along with assumptions as to the form of their effect, then identifiability may obtain. However it is difficult to check any assumptions about form since a more general model may lose identifiability. This paper considers a general framework for modelling the effect of covariates, with the single assumption that the copula dependency structure of the latent times is invariant to the covariates. This framework consists of a set of functions: the covariate-time transformations. The main result produces bounds on these functions, which are derived solely from the crude incidence functions. These bounds are a useful model checking tool when considering the covariate-time transformation resulting from any particular set of further assumptions. An example is given where the widely-used assumption of independent competing risks is checked.     

The Combined Model: A Tool for Simulating Correlated Counts with Overdispersion

The combined model as introduced by Molenberghs et al. (2007 Molenberghs, G., Verbeke, G., Demétrio, C. (2007). An extended random-effects approach to modeling repeated, overdispersed count data. Lifetime Data Analysis 13:513–531.[Crossref], [PubMed], [Web of Science ®] , [Google Scholar], 2010 Molenberghs, G., Verbeke, G., Demétrio, C., Vieira, A. (2010). A family of generalized linear models for repeated measures with normal and conjugate random effects. Statistical Science 25:325–347.[Crossref], [Web of Science ®] , [Google Scholar]) has been shown to be an appealing tool for modeling not only correlated or overdispersed data but also for data that exhibit both these features. Unlike techniques available in the literature prior to the combined model, which use a single random-effects vector to capture correlation and/or overdispersion, the combined model allows for the correlation and overdispersion features to be modeled by two sets of random effects. In the context of count data, for example, the combined model naturally reduces to the Poisson-normal model, an instance of the generalized linear mixed model in the absence of overdispersion and it also reduces to the negative-binomial model in the absence of correlation. Here, a Poisson model is specified as the parent distribution of the data conditional on a normally distributed random effect at the subject or cluster level and/or a gamma distribution at observation level. Importantly, the development of the combined model and surrounding derivations have relevance well beyond mere data analysis. It so happens that the combined model can also be used to simulate correlated data. If a researcher is interested in comparing marginal models via Monte Carlo simulations, a necessity to generate suitable correlated count data arises. One option is to induce correlation via random effects but calculation of such quantities as the bias is then not straightforward. Since overdispersion and correlation are simultaneous features of longitudinal count data, the combined model presents an appealing framework for generating data to evaluate statistical properties, through a pre-specification of the desired marginal mean (possibly in terms of the covariates and marginal parameters) and a marginal variance-covariance structure. By comparing the marginal mean and variance of the combined model to the desired or pre-specified marginal mean and variance, respectively, the implied hierarchical parameters and the variance-covariance matrices of the normal and Gamma random effects are then derived from which correlated Poisson data are generated. We explore data generation when a random intercept or random intercept and slope model is specified to induce correlation. The data generator, however, allows for any dimension of the random effects although an increase in the random-effects dimension increases the sensitivity of the derived random effects variance-covariance matrix to deviations from positive-definiteness. A simulation study is carried out for the random-intercept model and for the random intercept and slope model, with or without the normal and Gamma random effects. We also pay specific attention to the case of serial correlation.     

Admissible linear estimation in singular linear models with respect to a restricted parameter set

The admissibility results of Hoffmann (1977), proved in the context of a nonsingular covariance matrix are extended to the situation where the covariance matrix is singular. Admissible linear estimators in the Gauss-Markoff model are characterised and admissibility of the Best Linear Unbiased Estimator is investigated.     

A comparison of stein-like procedures for estimating linear regression models with multicollinear data

This paper compares several Stein-like estimation methods for estimating regression parameters. The criterion function was the mean-squared error of prediction and the parameter of interest was the mean of the response variable at the sampled values of the control variables. Large sample simulation techniques were used to evaluate the mean-squared error of the predictions. The parameters of interest were varied systematically over wide ranges.     

A Variable Selection Criterion for Two Sets of Principal Component Scores in Principal Canonical Correlation Analysis

Canonical correlation analysis (CCA) is often used to analyze the correlation between two random vectors. However, sometimes interpretation of CCA results may be hard. In an attempt to address these difficulties, principal canonical correlation analysis (PCCA) was proposed. PCCA is CCA between two sets of principal component (PC) scores. We consider the problem of selecting useful PC scores in CCA. A variable selection criterion for one set of PC scores has been proposed by Ogura (2010), here, we propose a variable selection criterion for two sets of PC scores in PCCA. Furthermore, we demonstrate the effectiveness of this criterion.     

The relative efficiency of the restricted estimators in linear regression models

This paper deals with the problem of multicollinearity in a multiple linear regression model with linear equality restrictions. The restricted two parameter estimator which was proposed in case of multicollinearity satisfies the restrictions. The performance of the restricted two parameter estimator over the restricted least squares (RLS) estimator and the ordinary least squares (OLS) estimator is examined under the mean square error (MSE) matrix criterion when the restrictions are correct and not correct. The necessary and sufficient conditions for the restricted ridge regression, restricted Liu and restricted shrunken estimators, which are the special cases of the restricted two parameter estimator, to have a smaller MSE matrix than the RLS and the OLS estimators are derived when the restrictions hold true and do not hold true. Theoretical results are illustrated with numerical examples based on Webster, Gunst and Mason data and Gorman and Toman data. We conduct a final demonstration of the performance of the estimators by running a Monte Carlo simulation which shows that when the variance of the error term and the correlation between the explanatory variables are large, the restricted two parameter estimator performs better than the RLS estimator and the OLS estimator under the configurations examined.