Asymptotic theory for maximum likelihood estimates in reduced-rank multivariate generalized linear models

Reduced-rank regression is a dimensionality reduction method with many applications. The asymptotic theory for reduced rank estimators of parameter matrices in multivariate linear models has been studied extensively. In contrast, few theoretical results are available for reduced-rank multivariate generalized linear models. We develop M-estimation theory for concave criterion functions that are maximized over parameter spaces that are neither convex nor closed. These results are used to derive the consistency and asymptotic distribution of maximum likelihood estimators in reduced-rank multivariate generalized linear models, when the response and predictor vectors have a joint distribution. We illustrate our results in a real data classification problem with binary covariates.     

Linear latent structure analysis: Mixture distribution models with linear constraints

A new method for analyzing high-dimensional categorical data, Linear Latent Structure (LLS) analysis, is presented. LLS models belong to the family of latent structure models, which are mixture distribution models constrained to satisfy the local independence assumption. LLS analysis explicitly considers a family of mixed distributions as a linear space, and LLS models are obtained by imposing linear constraints on the mixing distribution.LLS models are identifiable under modest conditions and are consistently estimable. A remarkable feature of LLS analysis is the existence of a high-performance numerical algorithm, which reduces parameter estimation to a sequence of linear algebra problems. Simulation experiments with a prototype of the algorithm demonstrated a good quality of restoration of model parameters.     

Simultaneous inference based on rank statistics in linear models

A class of simultaneous tests based on the aligned rank transform (ART) statistics is proposed for linear functions of parameters in linear models. The asymptotic distributions are derived. The stability of the finite sample behaviour of the sampling distribution of the ART technique is studied by comparing the simulated upper quantiles of its sampling distribution with those of the multivariate t-distribution. Simulation also shows that the tests based on ART have excellent small sample properties and because of their robustness perform better than the methods based on the least-squares estimates.     

Functional linear discriminant analysis for irregularly sampled curves

We introduce a technique for extending the classical method of linear discriminant analysis (LDA) to data sets where the predictor variables are curves or functions. This procedure, which we call functional linear discriminant analysis ( FLDA ), is particularly useful when only fragments of the curves are observed. All the techniques associated with LDA can be extended for use with FLDA. In particular FLDA can be used to produce classifications on new (test) curves, give an estimate of the discriminant function between classes and provide a one- or two-dimensional pictorial representation of a set of curves. We also extend this procedure to provide generalizations of quadratic and regularized discriminant analysis.     

A study on discriminant analysis techniques applied to multivariate lognormal data

The purpose of this paper is to examine the multiple group (>2) discrimination problem in which the group sizes are unequal and the variables used in the classification are correlated with skewed distributions. Using statistical simulation based on data from a clinical study, we compare the performances, in terms of misclassification rates, of nine statistical discrimination methods. These methods are linear and quadratic discriminant analysis applied to untransformed data, rank transformed data, and inverse normal scores data, as well as fixed kernel discriminant analysis, variable kernel discriminant analysis, and variable kernel discriminant analysis applied to inverse normal scores data. It is found that the parametric methods with transformed data generally outperform the other methods, and the parametric methods applied to inverse normal scores usually outperform the parametric methods applied to rank transformed data. Although the kernel methods often have very biased estimates, the variable kernel method applied to inverse normal scores data provides considerable improvement in terms of total nonerror rate.     

A note on multivariate linear regression

In this note, a hypothesis test based on relevant statistical differences is proposed for multivariate linear regression models whose design matrix rank does not equal the number of regression variables. A statistical example is also provided to illustrate the proposed hypothesis test.     

On analysis of binary response data in longitudinal factorial studies

Binary data are commonly used as responses to assess the effects of independent variables in longitudinal factorial studies. Such effects can be assessed in terms of the rate difference (RD), the odds ratio (OR), or the rate ratio (RR). Traditionally, the logistic regression seems always a recommended method with statistical comparisons made in terms of the OR. Statistical inference in terms of the RD and RR can then be derived using the delta method. However, this approach is hard to realize when repeated measures occur. To obtain statistical inference in longitudinal factorial studies, the current article shows that the mixed-effects model for repeated measures, the logistic regression for repeated measures, the log-transformed regression for repeated measures, and the rank-based methods are all valid methods that lead to inference in terms of the RD, OR, and RR, respectively. Asymptotic linear relationships between the estimators of the regression coefficients of these models are derived when the weight (working covariance) matrix is an identity matrix. Conditions for the Wald-type tests to be asymptotically equivalent in these models are provided and powers were compared using simulation studies. A phase III clinical trial is used to illustrate the investigated methods with corresponding SAS® code supplied.     

Union-intersection rank tests for ordered alternatives in some simple linear models

The theory of locally most powerful rank tests and the union intersection principle are incorporated in the formulation of some distribution-free rank tests for ordered alternatives in some simple linear models (including the classical one-way layout as a special case). Some approximations for the null-hypothesis distributions of the test statistics ( for finite sample sizes) are considered, and the theory is supplemented by some numerical and simulation results.     

Bayes linear analysis for graphical models: The geometric approach to local computation and interpretive graphics

This paper concerns the geometric treatment of graphical models using Bayes linear methods. We introduce Bayes linear separation as a second order generalised conditional independence relation, and Bayes linear graphical models are constructed using this property. A system of interpretive and diagnostic shadings are given, which summarise the analysis over the associated moral graph. Principles of local computation are outlined for the graphical models, and an algorithm for implementing such computation over the junction tree is described. The approach is illustrated with two examples. The first concerns sales forecasting using a multivariate dynamic linear model. The second concerns inference for the error variance matrices of the model for sales, and illustrates the generality of our geometric approach by treating the matrices directly as random objects. The examples are implemented using a freely available set of object-oriented programming tools for Bayes linear local computation and graphical diagnostic display.     

Johansen cointegration rank tests under local alternative hypotheses when related drift or linear trend is not dependent upon sample size in VARs

Abstract

This paper discusses Johansen’s likelihood ratio tests to determine the cointegration rank under local alternative hypotheses in the vector autoregressive models (VARs) in which drift or linear trend related to the hypotheses is not dependent upon the sample size, and evaluates related asymptotic properties. We show that the test statistics diverge at the rate of the sample size even under one of local alternative hypotheses, owing to the existence of such a deterministic term. This implies that under our situations, the tests are far more powerful than those under the conventional local alternative hypotheses.     

On variable selection in generalized linear and related regression models

This paper is concerned with selection of explanatory variables in generalized linear models (GLM). The class of GLM's is quite large and contains e.g. the ordinary linear regression, the binary logistic regression, the probit model and Poisson regression with linear or log-linear parameter structure. We show that, through an approximation of the log likelihood and a certain data transformation, the variable selection problem in a GLM can be converted into variable selection in an ordinary (unweighted) linear regression model. As a consequence no specific computer software for variable selection in GLM's is needed. Instead, some suitable variable selection program for linear regression can be used. We also present a simulation study which shows that the log likelihood approximation is very good in many practical situations. Finally, we mention briefly possible extensions to regression models outside the class of GLM's.     

Small area estimation using reduced rank regression models

Abstract

Small area estimation techniques have got a lot of attention during the last decades due to their important applications in survey studies. Mixed linear models and reduced rank regression analysis are jointly used when considering small area estimation. Estimates of parameters are presented as well as prediction of random effects and unobserved area measurements.     

A comparison of the classical and the linear programming approaches to the classification problem in discriminant analysis

Several mathematical programming approaches to the classification problem in discriminant analysis have recently been introduced. This paper empirically compares these newly introduced classification techniques with Fisher's linear discriminant analysis (FLDA), quadratic discriminant analysis (QDA), logit analysis, and several rank-based procedures for a variety of symmetric and skewed distributions. The percent of correctly classified observations by each procedure in a holdout sample indicate that while under some experimental conditions the linear programming approaches compete well with the classical procedures, overall, however, their performance lags behind that of the classical procedures.     

Residuals for log-Burr XII regression models in survival analysis

In this paper, we compare three residuals to assess departures from the error assumptions as well as to detect outlying observations in log-Burr XII regression models with censored observations. These residuals can also be used for the log-logistic regression model, which is a special case of the log-Burr XII regression model. For different parameter settings, sample sizes and censoring percentages, various simulation studies are performed and the empirical distribution of each residual is displayed and compared with the standard normal distribution. These studies suggest that the residual analysis usually performed in normal linear regression models can be straightforwardly extended to the modified martingale-type residual in log-Burr XII regression models with censored data.     

Weighted Wilcoxon Estimators in Nonlinear Regression

In this paper, we consider the estimation of parameters of a general near regression model. An estimator that minimises the weighted Wilcoxon dispersion function is considered and its asymptotic properties established under mild regularity conditions similar to those used in least squares and least absolute deviations estimation. As in linear models, the procedure provides estimators that are robust and highly efficient. The estimates depend on the choice of a weight function and diagnostics which differentiate between nonlinear fits are provided along with appropriate benchmarks. The behavior of these estimates is discussed on a real data set. A simulation study verifies the robustness, efficiency and validity of these estimates over several error distributions including the normal and a family of contaminated normal distributions.     

Small-sample properties of covariance-adjusted survivorship data tests for treatment effect

Monte Carlo methods are used to examine the small-sample properties of 11 test statistics that can be used for comparing several treatments with respect to their mortality experiences while adjusting for covariables. The test statistics are investigated from three distinct models: the parametric, semiparametric and rank analysis of covariance (Quade, 1967) models. Four tests (likelihood ratio, Wald, conditional and unconditional score tests) from each of the first two models and three tests (based on rank scores) from the last model are discussed. The empirical size and power of the tests are investigated under a proportional hazards model in three situations: (1) the baseline hazard is correctly assumed to be Exponential, (2) the baseline hazard is incorrectly assumed to be Exponential, and (3) a treatment-covariate interaction is omitted from the analysis.     

A study of linear model alysis by an error contour emulation

This paper considers the conditional approach to linear models in which the exact theoretical results are unavailable except in terms of multiple integrals. A class of multidimensional error distributions that emulate elongated error distributions is discussed. The appropriate conditional distributions are derived along with several properties of these distributions.     

Labelled Graphical Models

A class of log‐linear models, referred to as labelled graphical models (LGMs), is introduced for multinomial distributions. These models generalize graphical models (GMs) by employing partial conditional independence restrictions which are valid only in subsets of an outcome space. Theoretical results concerning model identifiability, decomposability and estimation are derived. A decision theoretical framework and a search algorithm for the identification of plausible models are described. Real data sets are used to illustrate that LGMs may provide a simpler interpretation of a dependence structure than GMs.     

Rank estimation for the functional linear model

This article discusses the estimation of the parameter function for a functional linear regression model under heavy-tailed errors' distributions and in the presence of outliers. Standard approaches of reducing the high dimensionality, which is inherent in functional data, are considered. After reducing the functional model to a standard multiple linear regression model, a weighted rank-based procedure is carried out to estimate the regression parameters. A Monte Carlo simulation and a real-world example are used to show the performance of the proposed estimator and a comparison made with the least-squares and least absolute deviation estimators.     

Bayesian inference for bivariate generalized linear models in diagnosing renal arterial obstruction

Generalized linear models are well-established generalizations of the linear models used for regression and analysis of variance. They allow flexible mean structures and general distributions, other than the linear link and normal response assumed in regression. Further enhancements using ideas from multivariate analysis improve power and precision by modelling dependencies between response variables. This paper focuses on the specific case of regression models for bivariate Bernoulli responses and investigates their analysis using a Bayesian approach. The important problem of renal arterial obstruction is considered, as a medical application of these models.