AN EVALUATION OF NON-ITERATIVE METHODS FOR ESTIMATING THE LINEAR-BY-LINEAR PARAMETER OF ORDINAL LOG-LINEAR MODELS

Parameter estimation for association and log-linear models is an important aspect of the analysis of cross-classified categorical data. Classically, iterative procedures, including Newton's method and iterative scaling, have typically been used to calculate the maximum likelihood estimates of these parameters. An important special case occurs when the categorical variables are ordinal and this has received a considerable amount of attention for more than 20 years. This is because models for such cases involve the estimation of a parameter that quantifies the linear-by-linear association and is directly linked with the natural logarithm of the common odds ratio. The past five years has seen the development of non-iterative procedures for estimating the linear-by-linear parameter for ordinal log-linear models. Such procedures have been shown to lead to numerically equivalent estimates when compared with iterative, maximum likelihood estimates. Such procedures also enable the researcher to avoid some of the computational difficulties that commonly arise with iterative algorithms. This paper investigates and evaluates the performance of three non-iterative procedures for estimating this parameter by considering 14 contingency tables that have appeared in the statistical and allied literature. The estimation of the standard error of the association parameter is also considered.     

Sampling from the posterior distribution in generalized linear mixed models

Generalized linear mixed models provide a unified framework for treatment of exponential family regression models, overdispersed data and longitudinal studies. These problems typically involve the presence of random effects and this paper presents a new methodology for making Bayesian inference about them. The approach is simulation-based and involves the use of Markov chain Monte Carlo techniques. The usual iterative weighted least squares algorithm is extended to include a sampling step based on the Metropolis–Hastings algorithm thus providing a unified iterative scheme. Non-normal prior distributions for the regression coefficients and for the random effects distribution are considered. Random effect structures with nesting required by longitudinal studies are also considered. Particular interests concern the significance of regression coefficients and assessment of the form of the random effects. Extensions to unknown scale parameters, unknown link functions, survival and frailty models are outlined.     

An approximate maximum likelihood procedure for parameter estimation in multivariate discrete data regression models

This paper considers an alternative to iterative procedures used to calculate maximum likelihood estimates of regression coefficients in a general class of discrete data regression models. These models can include both marginal and conditional models and also local regression models. The classical estimation procedure is generally via a Fisher-scoring algorithm and can be computationally intensive for high-dimensional problems. The alternative method proposed here is non-iterative and is likely to be more efficient in high-dimensional problems. The method is demonstrated on two different classes of regression models.     

Maximum Likelihood Estimation in Gaussian Chain Graph Models under the Alternative Markov Property

Abstract.  The Andersson–Madigan–Perlman (AMP) Markov property is a recently proposed alternative Markov property (AMP) for chain graphs. In the case of continuous variables with a joint multivariate Gaussian distribution, it is the AMP rather than the earlier introduced Lauritzen–Wermuth–Frydenberg Markov property that is coherent with data-generation by natural block-recursive regressions. In this paper, we show that maximum likelihood estimates in Gaussian AMP chain graph models can be obtained by combining generalized least squares and iterative proportional fitting to an iterative algorithm. In an appendix, we give useful convergence results for iterative partial maximization algorithms that apply in particular to the described algorithm.     

Multiple Time Series Modeling and Extended Sample Cross-Correlations

This article provides an expository account of the multivariate autoregressive moving average models and proposes an extended sample cross-correlation approach for practical model identification. An iterative model building procedure for applying these models to real data is discussed and demonstrated by analyzing the 5-series U.S. Hog Data.     

Nonconcave penalized estimation for partially linear models with longitudinal data

A nonconcave penalized estimation method is proposed for partially linear models with longitudinal data when the number of parameters diverges with the sample size. The proposed procedure can simultaneously estimate the parameters and select the important variables. Under some regularity conditions, the rate of convergence and asymptotic normality of the resulting estimators are established. In addition, an iterative algorithm is proposed to implement the proposed estimators. To improve efficiency for regression coefficients, the estimation of the covariance function is integrated in the iterative algorithm. Simulation studies are carried out to demonstrate that the proposed method performs well, and a real data example is analysed to illustrate the proposed procedure.     

ARMA MODELS REALIZATION AND IMPULSE RESPONSES

A parallel is made between the role played by covariances in the determination of auto-regressive models and the role played by impulse responses in the determination of ARMA models.

Auto-regressive models are known to maximize the Burg-entropy under covariance constraints. Auto-regressive-moving-average models give the maximum of the Burg-entropy among processes sharing the same covariances and impulse responses up to a certain lag. Such models are constructed by iterative or algebraic methods under the different constraints.

A new recursive method of identification of the order of an ARMA model is also developed, based on the generalized reflection coefficients.     

Nonparametric bayesian inference for dichotomous response models

A Bayesian least squares approach is taken here to estimate certain parameters in generalized linear models for dichotomous response data. The method requires that only first and second moments of the probability distribution representing prior information be specified* Examples are presented to illustrate situations having direct estimates as well as those which require approximate or iterative solutions.     

Numerical techniques for estimating probabilities

In this article we consider the pioblem of finding the maximum likelihood estimate for a certain class of discrete sampling models. Methods are adapted from parts of convex optimization theory, includiug aspects of equivalence theory, duality theorv and iterative procedures Their application is illustrated through example.     

Unconditional pseudo-maximum likelihood and adaptive estimation in the presence of conditional heterogeneity of unknown form

We consider parametric non-linear regression models with additive innovations which are serially uncorrelated but not necessarily independent, and consider the consequences of maximum likelihood and related one-step iterative estimation when the innovations are treated as being iid from their unconditional density. We find that the estimators' asymptotic covariance matrices will generally differ from those that would obtain if the errors actually were iid, except for the special case of strictly exogenous regressors. One important application of these results is to analysis of the properties of adaptive estimators, which employ nonparametric kernel estimates of the unconditional density of the disturbances in the construction of one-step iterative estimators. In the presence of strictly exogenous regressors, adaptive estimators are found to be asymptotically equivalent to the one-step iterative estimators that use the correct unconditional density. We illustrate our results through a brief Monte Carlo study.     

Generalized Weibull Linear Models

For the first time, a new class of generalized Weibull linear models is introduced to be competitive to the well-known generalized (gamma and inverse Gaussian) linear models which are adequate for the analysis of positive continuous data. The proposed models have a constant coefficient of variation for all observations similar to the gamma models and may be suitable for a wide range of practical applications in various fields such as biology, medicine, engineering, and economics, among others. We derive a joint iterative algorithm for estimating the mean and dispersion parameters. We obtain closed form expressions in matrix notation for the second-order biases of the maximum likelihood estimates of the model parameters and define bias corrected estimates. The corrected estimates are easily obtained as vectors of regression coefficients in suitable weighted linear regressions. The practical use of the new class of models is illustrated in one application to a lung cancer data set.     

Bayesian model averaging in longitudinal regression models with AR(1) errors with application to a myopia data set

We propose a new iterative algorithm, called model walking algorithm, to the Bayesian model averaging method on the longitudinal regression models with AR(1) random errors within subjects. The Markov chain Monte Carlo method together with the model walking algorithm are employed. The proposed method is successfully applied to predict the progression rates on a myopia intervention trial in children.     

A method for parameter estimation of non-linear regression with autocorrelated errors

For a class of non-linear models with stationary dependent residuals an estimating procedure is introduced and its statistical properties are derived. This procedure is useful when no basis exists for assuming a specific parametric model for the error process. For application of the procedure a two step iterative method is described and a small simulation study is performed.     

Diagnostics for glms with linear inequality parameter constraints

This paper proposes an iterative process, that can be implemented using GLIM, for fitting generalized linear models with linear inequality parameter constraints, when the maximum likelihood estimates exist and are unique. A one-step estimate is also introduced and some diagnostic measures are obtained. Finally an example is given for illustration.     

Ridge estimation in generalized linear models and proportional hazards regressions

Conventionally, a ridge parameter is estimated as a function of regression parameters based on ordinary least squares. In this article, we proposed an iterative procedure instead of the one-step or conventional ridge method. Additionally, we construct an indicator that measures the potential degree of improvement in mean squared error when ridge estimates are employed. Simulations show that our methods are appropriate for a wide class of non linear models including generalized linear models and proportional hazards (PHs) regressions. The method is applied to a PH regression with highly collinear covariates in a cancer recurrence study.     

Statistical Inference in Context Specific Interaction Models for Contingency Tables

Abstract.  Context specific interaction models is a class of interaction models for contingency tables in which interaction terms are allowed to vanish in specific contexts given by the levels of sets of variables. Such restrictions can entail conditional independencies which only hold for some values of the conditioning variables and allows also for irrelevance of some variables in specific contexts. A Markov property is established and so is an iterative proportional scaling algorithm for maximum likelihood estimation. Decomposition of the estimation problem is treated and model selection is discussed.     

Applying classification algorithms in practice

In this paper we present a perspective on the overall process of developing classifiers for real-world classification problems. Specifically, we identify, categorize and discuss the various problem-specific factors that influence the development process. Illustrative examples are provided to demonstrate the iterative nature of the process of applying classification algorithms in practice. In addition, we present a case study of a large scale classification application using the process framework described, providing an end-to-end example of the iterative nature of the application process. The paper concludes that the process of developing classification applications for operational use involves many factors not normally considered in the typical discussion of classification models and algorithms.     

Least absolute deviations estimation via the EM algorithm

This paper derives EM and generalized EM (GEM) algorithms for calculating least absolute deviations (LAD) estimates of the parameters of linear and nonlinear regression models. It shows that Schlossmacher's iterative reweighted least squares algorithm for calculating LAD estimates (E.J. Schlossmacher, Journal of the American Statistical Association 68: 857–859, 1973) is an EM algorithm. A GEM algorithm for computing LAD estimates of the parameters of nonlinear regression models is also provided and is applied in some examples.     

Numerical evaluation of observed sojourn time distributions for a single ion channel incorporating time interval omission

The dynamical aspects of single ion channel gating can be modelled by a semi-Markov process. There is aggregation of states, corresponding to the receptor channel being open or closed, and there is time interval omission, brief sojourns in either the open or closed classes of states not being detected. This paper is concerned with the computation of the probability density functions of observed open (closed) sojourn-times incorporating time interval omission. A system of Volterra integral equations is derived, whose solution governs the required density function. Numerical procedures, using iterative and multistep methods, are described for solving these equations. Examples are given, and in the special case of Markov models results are compared with those obtained by alternative methods. Probabilistic interpretations are given for the iterative methods, which also give lower bounds for the solutions.     

Tail Functions and Iterative Weights in Binary Regression

In interpreting the binary regression models often used in the analysis of dose-response data, it is common to introduce the idea of an underlying continuous tolerance distribution. Different choices of link function lead to different tolerance distributions. A useful way of comparing these alternatives is to compare the hazard functions or tail functions associated with each tolerance distribution. Tail functions can also be applied to give numerically preferable formulas for the iterative weights and the adjusted dependent variable in the fitting of binary regression models by the iteratively reweighted least-squares algorithm.