A general maximum likelihood analysis of overdispersion in generalized linear models

This paper presents an EM algorithm for maximum likelihood estimation in generalized linear models with overdispersion. The algorithm is initially derived as a form of Gaussian quadrature assuming a normal mixing distribution, but with only slight variation it can be used for a completely unknown mixing distribution, giving a straightforward method for the fully non-parametric ML estimation of this distribution. This is of value because the ML estimates of the GLM parameters may be sensitive to the specification of a parametric form for the mixing distribution. A listing of a GLIM4 algorithm for fitting the overdispersed binomial logit model is given in an appendix.A simple method is given for obtaining correct standard errors for parameter estimates when using the EM algorithm.Several examples are discussed.     

Constructing estimators of a mean vector through orthogonal reparametrization and differential equations

The simultaneous estimation of a mean vector is explored by reparametrizing it into its direction and norm components. A type of Pythagorean relation is employed to construct an estimate of the norm component, which results in solving an ordinary amerenuai equation, me james-Diem estimator is snown to be optimum in a class of estimators derived from general solutions of the ordinary differential equation. A new Stein-type estimator in the case of the inverse Gaussian distribution is constructed.     

A nonparametric mixture approach to density and null proportion estimation in large-scale multiple comparison problems

A new method for estimating the proportion of null effects is proposed for solving large-scale multiple comparison problems. It utilises maximum likelihood estimation of nonparametric mixtures, which also provides a density estimate of the test statistics. It overcomes the problem of the usual nonparametric maximum likelihood estimator that cannot produce a positive probability at the location of null effects in the process of estimating nonparametrically a mixing distribution. The profile likelihood is further used to help produce a range of null proportion values, corresponding to which the density estimates are all consistent. With a proper choice of a threshold function on the profile likelihood ratio, the upper endpoint of this range can be shown to be a consistent estimator of the null proportion. Numerical studies show that the proposed method has an apparently convergent trend in all cases studied and performs favourably when compared with existing methods in the literature.     

Proportion estimation in mixtures with covariates

Linear maps of a single unclassified observation are used to estimate the mixing proportion in a mixture of two populations with homogeneous variances in the presence of covariates. with complete knowledge of the parameters of the individual populations, the linear map for which the estimator is unbiased and has minimum variance amongst all similar estimators can be determined. Plug-in estimator based on independent training samples from the component populations can be constructed and is asymptotically equivalent to Cochran's classification statistic V* for covariate classification; see Memon and Okamoto (1970). Under normality assumptions, asymptotic expansion of the distribution of the plug-in estimator is available. In the absence of covariates, our estimator reduces to that suggested by Walker (1980) who has investigated the problem based on information on large unclassified samples from a mixture of two populations with heterogeneous variances. In contrast, distribution of Walker's estimator seems intractable in moderate sample sizes even with normality assumption.     

Wavelet designs for estimating nonparametric curves with heteroscedastic error

In this paper, we discuss the problem of constructing designs in order to maximize the accuracy of nonparametric curve estimation in the possible presence of heteroscedastic errors. Our approach is to exploit the flexibility of wavelet approximations to approximate the unknown response curve by its wavelet expansion thereby eliminating the mathematical difficulty associated with the unknown structure. It is expected that only finitely many parameters in the resulting wavelet response can be estimated by weighted least squares. The bias arising from this, compounds the natural variation of the estimates. Robust minimax designs and weights are then constructed to minimize mean-squared-error-based loss functions of the estimates. We find the periodic and symmetric properties of the Euclidean norm of the multiwavelet system useful in eliminating some of the mathematical difficulties involved. These properties lead us to restrict the search for robust minimax designs to a specific class of symmetric designs. We also construct minimum variance unbiased designs and weights which minimize the loss functions subject to a side condition of unbiasedness. We discuss an example from the nonparametric literature.     

On multivariate quantile regression analysis

This paper investigates the estimation of parameters in a multivariate quantile regression model when the investigator wants to evaluate the associated distribution function. It proposes a new directional quantile estimator with the following properties: (1) it applies to an arbitrary number of random variables; (2) it is equivalent to estimating the distribution function allowing for non-convex distribution contours; (3) it satisfies nice equivariance properties; (4) it has desirable statistical properties (i.e., consistency and asymptotic normality); and (5) its implementation involves a modest computational burden: our proposed estimator can be obtained by solving parametric linear programming problems. As such, this paper expands the range of applications of quantile estimation for multivariate regression models.     

On the heterogeneity of fecundability

We propose to use a general mixing distribution in modeling the heterogeneity of the fecundability of couples. We introduce a sequence of parameters called canonical moments, which is in one to one correspondence with the moments, to characterize the mixing distribution. By using the bootstrap method, we can estimate the standard errors of our estimates. Our method modifies the usual moment estimates so that the resulting mixing distribution is always supported on [0, 1]. Moreover, the downward bias of the moment estimate of the number of support points would be reduced. Our approach can be used for censored data. The application of our technique in finding the sterile subpopulation is also discussed. The theory is illustrated with several data examples and simulations.     

On the Bayesian analysis of the mixture of power function distribution using the complete and the censored sample

The power function distribution is often used to study the electrical component reliability. In this paper, we model a heterogeneous population using the two-component mixture of the power function distribution. A comprehensive simulation scheme including a large number of parameter points is followed to highlight the properties and behavior of the estimates in terms of sample size, censoring rate, parameters size and the proportion of the components of the mixture. The parameters of the power function mixture are estimated and compared using the Bayes estimates. A simulated mixture data with censored observations is generated by probabilistic mixing for the computational purposes. Elegant closed form expressions for the Bayes estimators and their variances are derived for the censored sample as well as for the complete sample. Some interesting comparison and properties of the estimates are observed and presented. The system of three non-linear equations, required to be solved iteratively for the computations of maximum likelihood (ML) estimates, is derived. The complete sample expressions for the ML estimates and for their variances are also given. The components of the information matrix are constructed as well. Uninformative as well as informative priors are assumed for the derivation of the Bayes estimators. A real-life mixture data example has also been discussed. The posterior predictive distribution with the informative Gamma prior is derived, and the equations required to find the lower and upper limits of the predictive intervals are constructed. The Bayes estimates are evaluated under the squared error loss function.     

Surrogate geometric programming estimates of restricted multinomial proportions

Maximum-likelihood estimation of multinomial proportions subject to linear inequality or equality constraints is considered. A demonstration that surrogate constraints yield a zero-degree-of-difficulty geometric programming problem is given. A general solution in terms of surrogate multipliers and the unconstrained estimates is exhibited. A computational algorithm is presented and used to solve an example problem pertaining to coal deliveries received by a power plant.     

Constrained nonparametric maximum-likelihood estimation for mixture models

A nonparametric mixture model specifies that observations arise from a mixture distribution, ∫ f(x, θ) dG(θ), where the mixing distribution G is completely unspecified. A number of algorithms have been developed to obtain unconstrained maximum-likelihood estimates of G, but none of these algorithms lead to estimates when functional constraints are present. In many cases, there is a natural interest in functional ?(G), such as the mean and variance, of the mixing distribution, and profile likelihoods and confidence intervals for ?(G) are desired. In this paper we develop a penalized generalization of the ISDM algorithm of Kalbfleisch and Lesperance (1992) that can be used to solve the problem of constrained estimation. We also discuss its use in various different applications. Convergence results and numerical examples are given for the generalized ISDM algorithm, and asymptotic results are developed for the likelihood-ratio test statistics in the multinomial case.     

Nonlinear Autoregression with Positive Innovations

Use of nonlinear models in analyzing time series data is becoming increasingly popular. This paper considers a broad class of nonlinear autoregressive models where the autoregressive part is additive and the terms are nonlinear functions of the past data. Also, the innovation distribution is supported on the non-negative reals and satisfies a tail regularity condition. The linear parameters of the autoregression are estimated using a linear programming recipe which yields much more accurate estimates than traditional methods such as conditional least squares. Limiting distribution of the linear programming estimators is obtained. Simulation studies validate the asymptotic results and reveal excellent small sample properties of the LPE estimator.     

The Sichel model and the mixing and truncation order

The analysis of word frequency count data can be very useful in authorship attribution problems. Zero-truncated generalized inverse Gaussian–Poisson mixture models are very helpful in the analysis of these kinds of data because their model-mixing density estimates can be used as estimates of the density of the word frequencies of the vocabulary. It is found that this model provides excellent fits for the word frequency counts of very long texts, where the truncated inverse Gaussian–Poisson special case fails because it does not allow for the large degree of over-dispersion in the data. The role played by the three parameters of this truncated GIG-Poisson model is also explored. Our second goal is to compare the fit of the truncated GIG-Poisson mixture model with the fit of the model that results from switching the order of the mixing and truncation stages. A heuristic interpretation of the mixing distribution estimates obtained under this alternative GIG-truncated Poisson mixture model is also provided.     

Designing of a mixed-chain sampling plan based on the process capability index with chain sampling as the attributes plan

In this article, we propose a new mixed chain sampling plan based on the process capability index C_pk, where the quality characteristic of interest having double specification limits and follows the normal distribution with unknown mean and variance. In the proposed mixed plan, the chain sampling inspection plan is used for the inspection of attribute quality characteristics. The advantages of this proposed mixed sampling plan are also discussed. Tables are constructed to determine the optimal parameters for practical applications by formulating the problem as a non linear programming in which the objective function to be minimized is the average sample number and the constraints are related to lot acceptance probabilities at acceptable quality level and limiting quality level under the operating characteristic curve. The practical application of the proposed mixed sampling plan is explained with an illustrative example. Comparison of the proposed sampling plan is also made with other existing sampling plans.     

Fourth Quarter Survey of the Economic Outlook

Written mainly for its pedagogical interest, this note deals with the computational formulas for the recursive updating of weighted least squares parameter estimates and the residual sum of squares in the general linear model under the assumption that the errors have a multivariate normal distribution. This approach simplifies considerably the derivations of Haslett (1985).     

Plug-in bandwidth choice for estimation of nonparametric part in partial linear regression models with strong mixing errors

Consider a regression model where the regression function is the sum of a linear and a nonparametric component. Assuming that the errors of the model follow a stationary strong mixing process with mean zero, the problem of bandwidth selection for a kernel estimator of the nonparametric component is addressed here. We obtain an asymptotic expression for an optimal band-width and we propose to use a plug-in methodology in order to estimate this bandwidth through preliminary estimates of the unknown quantities. Asymptotic optimality for the plug-in bandwidth is established.     

Correcting data for measurement error in generalized linear models

This paper discusses a general strategy for reducing measurement-error-induced bias in statistical models. It is assumed that the measurement error is unbiased with a known variance although no other distributional assumptions on the measurement-error are employed,

Using a preliminary fit of the model to the observed data, a transformation of the variable measured with error is estimated. The transformation is constructed so that the estimates obtained by refitting the model to the ‘corrected’ data have smaller bias,

Whereas the general strategy can be applied in a number of settings, this paper focuses on the problem of covariate measurement error in generalized linear models, Two estimators are derived and their effectiveness at reducing bias is demonstrated in a Monte Carlo study.     

Estimation and variable selection for generalised partially linear single-index models

In this paper, we study the problem of estimation and variable selection for generalised partially linear single-index models based on quasi-likelihood, extending existing studies on variable selection for partially linear single-index models to binary and count responses. To take into account the unit norm constraint of the index parameter, we use the ‘delete-one-component’ approach. The asymptotic normality of the estimates is demonstrated. Furthermore, the smoothly clipped absolute deviation penalty is added for variable selection of parameters both in the nonparametric part and the parametric part, and the oracle property of the variable selection procedure is shown. Finally, some simulation studies are carried out to illustrate the finite sample performance.     

Bias corrected estimates in multivariate student t regression models

The t distribution has proved to be a useful alternative to the normal distribution especially When robust estimation is desired. We consider the multivariate nonlinear Student-t regression model and show that the biased of the estimates of the regression coefficients can be computed from an auxiliary generalized linear regression. We give a formula for the biases of the estimates of the parameters in the scale matrix, which also can be computed by means of a generalized linear regression. We briefly discuss some important special cases and present simulation results which indicate that our bias-corrected estimates outperform the uncorrected ones in small samples.     

The Multiple Outlier Problem in Time Series Analysis

In this paper we consider the multiple outlier problem in time series analysis. The underlying undisturbed time series is assumed to be an autoregressive process. The location of the suspicious values is supposed to be known. We introduce conditional least squares estimators for the parameters. The estimates are shown to be strongly consistent. Using similar arguments as in the theory of linear models, we get a test statistic for the general linear hypothesis. Its asymptotic distribution is derived.     

An algorithm for inverting matrices whose elements are polynomials

A variety of statistical problems lead to the estimation of parameters in a linear model or solving a system of linear equations. The problem of estimating linear parameters can be solved by adopting well known matrix inversion techniques. In this paper a mathematical algorithm has been developed for inverting a special class of matrices.