Local likelihood density estimation for interval censored data

The authors propose a class of procedures for local likelihood estimation from data that are either interval‐censored or that have been aggregated into bins. One such procedure relies on an algorithm that generalizes existing self‐consistency algorithms by introducing kernel smoothing at each step of the iteration. The entire class of procedures yields estimates that are obtained as solutions of fixed point equations. By discretizing and applying numerical integration, the authors use fixed point theory to study convergence of algorithms for the class. Rapid convergence is effected by the implementation of a local EM algorithm as a global Newton iteration. The latter requires an explicit solution of the local likelihood equations which can be found by using the symbolic Newton‐Raphson algorithm, if necessary.     

An algorithm for the estimation of poisson regressions involving structural change

We consider the estimation of Poisson regression models in which structural variation in a subset of the parameters is permitted. It is noted that coventional estimation algorithms are likely to impose restrictions on the number of explanatory variables and the number of structural regimes. We propose an alternative algorithm that implements partitioned matrix inversion and thereby avoids restictions on the size of the model. The algorithm is applied to a model of shopping behavior Adjustments in the algorithm necessary for dealing with censored data are detailed.     

概化理论两侧面设计方差分量及其变异量估计方法比较

文章针对正态分布数据,对比Traditional方法、Bootstrap方法和MCMC方法在两侧面交叉设计(p×i×h)和两侧面嵌套设计(p×(i:h))下各个方差分量的估计精度,为实际应用提供参考。使用R软件模拟1000批数据,并在R软件上实现三种方法的方差分量及其变异量估计。结果表明:(1)相较于Traditional方法和MCMC方法,相同条件下,Bootstrap方法估计的方差分量及其变异量结果更为理想;(2)对于两侧面交叉设计和两侧面嵌套设计,在正态分布数据下,建议优先使用Bootstrap方法。     

Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm

Two new implementations of the EM algorithm are proposed for maximum likelihood fitting of generalized linear mixed models. Both methods use random (independent and identically distributed) sampling to construct Monte Carlo approximations at the E-step. One approach involves generating random samples from the exact conditional distribution of the random effects (given the data) by rejection sampling, using the marginal distribution as a candidate. The second method uses a multivariate t importance sampling approximation. In many applications the two methods are complementary. Rejection sampling is more efficient when sample sizes are small, whereas importance sampling is better with larger sample sizes. Monte Carlo approximation using random samples allows the Monte Carlo error at each iteration to be assessed by using standard central limit theory combined with Taylor series methods. Specifically, we construct a sandwich variance estimate for the maximizer at each approximate E-step. This suggests a rule for automatically increasing the Monte Carlo sample size after iterations in which the true EM step is swamped by Monte Carlo error. In contrast, techniques for assessing Monte Carlo error have not been developed for use with alternative implementations of Monte Carlo EM algorithms utilizing Markov chain Monte Carlo E-step approximations. Three different data sets, including the infamous salamander data of McCullagh and Nelder, are used to illustrate the techniques and to compare them with the alternatives. The results show that the methods proposed can be considerably more efficient than those based on Markov chain Monte Carlo algorithms. However, the methods proposed may break down when the intractable integrals in the likelihood function are of high dimension.     

基于连续混合正态分布的期货定价偏差研究

挖掘期货理论价格和实际价格之间的关系有助于提高期货市场定价效率、发挥期货价格发现功能。基于持有成本定价模型计算期货定价偏差,利用连续混合正态分布模型对定价偏差的分布进行拟合,先采用基于牛顿迭代的极大似然估计法对未知参数进行估计,再进一步利用模拟退火算法对牛顿迭代的结果进行优化。结果发现,模拟退火算法可以有效提高估计精度,连续混合正态分布模型能够更好地拟合期货定价偏差分布。     

Nested coordinate descent algorithms for empirical likelihood

Empirical likelihood (EL) as a nonparametric approach has been demonstrated to have many desirable merits. While it has intensive development in methodological research, its practical application is less explored due to the requirements of intensive optimizations. Effective and stable algorithms therefore are highly desired for practical implementation of EL. This paper bears the effort to narrow the gap between methodological research and practical application of EL. We try to tackle the computation problems, which are considered difficult by practitioners, by introducing a nested coordinate descent algorithm and one modified version to EL. Coordinate descent as a class of convenient and robust algorithms has been shown in the existing literature to be effective in optimizations. We show that the nested coordinate descent algorithms can be conveniently and stably applied in general EL problems. The combination of nested coordinate descent with the MM algorithm further simplifies the computation. The nested coordinate descent algorithms are a natural and perfect match with inferences based on profile estimation and variable selection in high-dimensional data. Extensive examples are conducted to demonstrate the performance of the nested coordinate descent algorithms in the context of EL.     

Sparse matrices,and the estimation of variance components by likelihood methods

It is generally considered that analysis of variance by maximum likelihood or its variants is computationally impractical, despite existing techniques for reducing computational effect per iteration and for reducing the number of iterations to convergence. This paper shows thata major reduction in the overall computational effort can be achieved through the use of sparse-matrix algorithms that take advantage of the factorial designs that characterize most applications of large analysis-of-variance problems. In this paper, an algebraic structure for factorial designsis developed. Through this structure, it is shown that the required computations can be arranged so that sparse-matrix methods result in greatly reduced storage and time requirements.     

A fast algorithm for univariate log‐concave density estimation

A new fast algorithm for computing the nonparametric maximum likelihood estimate of a univariate log‐concave density is proposed and studied. It is an extension of the constrained Newton method for nonparametric mixture estimation. In each iteration, the newly extended algorithm includes, if necessary, new knots that are located via a special directional derivative function. The algorithm renews the changes of slope at all knots via a quadratically convergent method and removes the knots at which the changes of slope become zero. Theoretically, the characterisation of the nonparametric maximum likelihood estimate is studied and the algorithm is guaranteed to converge to the unique maximum likelihood estimate. Numerical studies show that it outperforms other algorithms that are available in the literature. Applications to some real‐world financial data are also given.     

A new REML (parameter expanded) EM algorithm for linear mixed models

Linear mixed models are regularly applied to animal and plant breeding data to evaluate genetic potential. Residual maximum likelihood (REML) is the preferred method for estimating variance parameters associated with this type of model. Typically an iterative algorithm is required for the estimation of variance parameters. Two algorithms which can be used for this purpose are the expectation‐maximisation (EM) algorithm and the parameter expanded EM (PX‐EM) algorithm. Both, particularly the EM algorithm, can be slow to converge when compared to a Newton‐Raphson type scheme such as the average information (AI) algorithm. The EM and PX‐EM algorithms require specification of the complete data, including the incomplete and missing data. We consider a new incomplete data specification based on a conditional derivation of REML. We illustrate the use of the resulting new algorithm through two examples: a sire model for lamb weight data and a balanced incomplete block soybean variety trial. In the cases where the AI algorithm failed, a REML PX‐EM based on the new incomplete data specification converged in 28% to 30% fewer iterations than the alternative REML PX‐EM specification. For the soybean example a REML EM algorithm using the new specification converged in fewer iterations than the current standard specification of a REML PX‐EM algorithm. The new specification integrates linear mixed models, Henderson's mixed model equations, REML and the REML EM algorithm into a cohesive framework.     

Robust Regression Using Data Partitioning and M-Estimation

We propose a new robust regression estimator using data partition technique and M estimation (DPM). The data partition technique is designed to define a small fixed number of subsets of the partitioned data set and to produce corresponding ordinary least square (OLS) fits in each subset, contrary to the resampling technique of existing robust estimators such as the least trimmed squares estimator. The proposed estimator shares a common strategy with the median ball algorithm estimator that is obtained from the OLS trial fits only on a fixed number of subsets of the data. We examine performance of the DPM estimator in the eleven challenging data sets and simulation studies. We also compare the DPM with the five commonly used robust estimators using empirical convergence rates relative to the OLS for clean data, robustness through mean squared error and bias, masking and swamping probabilities, the ability of detecting the known outliers, and the regression and affine equivariances.     

On boosting kernel density methods for multivariate data: density estimation and classification

Statistical learning is emerging as a promising field where a number of algorithms from machine learning are interpreted as statistical methods and vice-versa. Due to good practical performance, boosting is one of the most studied machine learning techniques. We propose algorithms for multivariate density estimation and classification. They are generated by using the traditional kernel techniques as weak learners in boosting algorithms. Our algorithms take the form of multistep estimators, whose first step is a standard kernel method. Some strategies for bandwidth selection are also discussed with regard both to the standard kernel density classification problem, and to our 'boosted' kernel methods. Extensive experiments, using real and simulated data, show an encouraging practical relevance of the findings. Standard kernel methods are often outperformed by the first boosting iterations and in correspondence of several bandwidth values. In addition, the practical effectiveness of our classification algorithm is confirmed by a comparative study on two real datasets, the competitors being trees including AdaBoosting with trees.     

A generalization of the Solis-Wets method

In this paper we focus on the application of global stochastic optimization methods to extremum estimators. We propose a general stochastic method—the master method—which includes several stochastic optimization algorithms as a particular case. The proposed method is sufficiently general to include the Solis-Wets method, the improving hit-and-run algorithm, and a stochastic version of the zigzag algorithm. A matrix formulation of the master method is presented and some specific results are given for the stochastic zigzag algorithm. Convergence of the proposed method is established under a mild set of conditions, and a simple regression model is used to illustrate the method.     

Use of vector divisions in solving quasi-likelihood equations for a Poisson model

Maximum-likelihood estimation technique is known to provide consistent and most efficient regression estimates but often this technique is tedious to implement, particularly in the modelling of correlated count responses. To overcome this limitation, researchers have developed semi- or quasi-likelihood functions that depend only on the correct specification of the mean and variance of the responses rather than on the distribution function. Moreover, quasi-likelihood estimation provides consistent and equally efficient estimates as the maximum-likelihood approach. Basically, the quasi-likelihood estimating function is a non-linear equation constituting of the gradient, Hessian and basic score matrices. Henceforth, to obtain estimates of the regression parameters, the quasi-likelihood equation is solved iteratively using the Newton–Raphson technique. However, the inverse of the Jacobian matrix involved in the Newton–Raphson method may not be easy to compute since the matrix is very close to singularity. In this paper, we consider the use of vector divisions in solving quasi-likelihood equations. The vector divisions are implemented to form secant method formulas. To assess the performance of the use of vector divisions with the secant method, we generate cross-sectional Poisson counts using different sets of mean parameters. We compute the estimates of the regression parameters using the Newton–Raphson technique and vector divisions and compare the number of non-convergent simulations under both algorithms.     

ML estimation for factor analysis: EM or non-EM?

To obtain maximum likelihood (ML) estimation in factor analysis (FA), we propose in this paper a novel and fast conditional maximization (CM) algorithm, which has quadratic and monotone convergence, consisting of a sequence of CM log-likelihood (CML) steps. The main contribution of this algorithm is that the closed form expression for the parameter to be updated in each step can be obtained explicitly, without resorting to any numerical optimization methods. In addition, a new ECME algorithm similar to Liu’s (Biometrika 81, 633–648, 1994) one is obtained as a by-product, which turns out to be very close to the simple iteration algorithm proposed by Lawley (Proc. R. Soc. Edinb. 60, 64–82, 1940) but our algorithm is guaranteed to increase log-likelihood at every iteration and hence to converge. Both algorithms inherit the simplicity and stability of EM but their convergence behaviors are much different as revealed in our extensive simulations: (1) In most situations, ECME and EM perform similarly; (2) CM outperforms EM and ECME substantially in all situations, no matter assessed by the CPU time or the number of iterations. Especially for the case close to the well known Heywood case, it accelerates EM by factors of around 100 or more. Also, CM is much more insensitive to the choice of starting values than EM and ECME.     

Partial central subspace and sliced average variance estimation

Sliced average variance estimation is one of many methods for estimating the central subspace. It was shown to be more comprehensive than sliced inverse regression in the sense that it consistently estimates the central subspace under mild conditions while slice inverse regression may estimate only a proper subset of the central subspace. In this paper we extend this method to regressions with qualitative predictors. We also provide tests of dimension and a marginal coordinate hypothesis test. We apply the method to a data set concerning lakes infested by Eurasian Watermilfoil, and compare this new method to the partial inverse regression estimator.     

Partially projected gradient algorithms for computing nonparametric maximum likelihood estimates of mixing distributions

There exist primarily three different types of algorithms for computing nonparametric maximum likelihood estimates (NPMLEs) of mixing distributions in the literature, which are the EM-type algorithms, the vertex direction algorithms such as VDM and VEM, and the algorithms based on general constrained optimization techniques such as the projected gradient method. It is known that the projected gradient algorithm may run into stagnation during iterations. When a stagnation occurs, VDM steps need to be added. We argue that the abrupt switch to VDM steps can significantly reduce the efficiency of the projected gradient algorithm, and is usually unnecessary. In this paper, we define a group of partially projected directions, which can be regarded as hybrids of ordinary projected gradient directions and VDM directions. Based on these directions, four new algorithms are proposed for computing NPMLEs of mixing distributions. The properties of the algorithms are discussed and their convergence is proved. Extensive numerical simulations show that the new algorithms outperform the existing methods, especially when a NPMLE has a large number of support points or when high accuracy is required.     

Asymptotic properties of maximum quasi-likelihood estimator in quasi-likelihood nonlinear models with misspecified variance function

The quasi-likelihood function proposed by Wedderburn [Quasi-likelihood functions, generalized linear models, and the Gauss–Newton method. Biometrika. 1974;61:439–447] broadened the application scope of generalized linear models (GLM) by specifying the mean and variance function instead of the entire distribution. However, in many situations, complete specification of variance function in the quasi-likelihood approach may not be realistic. Following Fahrmeir's [Maximum likelihood estimation in misspecified generalized linear models. Statistics. 1990;21:487–502] treating with misspecified GLM, we define a quasi-likelihood nonlinear models (QLNM) with misspecified variance function by replacing the unknown variance function with a known function. In this paper, we propose some mild regularity conditions, under which the existence and the asymptotic normality of the maximum quasi-likelihood estimator (MQLE) are obtained in QLNM with misspecified variance function. We suggest computing MQLE of unknown parameter in QLNM with misspecified variance function by the Gauss–Newton iteration procedure and show it to work well in a simulation study.     

Clustering objects on subsets of attributes (with discussion)

Summary.  A new procedure is proposed for clustering attribute value data. When used in conjunction with conventional distance-based clustering algorithms this procedure encourages those algorithms to detect automatically subgroups of objects that preferentially cluster on subsets of the attribute variables rather than on all of them simultaneously. The relevant attribute subsets for each individual cluster can be different and partially (or completely) overlap with those of other clusters. Enhancements for increasing sensitivity for detecting especially low cardinality groups clustering on a small subset of variables are discussed. Applications in different domains, including gene expression arrays, are presented.     

Fitting semiparametric accelerated failure time models for nested case–control data

A nested case–control (NCC) study is an efficient cohort-sampling design in which a subset of controls are sampled from the risk set at each event time. Since covariate measurements are taken only for the sampled subjects, time and efforts of conducting a full scale cohort study can be saved. In this paper, we consider fitting a semiparametric accelerated failure time model to failure time data from a NCC study. We propose to employ an efficient induced smoothing procedure for rank-based estimating method for regression parameters estimation. For variance estimation, we propose to use an efficient resampling method that utilizes the robust sandwich form. We extend our proposed methods to a generalized NCC study that allows a sampling of cases. Finite sample properties of the proposed estimators are investigated via an extensive stimulation study. An application to a tumor study illustrates the utility of the proposed method in routine data analysis.     

Maximum likelihood estimation of the logarithmic series distribution

This paper discusses the maximum likelihood estimation of the parameter of the logarithmic series distribution. The univariate case is treated in Part I, the multivariate case in Part II. A simple numerical estimation procedure is suggested using a fixed point approach. Convergence to the maximum likelihood estimator is shown. In Part III convergence rate is proven to be linear which is also demonstrated through example. In addition, comparisons with Newton’s method and the secant method in the univariate case, and with Newton’s method and the projected gradient method in the multivariate case are provided.